Informacje o stanowisku

technologies-expected :

• Apache Kafka
• Apache Spark
• Cloudera Hadoop stack
• SQL
• PL/SQL
• Python
• Linux
• Bash
• ETL
• SFTP
• MQ
• APIs
• file shares
• Google Cloud Platform

• Scala

• Migration of an Oracle Data Warehouse to GCP (Google Cloud Platform).

• Building and maintaining data ingestion processes from various sources into a Data Lake.
• Designing, developing, and optimizing complex data pipelines to ensure reliable data flow.
• Creating, developing, and maintaining frameworks that facilitate the construction of data pipelines.
• Implementing comprehensive testing frameworks for data pipelines.
• Collaborating with data analysts and data scientists to ensure the delivery of high-quality data.
• Ensuring robust data management, security, and compliance practices.
• Researching and implementing new technologies to improve data pipeline performance.
• Utilizing and integrating data from various types of source systems, including Kafka, MQ, SFTP, databases, APIs, and file shares.

• Key Skills: GCP, PL/SQL, SQL, Python
• Proficiency in cloud platforms and services (GCP) – MUST HAVE
• Bachelor’s or Master’s degree in Data Science, Statistics, Computer Science, Economics, or a related field
• Minimum 3 years of experience as a Data Analyst or Data Quality Analyst in a data-driven organization
• Proven experience in data quality management and data governance practices
• Strong database expertise, including advanced SQL and PL/SQL
• Proficiency in Python; experience with Scala is a plus
• Experience with Linux and Bash scripting
• Solid understanding of the Cloudera Hadoop technology stack (Apache Spark, Apache Kafka)
• Knowledge of data processing principles, including ETL processes and real-time data processing

Warszawa - Oferty pracy w okolicznych lokalizacjach