Devire IT Outsourcing is a form of cooperation dedicated to IT professionals, based on the principles of own business - B2B, implementing projects for clients running innovative and modern projects.
Together with our client, a global leader in the field of financial solutions based on artificial intelligence, we are developing a technology center in Poland and are looking for people for the position of Senior Data Engineer.
ID: SNR-Data009
Devire IT Outsourcing is a form of cooperation dedicated to IT professionals, based on the principles of own business - B2B, implementing projects for clients running innovative and modern projects.
Together with our client, a global leader in the field of financial solutions based on artificial intelligence, we are developing a technology center in Poland and are looking for people for the position of Senior Data Engineer.
ID: SNR-Data009
,[Designing, optimizing, and maintaining reliable batch data pipelines for processing, filtering, and integrating large datasets., Managing the entire data lifecycle – from acquisition to quality control and storage – ensuring data integrity and availability., Optimizing data pipelines and warehouse architecture for scalability, low latency, and high throughput., Collaborating with analytics and engineering teams to translate business requirements into scalable data solutions., Implementing validation standards, quality control, and documentation to ensure data reliability., Developing and maintaining CI/CD pipelines using DevOps tools for seamless deployments and iteration., Monitoring, diagnosing, and implementing safeguards in data pipelines to minimize downtime., Mentoring junior team members and sharing knowledge. Requirements: Data pipelines, Python, Spark, Airflow, ETL, AWS S3, Glue, IAM, Redshift, AWS Lambda, CloudWatch, Kinesis, Data warehouses, Terraform, IaC, Cloud, Jenkins