A Data Engineers role involves crafting, constructing, and upholding the structure, tools, and procedures essential for an organization to gather, store, modify, and scrutinize extensive data amounts. This position involves creating data platforms using typically provided infrastructure and establishing a clear path for Analytics Engineers who utilize the system.
responsibilities :
Development and maintenance of ETL and data platforms (Python, Scala, Spark, HDFS, Hive)
Development and maintenance of access applications for business users and user support (Airflow, Jupyterhub, Trino, Superset, MLFlow) in the context of Kubernetes, Docker, ArgoCD
Automation and CICD (Gitlab-CI)
Monitoring (Prometheus)
R&D, maintenance, and monitoring of the platforms components
Implementing and executing policies aligned with the companys strategic plans concerning used technologies, work organization, etc.
requirements-expected :
Proficiency in a programming language like Python and Scala
Working with Spark messaging systems
Experience with Hadoop
Strong programming skills with a solid understanding of software engineering principles, best practices, and solutions
Experience with Version Control System, preferably GIT
Ability to actively participate/lead discussions with clients to identify and assess concrete and ambitious avenues for improvement
offered :
Salary: 110 - 160 PLN net + VAT/h B2B (depending on knowledge and experience)
100% remote work
Flexible working hours
Possibility to work from the office located in the heart of Warsaw
Opportunity to learn and develop with the best Big Data experts
