Informacje o stanowisku

Working on Real-Time Streaming Data Platform (Java & Apache Spark)

Tech stack:

• Apache Spark on Azure Databricks
• Apache Kafka
• Delta Lake
• Java
• MS SQL Server and NoSQL storages like Elastic Search, Redis, Azure Data Explorer
• Docker containers
• Azure DevOps and fully automated CI/CD pipelines with Databricks Asset Bundles, ArgoCD, GitOps, Helm charts
• Automated tests
•  Proficient in programming, not just data processing, we are not considering candidates with an ETL profile.

Personal requirements:

• Have 4+ years of Apache Spark experience and have faced various data engineering challenges in batch or streaming
• Have an interest in stream processing with Apache Spark Structured Streaming on top of Apache Kafka
• Have experience leading technical solution designs
• Have experience with distributed systems on a cloud platform
• Have experience with large-scale systems in a microservice architecture
• Are familiar with Git and CI/CD practices and can design or implement the deployment process for your data pipelines
• Possess a proactive approach and can-do attitude
• Are excellent in English and Polish, both written and spoken
• Have a higher education in computer science or a related field
• Are a team player with strong communication skills

Working on Real-Time Streaming Data Platform (Java & Apache Spark)

,[ Requirements: Java, Apache Spark, Azure Databricks, SQL, NoSQL, Elasticsearch, Redis, Azure, Docker, DevOps, CI/CD Pipelines, Databricks, ArgoCD, Helm Charts, ETL, Apache Kafka, Data engineering, Cloud platform, Microservice architecture, Git, Communication skills, Spark Tools: . Additionally: Sport subscription, Private healthcare, International projects, Masterclazz training.

Praca Lublin

Lublin - Oferty pracy w okolicznych lokalizacjach