.
Lead Data Engineer (PySpark)
  • Kraków
Lead Data Engineer (PySpark)
Kraków, Kraków, Lesser Poland Voivodeship, Polska
EPAM Systems (Poland) sp. z o.o.
23. 3. 2026
Informacje o stanowisku

Lead Data Engineer (PySpark)

Miejsce pracy: Kraków

Technologies we use

Expected

  • Azure Fabric
  • OneLake
  • Delta
  • OpenLake
  • Python
  • PySpark
  • SparkSQL
  • Cosmos DB
  • Azure DevOps
  • Microsoft Power BI

About the project

We are seeking a Lead Data Engineer with deep expertise in Microsoft Azure Fabric–based data platforms and AI-enabled data engineering.

This role blends hands-on technical leadership with architectural and team mentoring responsibilities, focusing on modern data engineering, big data processing, and AI-driven workflows in complex enterprise environments.

Your responsibilities

  • Lead the design, development and optimization of scalable data engineering solutions using Azure Fabric and cloud-native technologies
  • Own end-to-end data pipelines including ingestion, transformation, storage and analytics
  • Architect and implement solutions leveraging OneLake (Delta / OpenLake) and Fabric experiences
  • Develop and optimize PySpark, SparkSQL and Python-based data processing pipelines
  • Work with Cosmos DB (NoSQL API) and other Cosmos DB variants to support high-performance data access patterns
  • Implement and maintain CI/CD pipelines and promote DevOps best practices
  • Collaborate with data scientists, AI engineers and product stakeholders to enable AI-driven analytics and insights
  • Mentor and guide junior engineers, setting coding standards and best practices
  • Ensure data quality, security, governance and performance across platforms
  • Contribute to technical decision-making and solution architecture discussions

Our requirements

  • 9+ years of experience in data engineering with AI exposure
  • Expertise in Azure Fabric and end-to-end Fabric experience
  • Knowledge of OneLake (Delta / OpenLake)
  • Advanced skills in Python, PySpark and SparkSQL
  • Proficiency in Cosmos DB (NoSQL API) and other Cosmos DB variants
  • Understanding of DF Gen2 and M-code
  • Capability to implement CI/CD pipelines using Azure DevOps or equivalent tools
  • Experience with generic Azure services and Power BI integration, semantic models and performance considerations
  • Background in Agile or Scrum environments
  • Strong ownership mindset and ability to lead by example
  • Excellent communication skills for technical and non-technical audiences
  • Good understanding of the financial domain

Optional

  • Experience with code generation, including non-AI and AI-assisted approaches
  • Azure AI Foundry experience
  • Data Science fundamentals and collaboration with DS teams
  • Strong background in Big Data architectures and Spark ecosystems
  • Familiarity with financial instruments and financial services data
  • Hands-on experience with industry-standard LLMs (including GPT, Claude or similar)
  • Exposure to AI-enabled data platforms and intelligent analytics use cases

This is how we work on a project

  • Continuous Deployment
  • Continuous Integration
  • DevOps

Development opportunities we offer

  • conferences in Poland
  • development budget
  • external training
  • industry-specific e-learning platforms
  • intracompany training
  • mentoring
  • soft skills training
  • space for experimenting
  • substantive support from technological leaders
  • support of IT events
  • technical knowledge exchange within the company

What we offer

  • Engineering community of industry professionals
  • Friendly team and enjoyable working environment
  • Flexible schedule and opportunity to work remotely within Poland
  • Chance to work abroad for up to 60 days annually
  • Business-driven relocation opportunities
  • Outstanding career roadmap
  • Leadership development, career advising, soft skills, and well-being programs
  • Certification (GCP, Azure, AWS)
  • Unlimited access to LinkedIn Learning, Get Abstract, Cloud Guru
  • English language classes
  • Stable income (Employment Contract or B2B)
  • Participation in the Employee Stock Purchase Plan
  • Benefits package (health insurance, multisport, shopping vouchers)
  • Strategically located offices featuring entertainment and relaxation zones, table tennis and football, free snacks, fantastic coffee, and more
  • Referral bonuses
  • Corporate, social and well-being events

Benefits

  • private medical care
  • life insurance
  • flexible working time
  • fruits
  • integration events
  • retirement pension plan
  • no dress code
  • video games at work
  • coffee / tea
  • parking space for employees
  • leisure zone
  • employee referral program
  • charity initiatives

EPAM Systems (Poland) sp. z o.o.

EPAM is a leading global provider of digital platform engineering and development services. We are committed to having a positive impact on our customers, our employees, and our communities. We embrace a dynamic and inclusive culture. Here you will collaborate with multi-national teams, contribute to a myriad of innovative projects that deliver the most creative and cutting-edge solutions, and have an opportunity to continuously learn and grow. No matter where you are located, you will join a dedicated, creative, and diverse community that will help you discover your fullest potential.

Klikając w przycisk „Aplikuj” lub w inny sposób wysyłając zgłoszenie rekrutacyjne, zgadzasz się na przetwarzanie Twoich danych osobowych przez EPAM Systems (Poland) sp. z o.o. z siedzibą w: Fabryczna 1A, 31-553 Kraków (Pracodawca), jako administratora danych osobowych w celu przeprowadzenia rekrutacji na stanowisko wskazane w ogłoszeniu zgodnie Polityką Prywatności dostępną na stronie: https://www.epam.com/applicant-privacy-notice

  • Praca Kraków

    • Kraków - Oferty pracy w okolicznych lokalizacjach

    Odpowiedz na ogłoszenie
    Praca Kraków - Ciekawe oferty pracy w okolicy:
    Lead Data Engineer – AI & Cloud Technologies (f/m/x)
    Civil and Structural Lead Engineer
    Data Engineer
    Software Engineer (Data)
    Senior Data Engineer (Spark)
    Principal AI Engineer / AI Lead
    Lead Data Scientist / ML Engineer
    Senior Data Scientist / Tech Lead
    Senior Data Engineer (PySpark, NoSQL)
    • Kraków
      EPAM Systems (Poland) sp. z o.o.
    104 742
    15 416