.
Data Engineer with Microsoft Fabric @ AVENGA
  • Lublin
Data Engineer with Microsoft Fabric @ AVENGA
Lublin, Lublin, Lublin Voivodeship, Polska
AVENGA
11. 6. 2024
Informacje o stanowisku

Data Engineer to work on implementing a new data platform for a new Annual Motor Insurance Product. The platform will be a Lakehouse environment using the medallion architecture in Microsoft Fabric. The candidate will be expected to work with the Director of IT and Business Intelligence Analysts to ingest motor insurance data from multiple sources, assist with design of Lakehouse and warehouse table schemas, manage and enhance data quality and ensure project complies with Data Governance requirements. This role is pivotal in delivering management information and reporting for new Annual insurance product and providing key data to Pricing, Underwriting and Fraud teams.


  • Proficiency in SQL and Python as data processing languages
  • Experience with Apache Spark or Databricks as data processing engines
  • Strong experience with Lakehouse patterns and medallion architecture in data engineering
  • Familiarity with Azure Data Services and other relevant data management tools including Microsoft Fabric, Synapse Analytics and Data Factory.

Data Engineer to work on implementing a new data platform for a new Annual Motor Insurance Product. The platform will be a Lakehouse environment using the medallion architecture in Microsoft Fabric. The candidate will be expected to work with the Director of IT and Business Intelligence Analysts to ingest motor insurance data from multiple sources, assist with design of Lakehouse and warehouse table schemas, manage and enhance data quality and ensure project complies with Data Governance requirements. This role is pivotal in delivering management information and reporting for new Annual insurance product and providing key data to Pricing, Underwriting and Fraud teams.

,[ Implement Medallion Architecture: Apply the medallion architecture to organise data layers (Bronze, Silver, Gold) within the Lakehouse, ensuring progressive enhancement of data quality., Design and Optimize Data Storage: Develop scalable and secure data storage solutions that align with the medallion architecture principles. Develop Data Processing Pipelines: Construct and maintain robust data pipelines for efficient data ingestion, transformation, and consolidation., Ensure Data Quality and Compliance: Uphold high standards of data governance, ensuring compliance with industry regulations and data privacy laws. Collaborate with Cross-Functional Teams: Work closely with stakeholders across the company to understand data requirements and deliver insights that drive strategic decisions. Requirements: SQL, Python, Apache Spark, Databricks, Lakehouse Patterns, Medallion Architecture, Azure Data Services Tools: Agile, Scrum. Additionally: Sport subscription, Private healthcare, Flat structure, Small teams, International projects, Worksmile, LuxMED, Multisport, Integration events.

  • Praca Lublin
  • Lublin - Oferty pracy w okolicznych lokalizacjach


    85 605
    15 914