The Senior Data Engineer is a technical role responsible for designing, developing, and maintaining data pipelines. The pipelines will be realized in alignment with the future data architecture and the engineer will collaborate in cross-functional teams to gather requirements and develop the conceptual data models. This role plays a crucial part in driving data-driven decision-making across the organization, ensuring data availability, quality, and accessibility for various business needs.
responsibilities :
Data Pipeline Development:
Design, model, develop and maintain data pipelines to ingest, store, process, and present data.
Ensure data quality, accuracy, and consistency.
Collaborate with data architects to ensure data pipelines align with the overall data architecture strategy.
Data Transformation and Integration:
Perform data transformation tasks, including data cleansing, enrichment, and aggregation, to prepare data for analytics and reporting.
Integrate data from structured and unstructured sources, ensuring compatibility and alignment with data models and business requirements.
Automate data transformation processes to improve efficiency.
Data Quality Assurance:
Implement and maintain data quality checks and validation processes to identify and resolve data anomalies and errors.
Monitor data pipelines for data quality issues and implement data quality improvements.
Collaborate with business stakeholders to define data quality requirements.
Data Modelling and Schema Design:
Collaborate with data architects and data scientists to design and implement data models, schemas, and structures.
Ensure that data models support business reporting and analytics needs while optimizing query performance.
Maintain data dictionaries and metadata to document data structures and relationships.
Performance Optimization:
Optimize data storage, retrieval, and query performance by implementing indexing, partitioning, and caching strategies.
Monitor data processing performance and address bottlenecks as they arise.
Stay updated with best practices in data processing performance tuning.
Documentation and Knowledge Sharing:
Create and maintain documentation for data pipelines, data transformation processes, and data integration procedures.
Foster a culture of knowledge sharing within the data engineering team and across the organization.
Collaboration and Stakeholder Engagement:
Collaborate effectively with cross-functional teams, data stakeholders, and business units to understand data requirements and deliver data solutions that meet business needs.
Communicate technical concepts and data solutions to non-technical stakeholders in a clear and understandable manner.
requirements-expected :
Minimum 4 years commercial data engineering experience.
Commercial experience of DataBricks, SQL, Python, Power BI.
Experience in data engineering, including designing and developing data pipelines for retrieval / ingestion / presentation / semantics in an Azure environment.
Effective communication and collaboration skills to work with cross-functional teams and gather data requirements.
Skills in data modelling (both structured and unstructured data).
Power Automate.
Data optimization for performance, scalability, and efficiency.
Experience in automotive/distribution sector.
benefits :
sharing the costs of sports activities
private medical care
sharing the costs of professional training & courses