The person in this role will work with project teams developing a modern data platform in a cloud environment for global brands. They will work in an international environment, specializing in BI & Big Data cloud architecture and the latest technologies in that area, and promote the principles of the DataOps manifesto.
responsibilities :
Design, build, and optimize scalable data pipelines in Databricks platform using SQL/Python/Spark.
Developing existing projects in the Microsoft Azure environment to gain valuable insights for the business.
Ensuring that data is modelled and processed according to the architecture and both functional and non-functional requirements.
Planning and implementing processing pipelines for structured and unstructured data (e.g. video and images).
Working to automate and optimize internal processes in Azure.
Collaborating with cross-functional and international teams both internally and externally.
requirements-expected :
Minimum 5 years experience working with Databricks platform (Delta Lake, Workflows/Jobs, DLT, Unity Catalog).
Strong experience with programming in Python.
Solid understanding of SQL and relational databases.
Knowledge of Data Warehouse, Business Intelligence and ETL/ELT data processing.
Familiarity with Medallion Architecture.
Very good knowledge of English (particular emphasis on written English).
Proactive approach to tasks, problem-solving attitude and critical thinking skills.
Flexibility, independence and responsibility for assigned tasks.
A constant desire to improve your skills and learn new technologies.
Knowledge of Azure cloud components for data storage and processing: Azure Data Lake Storage, Azure Event Hubs, Azure Data Factory, SQL Database, Synapse, Fabric.
Experience with other big data technologies such as Hadoop, Hive, Kafka, and Flink would be an asset.
