Our client is seeking a passionate and experienced Senior Data Engineer to support the development of their blockchain crypto solution. This innovative application enables audit teams to verify their clients assets, such as cryptocurrencies. As a Senior Data Engineer, you will play a crucial role in managing and leveraging large datasets, ensuring data integrity, and supporting blockchain audit-related tasks.

Start Date: ASAP

Remote vs Onsite: Remote

HackerRank Challenge: Yes

US Hours Overlap Needed: Yes (2-6 PM CET)

Senior Data Engineer - Blockchain Solutions

Your responsibilities

• Design, develop, and maintain scalable data pipelines using Python, PySpark, and Databricks.
• Implement data orchestration workflows using Airflow or similar tools.
• Manage and optimize data storage solutions on Azure Cloud.
• Develop and maintain containerized microservices using Docker.
• Collaborate with cross-functional teams to ensure data quality and integrity.
• Support the setup of blockchains.
• Build tools to acquire data from blockchains and perform analysis using Databricks.
• Support production infrastructure, including monitoring and automation with tools like Datadog.
• Participate in agile development practices, contributing to planning exercises and working in cross-functional groups.

Our requirements

• Bachelor’s or Master’s degree in Computer Science, Data Science, or a related field.
• Advanced knowledge of Python for data processing and scripting.
• Proven experience with Spark and SQL for data engineering and analysis.
• Familiarity with data orchestration tools such as Airflow.
• Hands-on experience with cloud services (Azure, AWS, GCP).
• Proficiency in RDBMS/NoSQL data stores and appropriate use cases.
• Experience with Data as Code, including version control, regular commits, unit tests, CI/CD, packaging, and familiarity with containerization tools such as Docker (must-have) and Kubernetes (a plus).
• Solid understanding of the software development life cycle.
• Understanding of blockchain technologies and data structures.
• Knowledge of cryptography and its applications in blockchain is a plus.
• Experience with blockchain indexing and Databricks for data ingestion and transformation is a plus.
• Familiarity with Delta Lake and data warehousing concepts is a plus.