Our client is seeking a passionate and experienced Senior Data Engineer to support the development of their blockchain crypto solution. This innovative application enables audit teams to verify their clients assets, such as cryptocurrencies. As a Senior Data Engineer, you will play a crucial role in managing and leveraging large datasets, ensuring data integrity, and supporting blockchain audit-related tasks.
Start Date: ASAP
Remote vs Onsite: Remote
HackerRank Challenge: Yes
US Hours Overlap Needed: Yes (2-6 PM CET)
responsibilities :
Design, develop, and maintain scalable data pipelines using Python, PySpark, and Databricks.
Implement data orchestration workflows using Airflow or similar tools.
Manage and optimize data storage solutions on Azure Cloud.
Develop and maintain containerized microservices using Docker.
Collaborate with cross-functional teams to ensure data quality and integrity.
Support the setup of blockchains.
Build tools to acquire data from blockchains and perform analysis using Databricks.
Support production infrastructure, including monitoring and automation with tools like Datadog.
Participate in agile development practices, contributing to planning exercises and working in cross-functional groups.
requirements-expected :
Bachelor’s or Master’s degree in Computer Science, Data Science, or a related field.
Advanced knowledge of Python for data processing and scripting.
Proven experience with Spark and SQL for data engineering and analysis.
Familiarity with data orchestration tools such as Airflow.
Hands-on experience with cloud services (Azure, AWS, GCP).
Proficiency in RDBMS/NoSQL data stores and appropriate use cases.
Experience with Data as Code, including version control, regular commits, unit tests, CI/CD, packaging, and familiarity with containerization tools such as Docker (must-have) and Kubernetes (a plus).
Solid understanding of the software development life cycle.
Understanding of blockchain technologies and data structures.
