The project focuses on designing, configuring, and operating a scalable Databricks environment that supports business intelligence, advanced analytics, and AI/ML workloads within a unified lakehouse architecture. The platform will serve multiple teams, including BI Analysts, Data Engineers, and ML Engineers, with strong emphasis on security, governance, automation, and operational excellence.
We are looking for a Regular DevOps Engineer to take ownership of the Databricks platform setup and operations, ensuring it is production-ready, secure, cost-efficient, and fully integrated with BI and downstream analytics tools.
The role requires close collaboration with ML Ops Engineers, Data Engineers, and Security teams to deliver a shared, enterprise-grade data platform supporting both traditional BI and AI use cases.
The position includes On-Call Duty.
responsibilities :
Deploy and configure Azure Databricks workspaces for multi-team usage
Design and manage shared clusters, job clusters, and interactive analysis clusters
Implement role-based access control aligned with data governance policies
Configure secure connectivity to on-premise and cloud data sources (SQL Server, Data Lakes, APIs)
Build and maintain shared ingestion pipelines for BI and analytics teams
Automate daily and weekly data refresh processes
Integrate Databricks with BI platforms (e.g. Power BI)
Optimize JDBC/ODBC connectors for performance and scalability
Implement monitoring, alerting, and logging for Databricks jobs and pipelines
Define and maintain backup and disaster recovery procedures
Track and optimize infrastructure and cluster costs
requirements-expected :
Minimum experience: 3+ years in DevOps or data platform operations
Strong hands-on experience with cloud-based DevOps environments
Practical knowledge of Databricks platform administration
Proficiency in Python for automation and scripting
Solid understanding of SQL and data engineering fundamentals
Experience with orchestration and scheduling tools (Databricks Workflows, Airflow, Azure Data Factory)
Knowledge of cloud Identity & Access Management (IAM)
