Design and implement infrastructure to host and manage up to 1,500 ML scoring processes within Databricks environment.

Operationalize ML scoring pipelines with scalable, secure, and monitored infrastructure.

Ensure integration with AI and BI tools and optimized data pipelines.

Enable rapid scaling of AI capabilities.

Regular AI/ML Ops Engineer

Twój zakres obowiązków

• Set up Databricks clusters, jobs, and workflows for ML scoring use cases.
• Implement scalable infrastructure capable of running thousands of ML scoring tasks.
• Develop deployment processes for ML models using Databricks MLflow or equivalent.
• Build frameworks to orchestrate scoring of >1,500 ML models or scoring jobs.
• Integrate logging, alerting, and dashboards to monitor scoring throughput.
• Work alongside Dev Ops Engineers to automate provisioning and deployments.
• Define operational SLAs for scoring workloads.
• Ensure role-based access control and credential management.
• Implement policies for secure handling of model artifacts and scoring data.

Nasze wymagania

• Azure knowledge.
• Experience with Databricks, MLflow, Jobs, Workflows, Delta Lake.
• MLOps skills.
• Python knowledge for workflow scripting.
• Terraform skills for Infrastructure as Code.
• Proven ML Ops experience in production ML environments.
• Experience in CI/CD pipelines for ML model deployment (e.g., Azure DevOps, GitHub Actions).
• Proficiency with monitoring tools (e.g., Datadog, Prometheus, Grafana).
• Knowledge of model lifecycle management, versioning, and reproducibility.
• Understanding of Infrastructure as Code and cloud environment automation.Knowledge of model lifecycle management, versioning, and reproducibility.
• English Level: B2
• Hybrid work from client's office in Warsaw/Poznań/Lublin
• Experience executing high-volume ML scoring in Databricks.
• Familiarity with industry best practices for regulated environments.
• Knowledge of job queueing systems and parallel execution patterns.
• Awareness of cost optimisation for ML scoring infrastructure.
• Exposure to Azure Databricks and the Azure ecosystem.
• German language knowledge.