We are looking for you, if you:
- have hands-on experience building complex data pipelines,
- have strong experience in data processing and optimization using Apache Spark,
- have good knowledge of data manipulation and transformation,
- have experience with Apache Airflow, Python, SQL,
- have knowledge of software engineering best practices,
- have deployment and provisioning automation tools (e.g., Docker, Kubernetes, CI/CD),
- have excellent communication skills, writing and verbal in English,
- want to learn more and proactively acquire knowledge.
Youll get extra points for:
- hands-on experience managing and further developing distributed systems and clusters for both batch as well as streaming data (S3/Spark and/or Kafka/Flink),
- hands-on experience in training and evaluating ML models,
- proven experience with Data Science tools and practices,
- experience in setting up noSQL databases,
- experience with Golang,
- experience with monitoring and observability (ELK stack),
- bash scripting and Linux systems,
- experience with buildpacks,
- Security, authentication, and authorisation (LDAP / Kerberos / PAM),
- experience working in cloud environment (e.g., GCP).
Your responsibilities:
- creation, schedule and optimization of the data processing pipelines using Apache Spark and Apache Airflow deployed on our data analytics platform,
- improve data flow, establish new data sources and deployment,
- troubleshoot data issues, providing robust solutions to ensure optimal performance and reliability,
- continuously optimize our systems for performance and cost-effectiveness.
Information about the squad:
You will work as an engineer in our ING Bank Netherlands Fraud domain. Here we employ state-of-the-art machine learning models and processing algorithms to identify potential fraud activities within our payment chain.
Working at ING means working in a dynamic and international setting. Individual development of our employees is particularly important and that is why ING offers excellent courses and programs.
The role naming convention in the global ING job architecture will be “Engineer IV”.
