Informacje o stanowisku

Social network you want to login/join with:

Data Engineer with Expertise in Groovy and Spark, Łódź

Client:

DataArt

Location:

Łódź

Job Category:

Other

EU work permit required:

Yes

Job Reference:

415b898ef7a1

Job Views:

20

Posted:

23.01.2025

Expiry Date:

09.03.2025

Job Description:

Responsibilities

1. Work closely with data engineers, the product team, and other stakeholders to gather data requirements, and design and build efficient data pipelines.
2. Create and maintain algorithms and data processing code in Java/Groovy.
3. Implement processes for data validation, cleansing, and transformation to ensure data accuracy and consistency.
4. Develop Python scripts to automate data extraction from both new and existing sources.
5. Monitor and troubleshoot the performance of data pipelines in Airflow, proactively addressing any issues or bottlenecks.
6. Write SQL queries to extract data from BigQuery and develop reports using Google’s Looker Studio.
7. Participate in daily stand-ups, sprint planning, and retrospective meetings.
8. Engage in peer code reviews, knowledge sharing, and assist other engineers with their work.
9. Introduce new technologies and best practices as needed to keep the product up to date.
10. Assist in troubleshooting and resolving production escalations and issues.

Requirements

1. Bachelors degree or equivalent programming experience.
2. 4-5 years of overall experience as a backend software developer, with at least 2 years as a Data Engineer using Spark with Java/Groovy and/or Python.
3. Strong coding skills, and knowledge of data structures, OOP principles, databases, and API design.
4. Highly proficient in developing programs and data pipelines in Java/Groovy or Python.
5. 2+ years of professional experience with Apache Spark/Hadoop.

Nice to have

1. Work experience with AWS (EMR, S3, lambda, EC2, glue, RDS).
2. Work experience with SQL (MYSQL is a Plus) and NoSQL Databases.
3. Experience with Elasticsearch.
4. Experience with Python.
5. Experience with Scala (Zeppelin).
6. Experience with Airflow or other ETL.
7. Certification or verified training in one or more of the following technologies/products: AWS, ElasticSearch.

Łódź - Oferty pracy w okolicznych lokalizacjach