As Big Data & Analytics Center of Excellence - a data consulting and data engineering branch of SoftServe – we are looking for a skilled Big Data Lead Engineer who will contribute to our client - a titan in the energy sector who is dedicated to innovating and investing in clean energy electrifying across 49 states in North America.
responsibilities :
Collaborate with the team of data-focused engineers who specialize in ML, ML/Ops, Analytical AI, GenAI, large data engineering efforts, and AWS
Work on a project that involves such languages and technologies as vector embeddings, R, Python, AWS Glue, RedShift, DynamoDB, Neo4j, Elastic and Milvus for vector stores for AI projects
Design and implement data pipelines to support AI solutions, meaning moving data between different data storages (Redshift, Aurora, S3, Neo4J) and serving that data to AI models
Lead data Team responsible for the development of data pipelines, creating solutions like data marts by Aurora and Redshift use case by use case, eventually pushing that data to DynamoDB
Support customers from all over the world, from huge global corporations to crypto startups that are about to launch their first product
Attend in the entire project lifecycle, from initial design through PoCs and MVP up to the full-scale implementation
Lead data engineering or generic software development teams in the role of Architect or Team Leader
Design and implement Data & Analytics solutions in AWS
Be a customer-facing consultant responsible for eliciting customer requirements, translating them into technical architecture, and presenting your decisions and proposals to a customer
Closely cooperate up to three days a week in our comfortable office located near Rondo ONZ
requirements-expected :
A seasoned Data Lead Engineer focused on data pipeline creation and comfortable with batch processing, streaming processing, or both
Used to leading a team or considering yourself ready to do it
Experienced in Python, Scala, or Java programming languages (at least one) and SQL
Skilled in data solutions development within AWS
Confident with Apache Spark, Databricks, Flink, Apache Airflow or MWAA
Practiced in Apache Kafka, MSK, or Kinesis and Amazon Redshift or Snowflake
Familiar with leading a team or being an Architect
Effective as a communicator, showing Upper Intermediate+ English level
benefits :
sharing the costs of sports activities
private medical care
sharing the costs of foreign language classes
sharing the costs of professional training & courses
