Expert Data Engineer

Your responsibilities

• Building distributed and highly parallelized Big data processing pipeline which process massive amount of data (both structured and unstructured data) in near real-time
• Leveraging Spark to enrich and transform corporate data to enable searching, data visualization, and advanced analytics
• Working closely with analysts and business stakeholders to develop analytics models
• Continuous delivering on Hadoop and other Big Data Platforms
• Automating processes where possible and are repeatable and reliable
• Working closely with QA team

Our requirements

• Big Data
• ETL
• Data Testing
• SQL
• Hadoop stack: YARN, EMR, Sqoop, Hive, Oozie
• Hive, Oozie
• Python
• Scala
• Spark and Spark SQL