We are seeking an experienced Data Migration & Integration Specialist with expertise in Hive and AWS Glue technologies. In this role, you will lead the migration, integration, and optimization of data systems in a cloud environment, ensuring seamless functionality and collaboration with analytics teams.
responsibilities :
Lead the migration of metadata from Hive Metastore to AWS Glue Catalog, maintaining data consistency and integrity.
Leverage AWS Glue Hive Metastore Connector and Glue Crawlers to automate metadata migration and table creation.
Replicate Hive partitioning, data formats (e.g., Parquet, ORC), and schema to Glue, ensuring compatibility with AWS services like Athena and EMR.
Collaborate closely with data analysts and scientists to enable seamless integration with data-driven analytics platforms.
Configure and manage AWS Glue security, including IAM roles and Lake Formation, to ensure robust data access control and governance.
Monitor data quality, performance, and cost efficiency in Glue and Athena, ensuring optimal query performance and cost management.
Conduct thorough testing and validation of Glue Catalog migrations to guarantee data integrity and system functionality.
Troubleshoot and resolve issues related to the migration process and post-migration data pipelines.
requirements-expected :
Knowledge of English enabling work in an international team (level C1)
Proven expertise in Hive Metastore and AWS Glue.
Experience with metadata migration, including use of AWS Glue Crawlers and Hive Metastore Connectors.
Strong knowledge of Hive partitioning, data formats (Parquet, ORC), and schema replication in Glue.
Familiarity with AWS Athena, EMR, and other AWS data services.
Proficiency in configuring AWS Glue security features such as IAM roles and Lake Formation.
Experience in data quality monitoring, performance optimization, and cost management in AWS environments.
Excellent troubleshooting skills with a focus on migration and post-migration processes.
Effective communication and collaboration skills, particularly with data analysts and scientists.
offered :
100% remote work opportunity
Hourly rate: 130-150zł/h+VAT depending on your experience
Opportunity for long-term engagement and career development in cutting-edge data technologies.
A range of various benefits (Multisport card, Medicover private medical care with access to the Damian Medical Center, group life insurance) provided after onboarding period
Training and development aimed at upskilling
One-of-a-kind team-building events, competitions and challenges
Sports activities and language courses
benefits :
sharing the costs of sports activities
private medical care
sharing the costs of foreign language classes
sharing the costs of professional training & courses
