115 886
19 661
Lead Data Engineer (Databricks) Miejsce pracy: Wrocław Technologies we use Expected Python Databricks Spark SQL Apache Spark PySpark Airflow Dagster Prefect AWS Azure Optional Kafka Azure Event Hubs Terraform CloudFormation Unity Catalog Delta Lake Lakehouse Operating system Windows macOS About the project At Grape Up, we transform businesses by unlocking the potential of AI and data through innovative software solutions. We partner with industry leaders, from the automotive and finance industry, to build sophisticated Data & Analytics platforms that transform how organizations manage and leverage their data assets. Our solutions provide comprehensive capabilities spanning data storage, management, advanced analytics, machine learning, and AI, enabling enterprises to accelerate innovation and make data-driven decisions. Your responsibilities Implement a scalable architecture capable of handling the high volume of simulation data Build a flexible data preprocessing pipelines that are extensible and that can be integrated into customer’s existing platform Define KPIs to measure the improved reusability and automation of the new pipelines and test their performance in an end-to-end setting with model training Develop and implement processes and best practices for data management and governance Optimize and enhance system setup and improve data structures following industry best practices Collaborate effectively with data engineering team members while partnering closely with analytics and data science teams to meet user needs Collaborate with business stakeholders and technical teams to understand data requirements, translate business needs into technical solutions Lead technical discussions and solution design sessions with clients or internal stakeholders, presenting complex data engineering concepts in accessible ways Our requirements Master’s degree in computer science, Data Engineering, AI, or a related field 4 years of professional experience in Data Engineering and Big Data, building production-grade data platforms and pipelines Proven experience with Databricks platform (Azure Databricks or Databricks on AWS) Strong experience with Apache Spark (PySpark), including performance optimization, and large-scale data processing Proficiency in Python and SQL for data transformations and analytics workloads Experience with data pipeline orchestration (Airflow, Dagster, Prefect or similar) Experience with data governance and quality frameworks Strong problem-solving skills and ability to work independently Fluency in English, both written and spoken Optional PhD degree in computer science, Data Engineering, AI, or a related field (completed or in progress) Advanced Databricks expertise (Unity Catalog, Delta Lake and Lakehouse architecture) Data Engineer (Databricks) certificate Experience processing high-volume Automotive or IoT data (e.g., telemetry, sensor, or event-based data) Experience in pre-sales and consulting activities Experience with streaming tools like Kafka or Azure Event Hubs, etc. Experience with Infrastructure as Code (Terraform, CloudFormation) This is how we organize our work This is how we work in house you focus on a single project at a time you have influence on the choice of tools and technologies you have influence on the technological solutions applied you have influence on the product agile Team members data scientist product owner project manager Development opportunities we offer assistance in preparation to public speeches industry-specific e-learning platforms substantive support from technological leaders support of IT events technical knowledge exchange within the company What we offer Cutting-Edge Technology: Drive innovation with the latest tech solutions Work Your Way: Enjoy complete flexibility in choosing your ideal work environment – office, hybrid or remote. Non-corporate work environment Equipment of your choice Language lessons (English, German, and Polish) LuxMed private medical care Weekly Lunch & Learn where we meet up in the office, lunch together, and share our knowledge Employee referral program Rewards for the Success of the Month and Year awarded by our employees G-Man work anniversary awards Integration activities Benefits private medical care sharing the costs of foreign language classes sharing the costs of professional training & courses remote work opportunities flexible working time fruits integration events no dress code video games at work coffee / tea drinks parking space for employees leisure zone extra social benefits baby layette employee referral program charity initiatives Recruitment stages Application HR interview Meeting with the Manager Offer Feedback Grape Up Sp. z o.o. Hi, we are Grape Up! Our company help enterprises design, modernize, and scale their technology landscape - building a data-driven foundation for sustainable innovation and business intelligence powered by AI. Information clausefor job candidates in companies from Grape Up capital groupThis clause has been created to explain how your data as an applicant for a job will be processed by the data controller.The controller of the processing of your personal data (hereinafter: Controller) is the company from Grape Up capital group which conducts the recruitment process.If you apply for a position in:Poland or United Kingdom of Great Britain and Northern Ireland: the Controller is Grape Up sp. z o.o. with its registered office in Kraków, Hetmana Żółkiewskiego Str. 17A, zip code 31-539, KRS number 0000513816Germany: the Controller is Grape Up GmbH with its registered office in Munich, Wendl-Dietrich-Straße 18, 80634 Munich, Germany, HRB 288851United States of America: the Controller is GRAPE UP INC. 5201 Great America Parkway, Suite 320, CA 95054, EIN: 47-4475207What is the scope of data that we process?We only process data that you provide in your application form/email and the attached documents (CV, cover letter, etc.) and that we collect during interviews, to enable the recruitment process and eventually employment. We may process other data only if you provide us with specific consent for processing of such data (e.g. to confirm your statements or grant references by contacting other entities indicated by you).What is the purpose and legal basis for processing?The personal data we collect is necessary for the fulfilment of the following purposes:to fulfil our legal obligation under labour law regulations applicable in a country where you apply for an open position (Article 6(1)(c) GDPR), i.e., data on the first name, surname, contact details such as e-mail address, date of birth, education, professional qualifications, and previous employment orfor the conclusion and subsequent performance of the contract with you, i.e., in particular, the data included in the contract concluded in the event of a successful recruitment process (Article 6(1)(b) GDPR) or for the legitimate interests pursued by us (Article 6(1)(f) GDPR), such as:the selection of the right candidate to ensure the proper performance of the tasks assigned to them, i.e., the data collected by us during the recruitment activity, i.e.,the records made during the interviews with you (other than those indicated in points 1 and 2 above, but necessary to achieve the stated purpose);the assertion and defense of claims before courts, administrative authorities, and other bodies (in respect of all data collected);for which you have given us your consent (Article 6(1)(a) GDPR) – by your declaration or explicit act of confirmation, such as providing us your CV or cover letter containing such data, i.e., data in the recruitment documents and references received from you, messages, conversations with you and with entities (e.g. your former or current employers and principals/service recipients) that you indicate to us to confirm your employment/co-operation with them (within the scope of your consent which can include processing of such data as your employment/co-operation with other entities, indicated by you, its duration, termination and/or basis and working time of the current employment/cooperation) (in respect of personal data other than those mentioned in points 1-3 above).We may process your data in an automated or paper form.How long will your data be stored?Your personal data will be processed by us:for the purposes of recruitment for a specific job offer – from the date of their collection:for a period of up to 6 months after the end of the recruitment process – in case of rejection of your application (while you do not consent to the processing of your personal data in future recruitments);until the conclusion of the contract – in the event of a successful recruitment process;for the purposes related to participation in future recruitment (in case you consented to further processing of your data for future recruitment processes) from the date of obtaining your consent for a period of the next 36 months, calculated from the end of the calendar year in which you gave your consent or until you withdraw your consent;in exceptional cases to defend your potential claims or pursue our claims – from the date of completion of any recruitment process in which you participate until the expiry of the limitation period for your or our claims (but no longer than the date of your effective objection to the processing).Where did we collect your data from?We have received your data directly from you or third parties, in particular:from providers of recruitment services and web applications that improve the recruitment process and the management of recruiter databases (the legal basis for obtaining your data from such providers is your consent);from people who refer you as a candidate – in a situation where your data is provided to us by a participant in the employee referral program (in such a situation, the person referring you should ensure that they have obtained a legal basis for providing your data);from publicly available sources, such as social media and your profile published there.from entities indicated by your (such as your former or current employers or principals/ service recipients).Who is your personal data recipient?As a Controller, we may share your personal information with third parties in the following circumstances:third parties who provide us with services necessary to achieve the purposes for which we process your data (e.g., IT, recruitment services, skills testing, electronic communications, hosting) and their contractors;recipients, the disclosure of which is required by relevant law or order of a court or other authority;other recipients if you give them your consent or if the transfer of data to them is necessary to protect your vital interests or the vital interests of others or for the public common good;Grape Up sp. z o.o. (in case where the Administrator is another company from Grape Up capital group) – if Grape Up sp. z o.o. supports the Controller in the recruitment process.Companies from Grape Up capital group – if you consent to the transfer of your personal data to this company for the purpose of recruitment.In case of transfer of personal data to countries outside the EEA, we ensure that they provide guarantees of a high level of personal data protection, including the use of standard contractual clauses approved by the European Commission.What are your data subject rights?The right to access data and obtain a copy of the data. You have the right to request information, at any time, about your personal data that we store or to which we have access. Upon your request, you will be provided, free of charge, with a copy of your personal data that is the subject of processing. For sending each subsequent copy of data, we have the right to charge a fee to cover the reasonable costs of processing such a request.Right to withdraw consent. Each time your data is processed based on consent, you have the right to withdraw this consent at any time, and the withdrawal of the consent does not affect the lawfulness of the data processing that took place before the withdrawal of consent.The right to rectify personal data. We take reasonable steps to ensure that your personal data is accurate, complete, and up-to-date. If it is necessary to change (update or complete) this data, please let us know.Right to data portability. You have the right to request the transfer of your data in a structured, commonly used, and machine-readable format, as well as the transfer of the data to another data controller, if your consent is the legal basis for the processing of your data.The right to delete data and to limit its processing. In the cases indicated in the legal provisions on the protection of personal data, you have the right to request the erasure of your data. However, this right is not absolute; there may be cases where we are still entitled to process your data. You may also request restrictions on further processing of your data.Right to object to processing. In the cases specified in the legal provisions, you have the right to object to the further processing of your data if the legal basis for the data processing is in our legitimate interest.The right to file a complaint with a supervisory authority. You have the right to file a complaint with the applicable supervisory authority that deals with the protection of personal data, e.g. the President of the Office for Personal Data Protection (in case of Poland), Der Bundesbeauftragte für den Datenschutz und die Informationsfreiheit (in case of Germany).In matters regarding your personal data, you can contact us directly:by mail: Hetmana Żółkiewskiego Str. 17A, 31-539 Kraków, Poland,by email: careers@grapeup.com