Data Engineer
Is this your next job Read the full description below to find out, and do not hesitate to make an application.
Responsible for ensuring high-quality access to and management of data sources, guaranteeing data quality through cataloging, normalization, and qualification to support analytics teams. Collaborates on defining data governance policies and structuring the data lifecycle in compliance with regulations. Oversees data integration and supervision from various application systems and Big Data/IoT platforms, ensuring data quality in the Data Lake through validation and duplicate removal.
Key responsibilities
Ensure high-quality access to data sources
Master and maintain data quality through cataloging, normalization, and qualification to facilitate use by analytics teams
Capture, integrate, and consolidate structured and unstructured data from diverse systems into the Data Lake
Structure, MAP, clean (duplicate removal), and validate data sets
Collaborate on establishing data governance policies and managing the data lifecycle in compliance with regulations
Oversee and monitor data pipelines on Big Data and IoT platforms
Skills and competences
Agile methodology
Multi-tasking
Analytical thinking
Organizational skills
Qualifications
At least 5 years of experience with Agile methodology
At least 5 years of experience in Big Data architecture and technologies
At least 5 years of experience with Python
At least 5 years of experience with SQL
xlxgzvr
At least 5 years of experience developing and consuming APIs
At least 3 years of experience working on multiple topics simultaneously
At least 3 years of experience in analytical and synthesis work
At least 3 years of experience demonstrating organizational rigor
At least 3 years of experience with Spark
At least 3 years of experience with AWS
At least 3 years of experience with Databricks
At least 3 years of experience with Shell scripting