You have experience as data engineer working with Databricks (preferrably hosted in AWS)
Your responsibilities will be to:
- Design, develop and maintain scalable data pipelines on the Databricks platform
- Manage data ingestion processes from various sources into the Databricks environment (xml ingestion, Kakfa, etc.).
- Implement and optimize ETL (Extract, Transform, Load) processes to ensure efficient data transformation and integration.
- Ensure data quality, integrity, and security across all data pipelines.
- Monitor and troubleshoot data pipeline performance, implementing necessary maintenance and updates.
- Document data pipeline processes, technical specifications and best practices.
- Realize knowledge transfer towards the Data engineer team which will take over the responsibility to run the platform in the future
Being fluent in either English/Dutch or English/French is a MUST