Responsibilities
* Design and maintain production-ready data pipelines using medallion architecture.
* Optimize ETL/ELT processes for large-scale data processing on Delta Lake.
* Implement data governance through Unity Catalog, managing access controls and data lineage.
* Build CI/CD pipelines for automated testing and deployment of data workloads.
* Configure Databricks workspaces and compute resources for optimal performance and cost.
* Collaborate with cross-functional teams in an Agile environment to translate requirements into technical solutions.
Requirements
* You have experience with Python, PySpark, and SQL for data engineering.
* You possess in-depth knowledge of Databricks, Delta Lake, and Unity Catalog.
* You bring experience with AWS services including S3, IAM, KMS, and VPC.
* You're proficient in CI/CD and Infrastructure as Code using GitHub Actions, Terraform, and Databricks Asset Bundles.
* You have knowledge of medallion architecture, ETL/ELT patterns, and CDC.
* You possess the ability to work effectively in an Agile/Scrum environment.
* You are fluent in English and preferably Dutch as well.
Nice to Haves
* Experience with data quality frameworks such as Great Expectations.
* Familiarity with Lakehouse monitoring or CloudWatch dashboards.
Offer
* Start date: ASAP – long term engagement.
* Location: Gent/Zwijnaarde – hybrid (minimum 2 days/week onsite).
* Contract: open to both permanent employees and freelancers.
#J-18808-Ljbffr