We are seeking an experienced Spark Developer to design and implement the Gold layer of our Data Warehouse using the Medallion Architecture. The ideal candidate will have strong expertise in Apache Spark and a solid understanding of data engineering best practices.
Key Responsibilities:
Develop and optimize Spark-based data pipelines for the Gold layer.
Ensure data quality, performance, and scalability in the data warehouse architecture.
Collaborate with data analysts to deliver business-ready datasets.
Implement transformations and aggregations aligned with business requirements.
Technical skills:
Minimum 3 years of experience in
Airflow
Delta Lake
Java
Spark
Data warehousing