We are seeking a skilled Data Engineer to join our Data & Analytics team. In this role, you will design and build robust, scalable data pipelines and well-structured data models to support advanced analytics and Power BI dashboards. You will work with sales data from our stores and deliver high-quality, reliable datasets for business stakeholders.
Responsibilities:
· Design, develop, and maintain efficient and scalable data pipelines to ingest, transform, and load sales data.
· Build and optimize data models to power Power BI dashboards and analytics use cases.
· Collaborate with analysts, data scientists, and business teams to understand data requirements and translate them into technical solutions.
· Ensure high data quality, integrity, and performance across the entire pipeline lifecycle.
· Work within a modern data stack including Databricks, Azure Data Services, and Unity Catalog.
· Use Terraform to manage infrastructure as code and GitHub for version control and CI/CD workflows.
· Apply best practices in data engineering, including modular design, testing, and documentation.
Required Skills and Qualifications:
· Proficient in Python and SQL for data transformation and development.
· Experience with Spark (especially PySpark) for distributed data processing.
· Strong working knowledge of Databricks, Azure Data Lake, and Azure Data Factory.
· Experience implementing Unity Catalog for data governance and cataloging.
· Familiarity with Terraform for infrastructure automation.
· Solid experience working with GitHub in a collaborative development environment.
· Deep understanding of data modeling concepts (e.g., dimensional modeling, star/snowflake schemas).
· Proven experience working with sales or transactional data.
· Ability to build and maintain clean, performant data layers for Power BI dashboards.