As part of the implementation of a Common Data Platform. A central validation engine, developed in Python on Databricks, will execute shared rules to measure and ensure the quality of the data entering the CDP. This engine is a shared component across multiple stakeholders, and its success relies on both a robust technical design and close collaboration between organizations.
Within this context, we are looking for a Senior Software Engineer who will act as the technical and architectural owner of the validation engine. He/she will guarantee technical consistency, provide supervision, and ensure stakeholder alignment around this key component.
Functie
This role is part of the Nihdi Data Office, the cornerstone of all data-related activities within the institution. The Data Office ensures efficient, ethical, and compliant collection, integration, sharing, and use of data, both within Nihdi and with Belgian and European institutions and medical insurances. By joining, you will integrate a multidisciplinary agile team composed of data engineers, reporting experts, functional analysts, project managers, data architects, data quality specialists, and AI experts.
The Data Office relies on a modern technology stack, including MS SQL Server, SSIS, SSAS, Enterprise Architect (or similar), SAS, Azure, Power BI, and Databricks. It manages the central Data Warehouse (Kimball methodology), which feeds a large number of Data Marts and Reports.
Functie-eisen
This is a long-term, strategic project extending well beyond 2025, with a clear vision to support the sustainability and effectiveness of the Common Data Platform.
Roles & responsibilities:
Define the technical and software architecture of the validation engine (Python / Databricks).
Establish the functional design for the developments in full collaboration with the Data Governance team, the Technical Leads and the Program Manager.
Explore and define the business requirements of the different stakeholders.
Ensure alignment and consultation with all business/technical stakeholders (Nihdi, IMA, Mutualities).
Supervise the dedicated developer
Play also an active role in testing the data products developed.
Challenge and guide technical choices for the implementation of the rules (JSON, SQL or alternatives).
Ensure performance, robustness, and scalability of the engine in a Cloud context.
Document architecture decisions, standards, and best practices.
Oversee smooth integration of the engine into the CDP and its pipelines (Azure / Databricks).
Act as the technical reference point for all topics related to the validation engine.
Key skills
Master's degree in Computer Science or equivalent.
Minimum 10 years of experience in software engineering (architecture, design, supervision) especially in back-end.
Proven experience with Python and Databricks (3 years) / Spark environments.
Knowledge of rule-based approaches.
Excellent communication and stakeholder management skills.
Strong expertise in ETL/ELT (8 years).
Strong experience in Data Quality and Data Validation (5 years).
Solid background in Cloud environments, ideally Azure (3 years).
Maitrise of software architecture, design patterns (5 years).
Ability to understand and document business processes and to translate business requirements into technical requirements.
Ability to challenge and mentor a developer (Python code, SQL integration, performance).
Experience in multi-stakeholder or inter-organizational projects is a strong asset.
BPMN (3 years).
Excellent command of French/Dutch/English.
Profile
Technical leadership combined with strong diplomacy skills.
Acts as guarantor of technical quality without being hands-on in development.
Excellent interpersonal skills to align multiple stakeholders.
Rigorous, strategic mindset with a strong focus on quality.
Strong affinity with data quality and the industrialization of validation processes.
Job summary
As a Senior Software Engineer / Architect specialized in Data Quality, you will be responsible for the design and technical supervision of the validation engine, developed in Python on Databricks.
You will not define business rules (handled by the Functional Analyst) nor directly develop (done by the dedicated developer), but you will act as the guarantor of the architecture, technical supervision, and inter-organizational alignment.
This strategic role is designed as a long-term mission, well beyond 2025, to support the sustainability and evolution of the CDP.
What we offer
An interesting job in the realization of SDO (ICT) and Data services that has an impact on society, with many advantages:
A good work-life balanceflexible working hours within the 40-hour week (7.30 a.m. to 8 p.m. with fixed slots between 9 a.m. and 12 p.m. and 1 p.m. and 4 p.m.)
possibility of teleworking (In Belgium)
workspaces organized according to activities
easy accessibility via public transport
car and bike parking in the building
individual locker
Several opportunities for self-developmenttraining and apprenticeship opportunities (to be taken outside working hours) via our e-academy self-learning platform or via the Microsoft Enterprise Skills Initiative platform
Company restaurant