Description:
Tasks:
* Develop and update a data architecture strategy that adapts to evolving needs and accommodates both Business Intelligence and AI workloads.
* Design and implement architectures for the cloud that are vendor agnostic.
* Design a modern scalable data platform to replace a large legacy data system in a phased approach.
* Align architectural decisions with data governance policies and the department’s vision on cloudification.
* Establish and enforce data management policies and processes, including data quality, security and platform health monitoring.
* Ensure regulatory compliance and adherence to audit requirements.
* Provide guidance and mentorship to data analysts and data engineers.
* Facilitate change management by guiding colleagues and users through the migration process.
* Document and maintain data architecture and data assets in detail.
* Assistance with deployment, configuration and testing of the system.
* Participation in meetings with other project teams.
Skills Required:
* Experience in migrating legacy data systems to a modern cloud-based, open-source data platform solution (preferably Data Lakehouse).
* Excellent knowledge of designing scalable and flexible modern cloud-based and open-source data architectures.
* Experience with AI-powered assistants like Amazon Q for innovative data solutions design.
* Strong understanding of Kubernetes.
* Knowledge of database systems, both relational (PostgreSQL, Oracle) and non-relational (Elasticsearch, MongoDB).
* Experience with ETL/ELT processes and related data ingestion and transformation tools (like Spark, dbt, Trino).
* Proficiency in data pipeline orchestration tools (like Airflow, Dagster, Luigi).
* Knowledge of data governance frameworks and tools (like DataHub, Open Metadata), data quality management, data security, access control, and regulatory compliance.
* Proficiency with system-to-system integration via RESTful APIs.
* Experience with DevSecOps practices and tools related to data pipelines, including CI/CD for data infrastructure.
* Excellent communication skills to articulate data architecture concepts to technical and non-technical stakeholders. Good knowledge of modeling tools.