Senior Data Engineer – AI Infrastructure & Data Pipelines
Qlik is seeking a high-performance Data Engineering builder to serve as a key execution partner in constructing our 2026 internal AI infrastructure. This role is dedicated to moving beyond traditional maintenance into foundational, creative engineering. You will be responsible for the end‑to‑end plumbing of our model orchestration layers and the migration of our enterprise data estate to an open, interoperable Lakehouse architecture.
As a "Customer Zero" engineer, you will use Qlik’s own Talend Cloud and Open Lakehouse products to build resilient, enterprise‑grade AI systems. You will work in a small, elite team where high‑speed innovation and a "fail‑fast" mentality are celebrated. This position requires a technically curious mind, ready to tackle the complexities of Apache Iceberg, the Model Context Protocol (MCP), and Knowledge Graph data flows for RAG.
Responsibilities
* Build a modern open Lakehouse architecture: Lead the migration of enterprise data including Salesforce and other core enterprise applications into an Apache Iceberg Lakehouse, implementing lake landing patterns, scalable ingestion strategies using CDC and incremental pipelines, schema governance, and the delivery of AI‑ready curated datasets.
* Design infrastructure connecting enterprise data and AI systems: Build, host, and secure Model Context Protocol (MCP) servers on Qlik’s SaaS platform, creating a robust bridge between internal data products and third‑party LLM agents.
* Engineer high‑performance RAG data pipelines: Develop and optimize pipelines supporting vector databases and retrieval systems, implementing semantic chunking, embedding generation, and CDC‑driven embedding updates to ensure data freshness and accuracy for AI workflows.
* Optimize performance and cost efficiency at scale: Leverage advanced Iceberg capabilities including the Adaptive Iceberg Optimizer to automate table compactions and metadata cleanup, improving query performance while reducing compute consumption.
* Build secure and governed AI data systems: Integrate security‑by‑design principles into agentic AI workflows, protecting enterprise data and mitigating risks such as confused deputy attacks and unauthorized data access.
* Work on cutting‑edge AI data architectures: Contribute to the design of knowledge graph pipelines, vector indexing systems, and context‑aware data flows that power autonomous AI agents and multi‑step agentic AI workflows.
* Collaborate with global AI engineering teams: Partner closely with the Principal Data Engineer and collaborate with engineering teams across the United States, India, and other global hubs to align local engineering execution with Qlik’s global AI strategy.
Impact
* Accelerate enterprise AI adoption by delivering robust pipelines and infrastructure that enable teams to deploy RAG‑powered applications and agentic AI workflows with reliable, context‑aware data.
* Migrate enterprise data to a scalable Lakehouse platform, ensuring scalable ingestion, schema governance, and high‑quality curated datasets.
* Improve data platform performance and efficiency by optimizing ingestion pipelines, compute usage, and lakehouse query performance.
* Deliver AI‑ready datasets at scale with semantic chunking, embeddings, and freshness guarantees.
* Maintain strong data governance and quality to ensure data accuracy, completeness, and consistency.
* Support global AI teams by delivering timely data pipelines and ensuring data usability for analytics and platform teams.
* Strengthen security and compliance across AI data systems following enterprise best practices.
Qualifications
* Deep experience with Apache Iceberg v2, manifest management, hidden partitioning, and schema evolution.
* Advanced proficiency in Snowflake (External Volumes, Open Catalog), Amazon S3, and AWS EC2.
* Mastery of Python (FastMCP, PySpark) and SQL optimization.
* Migrations from Salesforce and other enterprise applications using APIs, bulk exports, and CDC streams.
* Proficiency with Docker, Kubernetes, and Helm for deploying scalable containerized MCP servers.
* Expertise in semantic modeling for LLMs, including Knowledge Graph construction (Neo4j) and vector indexing.
* Knowledge of row‑level and column‑level security, PII protection, and enterprise data governance best practices.
* Fluent in English (Native or Professional) for global collaboration.
Location
Office Location, São Paulo, Brazil. Remote: Remote; Hybrid:
Benefits
* Career progression pathways and mentoring programs.
* Culture of innovation, technology, collaboration, and openness.
* Flexible, diverse, and international work environment.
#J-18808-Ljbffr