Data Engineer

As a Data Engineer, you will design and deploy scalable data pipelines and intelligent AI solutions using Azure Agent Foundry, OpenAI, LangChain, and Databricks. The role involves building RAG pipelines, vector databases, and knowledge graphs, as well as maintaining APIs and supporting AI analytics workloads. You will optimise ML workflows on Azure Databricks, ensure data quality and governance, and collaborate with cross-functional teams.

Responsibilities

• Develop and deploy AI agents using Azure Agent Foundry, OpenAI, LangChain, and N8N.
• Build RAG pipelines, embeddings, vector database integrations, and knowledge graph solutions.
• Design and maintain scalable data pipelines, ETL/ELT workflows, and data architectures.
• Build and maintain APIs and data services supporting AI and analytics workloads.
• Ensure data quality, reliability, governance, and compliance across all systems.
• Collaborate with data scientists, ML engineers, and product teams to deliver intelligent features.
• Support MLOps and DevOps processes including CI/CD, model lifecycle management, and containerized deployments.

Requirements

• Strong proficiency in Python, SQL, PySpark, and ML/AI frameworks.
• Experience with Azure Agent Foundry, OpenAI APIs, LLMOps, RAG systems, and vector databases.
• Hands-on experience with Azure Databricks, Azure cloud services, and data platforms.
• Knowledge of big data tools (Spark, Hadoop, Flink) and modern data storage (SQL, NoSQL).
• Experience with REST/GraphQL APIs, Kafka/RabbitMQ, Docker/Kubernetes.
• Solid understanding of data modeling, ETL, and data quality best practices.
• Bachelor’s in Computer Science, Engineering, Data Science, or related field.
• 3+ years experience in Data Engineering, AI Engineering, or ML Engineering roles.
• Experience delivering scalable AI/data solutions in cloud environments (Azure preferred).

Work Conditions

• Hybrid work policy to balance office and home.
• Competitive remuneration package with additional perks including healthcare and education support.