About the Role
The company is seeking an experienced Lead Data Engineering / Platform Operations professional to lead the engineering, operations, reliability, and continuous evolution of the enterprise data platform. This role will be responsible for building and operating scalable data pipelines, managing the Databricks Lakehouse platform, driving DataOps best practices, and enabling analytics, AI, regulatory reporting, and operational intelligence across the company. Working closely with Enterprise Architecture, Data Modelling, AI Engineering, and Infrastructure teams, you will ensure the platform remains secure, governed, reliable, and cost-efficient.
Ready to apply for roles like this?
Unlock the company name and direct application link. Subscribers get instant access to fresh jobs across Dubai, Abu Dhabi and Riyadh, many with visa support.
Unlock employer & apply directly
What You Will Do
- Lead the design, development, and operation of enterprise-scale data ingestion pipelines across batch, micro-batch, and real-time streaming environments.
- Engineer and manage the Databricks Lakehouse platform, including Delta Lake optimisation, performance tuning, and platform reliability.
- Implement and govern Unity Catalog, including data security controls, lineage tracking, access management, and compliance requirements.
- Build and maintain dbt transformation frameworks, testing standards, documentation, and CI/CD integration.
- Drive DataOps excellence through automated testing, monitoring, observability, incident management, and platform support processes.
- Manage Confluent/Kafka streaming and CDC capabilities, ensuring resilient and scalable real-time data movement.
- Partner with AI Platform Engineering teams to support feature pipelines, vector data services, and AI-powered data products.
- Own platform FinOps activities, optimizing cloud spend across Databricks, Azure Data Factory, Confluent, and storage services.
- Ensure compliance with data security, privacy, governance, and regulatory requirements.
- Lead, coach, and develop a high-performing team of Data Engineers while establishing engineering best practices and standards.
What We Are Looking For
- 10+ years of experience in Data Engineering, including at least 3 years in a technical leadership capacity.
- Strong hands-on expertise with Databricks, including Delta Lake, Unity Catalog, Databricks Workflows, PySpark, and Spark SQL.
- Proven experience with Azure Data Factory (ADF) for enterprise data integration and orchestration.
- Deep experience with Confluent/Kafka, CDC technologies, schema management, and event-driven architectures.
- Strong knowledge of dbt, automated testing frameworks, DataOps practices, and CI/CD pipelines.
- Experience managing platform reliability, data quality, observability, and incident response.
- Understanding of cloud cost optimisation, FinOps, and platform governance.
- Ability to collaborate effectively with Data Architects and Data Modelers while translating architecture into scalable engineering solutions.
- Bachelor's degree in Computer Science, Data Engineering, Information Technology, or a related discipline.
- Databricks Certified Data Engineer Professional and/or Azure Data Engineer certifications are highly desirable.
