About the Role
As a Data Engineer, you will design and deliver end-to-end data products across various business domains. Your responsibilities will span the bronze, silver, and gold layers, including the creation of dimensional and semantic models for reporting and AI integration. You will be instrumental in building robust data ingestion from various enterprise systems and leading the migration of analytical workloads to Databricks. Your role also encompasses the implementation of data quality frameworks, governance policies, and performance tuning strategies to ensure operational excellence and cost efficiency.
Ready to apply for roles like this?
Unlock the company name and direct application link. Subscribers get instant access to fresh jobs across Dubai, Abu Dhabi and Riyadh, many with visa support.
Unlock employer & apply directly
Key Responsibilities
Data Product Design & Delivery
- Design and deliver end-to-end data products for assigned business domains across the bronze, silver, and gold layers, including the dimensional and semantic models that serve reporting and AI.
- Build robust ingestion from enterprise systems, including Oracle Fusion (Finance, HCM, P2P, CRM), Hyperion, CMiC, and others, using change data capture, incremental, and API-based patterns.
- Lead workstreams within the Synapse Analytics to Databricks migration: re-engineering pipelines, running parallel loads, reconciling results, and preparing for cutover.
- Contribute reusable frameworks, templates, and patterns, and propose improvements to the standards owned by the Data Engineering Lead (Data Architect).
Data Quality, Governance & Security
- Implement data quality frameworks, monitoring, and reconciliation across your domains, and drive issues to resolution at source with data owners.
- Implement access control, lineage, classification, and data-handling rules in Unity Catalog and Microsoft Purview, including PDPL controls for personal and HR data.
- Partner with the Data Governance Office to turn domain policies into working controls and documented, certified datasets.
Performance, Reliability & Cost
- Tune Spark and SQL workloads for performance and cost within the platform's cluster and workload policies.
- Own the operational health of your domains: monitoring, alerting, incident response, and root-cause fixes.
- Apply CI/CD, automated testing, and DataOps practices using Git and Azure DevOps.
Analytics & AI Enablement
- Build AI-ready data products — feature tables and document and vector pipelines — for machine learning, RAG, and agent use cases, in partnership with the Data Scientist and AI Engineer.
- Deliver Power BI semantic models and certified datasets with the Data Analyst, supporting self-service adoption through the Digital Academy.
- Apply AI-assisted engineering — code generation, testing, documentation, and lineage discovery — with appropriate human review, and help the team adopt it safely.
Collaboration & Mentoring
- Translate requirements from finance, project, and operations stakeholders into technical designs with clear delivery outcomes.
- Mentor Data Engineers through pairing, design reviews, and code review.
- Explain design choices, trade-offs, and risks clearly to technical and business audiences.
Qualifications
Required Qualifications
- 5+ years of hands-on data engineering experience, including at least 2 years with Databricks, Apache Spark, and Delta Lake in production.
- Designed and delivered end-to-end data products — from ingestion to modeled, consumable data — for one or more business domains.
- Experience integrating complex enterprise sources such as Oracle Fusion, SAP, Microsoft Dynamics, or CMiC, including incremental and API-based extraction.
- Data quality and access controls implemented in practice using Unity Catalog, Microsoft Purview, or equivalent tooling.
- Strong SQL, Python, and PySpark skills, with Git-based CI/CD delivery (Azure DevOps or equivalent).
- Excellent written and verbal communication skills across technical and business audiences.
Certifications Required (held, or achieved after joining within the period shown):
- Databricks Certified Data Engineer Associate or Microsoft Certified: Azure Databricks Data Engineer Associate (DP-750) — within 6 months
- Databricks Certified Data Engineer Professional — within 12 months
Preferred Certifications (one or more):
- Databricks Certified Generative AI Engineer Associate
- Microsoft Certified: Fabric Data Engineer Associate (DP-700)
- Microsoft Certified: Fabric Analytics Engineer Associate (DP-600)
- Microsoft Certified: Azure Data Engineer Associate (DP-203 — retired 2025; accepted if held)
