We are seeking a highly skilled and versatile Data Engineer with expertise in building robust data solutions using the Microsoft Data Platform and hands-on experience in AI/ML model development. This role focuses on designing, developing, and maintaining scalable and efficient data pipelines and architectures while also contributing to the development and deployment of AI scoring engines and predictive models to support advanced analytics. The ideal candidate will have deep expertise in Azure cloud services, data governance, data modeling, and advanced tools like Microsoft Fabric and Databricks, with a complementary focus on AI/ML workflows and MLOps.
Ready to apply for roles like this?
Unlock the company name and direct application link. Subscribers get instant access to fresh jobs across Dubai, Abu Dhabi and Riyadh, many with visa support.
Unlock employer & apply directly
Responsibilities
-
Design & Develop Data Pipelines
- Build, optimize, and maintain ETL/ELT pipelines to ingest, transform, and process large volumes of structured and unstructured data from diverse sources.
- Leverage Azure Data Factory, Azure Data Lake, Azure Synapse Analytics, Microsoft Fabric, or Databricks for scalable data integration and transformation.
-
Data Modeling & Architecture
- Design and implement data models and schemas optimized for analytics, machine learning, and AI-driven decision-making.
- Create and manage data warehouses, data lakes, and lakehouses to support data analytics and AI workloads.
-
Data Governance & Security
- Ensure data quality, compliance, and security by implementing governance frameworks and leveraging tools like Microsoft Purview.
- Enforce data security protocols, including role-based access control, data masking, and encryption.
-
AI/ML Data Integration
- Collaborate with data scientists to integrate data pipelines with machine learning workflows, enabling seamless training and deployment of models.
-
Performance Optimization
- Monitor and optimize the performance of data pipelines and cloud resources to ensure high availability, scalability, and cost efficiency.
Qualifications
- Education: Bachelor’s or Master’s in Computer Science, Data Engineering, Data Science, or related field.
- 6–10 years of experience in data engineering, with a strong focus on Microsoft Data Platform technologies.
- 4+ years of hands-on experience in AI/ML model development, including building and deploying scoring engines.
Technical Skills
- Advanced knowledge of Azure Data Factory, Azure Synapse Analytics, Azure Data Lake, Microsoft Fabric, Databricks, and SQL Server.
- Proficient in Python and SQL for data engineering and ML tasks.
- Experience with distributed data processing frameworks like Apache Spark for handling large-scale datasets.
- Strong understanding of MLOps best practices, including model versioning, monitoring, and retraining.
Certifications
- Azure Data Engineer Associate
- Azure AI Engineer Associate
- Databricks Certified Data Engineer Associate
- Microsoft Fabric Certification
This position offers the opportunity to work at the intersection of data engineering and artificial intelligence to drive meaningful business outcomes.
