The Data Engineer will be responsible for designing, building, and maintaining the organization’s modern data infrastructure. This includes the development of scalable data pipelines and managing large, complex datasets within Azure Databricks and the broader Azure data ecosystem. Working in partnership with engineers, analysts, and data scientists, the Data Engineer will ensure data is reliable, accessible, and optimized for analytics and decision-making, while contributing to the ongoing evolution of the organization’s data platform.
Ready to apply for roles like this?
Unlock the company name and direct application link. Subscribers get instant access to fresh jobs across Dubai, Abu Dhabi and Riyadh, many with visa support.
Unlock employer & apply directly
Key Responsibilities:
- Design, develop, and maintain scalable data pipelines on Azure Databricks.
- Build robust workflows for both batch and streaming data pipelines using Spark and Delta Lake.
- Ingest and process data from diverse sources into Azure Data Lake Storage (ADLS) and Azure Synapse Analytics.
- Collaborate with cross-functional teams to define data models, transformation logic, and business rules.
- Apply data warehouse design principles, including data modeling and partitioning strategies.
- Work with structured and unstructured data formats.
- Implement data governance practices, such as cataloging, lineage tracking, and compliance.
- Monitor, troubleshoot, and optimize pipeline performance, reliability, and data quality.
- Automate workflows using orchestration tools like Azure Data Factory or Apache Airflow.
- Enforce data governance and security standards in alignment with organizational compliance requirements.
- Troubleshoot and resolve data-related issues and pipeline failures.
- Continuously drive process improvements for faster, more reliable data delivery.
Education & Experience Requirements:
- 4+ years of experience as a Data Engineer, Data Platform Engineer, or in a similar role.
- Bachelor's degree in Computer Science, Engineering, Information Systems, or a related field.
- Hands-on experience with Azure Databricks, Spark, and Delta Lake.
- Strong skills in Python, PySpark, and SQL for data engineering and scripting tasks.
- Experience with Azure Synapse, SQL Data Warehouse, or similar technologies.
- Proficient in working with Azure Data Lake (Gen2) for storage and data processing.
- Experience using Azure Data Factory for data orchestration.
- Familiarity with handling structured and semi-structured data formats (e.g., JSON, Parquet).
- Basic knowledge of Linux shell scripting and source control systems such as Git.
- Exposure to other cloud platforms such as AWS or GCP is a plus.
Work Conditions:
On-site, Full-time
Location:
Abu Dhabi, Abu Dhabi Emirate, United Arab Emirates
