We are looking for an experienced Data Engineer to join the Data & AI team, focusing on delivering core data platform capabilities. This hands-on engineering role involves designing and building robust Python-based data pipelines, ETL processes, and data models across a modern data platform. The ideal candidate will be a proficient Python developer with solid SQL skills and a good understanding of data lake architecture. Exposure to data quality tools (e.g., Informatica, Microsoft Purview) is considered a valuable plus.
Ready to apply for roles like this?
Unlock the company name and direct application link. Subscribers get instant access to fresh jobs across Dubai, Abu Dhabi and Riyadh, many with visa support.
Unlock employer & apply directly
Key Responsibilities:
- Design, build, and maintain end-to-end data pipelines using Python, ensuring reliable data ingestion, transformation, and delivery across the platform.
- Develop and optimize ETL/ELT processes to move and transform data across bronze, silver, and gold layers following the Medallion architecture pattern.
- Write clean, modular, production-grade Python code for data processing, orchestration, and automation tasks.
- Support design and implementation of data models, schemas, and storage strategies.
- Build and maintain SQL-based transformations, stored procedures, and views for data validation and reconciliation.
- Develop and manage data ingestion frameworks from various source formats (flat files, APIs, databases, streaming).
- Implement data quality checks and validations within pipelines, ensuring completeness, validity, consistency, uniqueness, accuracy, and timeliness.
- Monitor pipeline health, build alert mechanisms, and troubleshoot data issues in production environments.
- Contribute to CI/CD pipelines for data workloads, including automated testing, deployment, and version control.
- Produce clear technical documentation for pipelines, data models, and operational runbooks.
Required Skills & Experience:
- Strong Python development skills with hands-on experience building production data pipelines (e.g., pandas, PySpark).
- Solid SQL skills for complex queries, data transformations, and performance tuning.
- Experience designing and implementing ETL/ELT processes at scale.
- Good understanding of the Medallion architecture (Bronze/Silver/Gold) and modern data lake/lakehouse design patterns.
- Familiarity with data orchestration tools.
- Working knowledge of cloud data platforms (Azure, AWS, or GCP).
- Familiarity with relational and non-relational databases.
- Strong problem-solving skills to debug complex data pipeline issues.
Desirable Skills:
- Exposure to data quality tools such as Informatica IDQ, Microsoft Purview, or similar frameworks.
- Experience with data cataloging and lineage tools.
- Familiarity with data governance platforms for policy management and stewardship workflows.
- Knowledge of data visualization tools like Power BI.
Location: Abu Dhabi Emirate, United Arab Emirates
Work Conditions: On-site, Contract
For more details, please contact: moumita.p@the company.
