About the Job:
We are hiring for projects in the Middle East, where our team consists of frontend and backend developers, data analysts, data scientists, architects, and project managers. As a Lead Data Engineer, you will manage the development, implementation, and maintenance of data infrastructure for a modern data platform.
Ready to apply for roles like this?
Unlock the company name and direct application link. Subscribers get instant access to fresh jobs across Dubai, Abu Dhabi and Riyadh, many with visa support.
Unlock employer & apply directly
Responsibilities:
- Platform Design: Collaborate with cross-functional teams to refine and design the architecture of our data platform.
- Platform Optimization: Continuously enhance the performance, scalability, and reliability of the data platform to meet evolving business requirements.
- Platform Maintenance: Implement monitoring, alerting, and maintenance processes to ensure the availability and health of the data platform.
- Data Governance: Establish and enforce data governance best practices to uphold data quality, security, and compliance standards.
- Platform Integration: Integrate new technologies seamlessly into the data platform ecosystem, ensuring compatibility and interoperability.
- Collaboration: Work closely with data engineers, data scientists, analysts, and business stakeholders to comprehend requirements and offer technical expertise and support.
- Documentation: Create and maintain comprehensive documentation for the data platform architecture, processes, and workflows.
- Mentorship: Provide guidance and mentorship to junior members of the data engineering team.
Technical Responsibilities:
- Design, develop, and maintain data pipelines for data ingestion and transformation using Databricks.
- Implement declarative ETL pipelines and optimize data models and pipelines for performance, scalability, and reliability.
- Ensure data governance and security best practices are followed.
Skills Required:
- Overall 5-7 years of experience in data engineering and transformation on Cloud.
- 3+ Years of strong experience in Azure Data Engineering and Databricks.
- Experience in PySpark and developing workloads for Spark distributed computing.
- Preferably a degree in Computer Science/IT, Engineering (Computer/Telecommunication), or equivalent.
- Languages: English (C2 Proficient)
Desired Experience:
- Experience in implementing platform and application monitoring using cloud-native tools.
- Familiarity with DevOps practices for continuous integration and deployment.
Seniority: Lead
Location: Abu Dhabi Emirate, United Arab Emirates
Work Conditions: On-site, Full-time
