About the Job:
The main objective of this role is to design, build, and maintain scalable data architectures and pipelines using AWS services. The focus will be on efficiently ingesting, processing, and transforming data to ensure high quality and accessibility for analytics and reporting.
Ready to apply for roles like this?
Unlock the company name and direct application link. Subscribers get instant access to fresh jobs across Dubai, Abu Dhabi and Riyadh, many with visa support.
Unlock employer & apply directly
Key Responsibilities:
- Design, build, and maintain scalable data pipelines to ingest and process data from various sources.
- Leverage Amazon Web Services (AWS) tools such as AWS Glue, Amazon S3, Amazon Redshift, and Amazon RDS to manage data pipeline, storage, transformation, and analytics.
- Build and maintain a cloud framework for orchestrating data pipelines, continuously optimizing it to integrate new data sources effectively.
- Develop Extract, Transform, Load (ETL) processes to ensure data is cleaned, transformed, and loaded into data warehouses, utilizing scripting languages such as PySpark, Python, and Scala for data transformations.
- Automate data integration and processing tasks to enhance efficiency and reduce manual intervention.
- Set up monitoring tools to track data pipeline performance and troubleshoot issues.
- Maintain comprehensive documentation of data architectures, processes, and workflows to ensure updates are consistent with each release.
- Follow defined Data Governance policies and guidelines, enforcing team compliance with standard procedures.
- Ensure data security and compliance with relevant regulations, including GDPR or industry-specific guidelines.
- Lead junior data engineers, provide technical guidance, and manage work allocation and delivery.
- Mentor junior data analysts, collaborate with cross-functional teams, implement data governance practices, and optimize performance to support informed decision-making within the organization.
Qualifications:
- Bachelor's Degree in Computer Science, Information Systems, Engineering, or a related field.
- Fluent in English.
- Minimum of 8 years of industry experience, with 5 years focused on designing and developing data pipelines at an enterprise scale.
- Proven experience in building streaming and batch pipelines for structured and semi-structured data.
- Solid experience in handling large volumes of data and performing transformations using scripts such as PySpark, Scala, and Python.
- Experience working with AWS cloud services including EMR, Lambda, MKS, Redshift, and Glue.
- Preference for candidates with experience in the airline industry.
Competencies:
- Customer Focus
- Teamwork
- Effective Communication
- Personal Accountability & Commitment to Achieve
- Resilience & Flexibility
Language Requirements:
Fluent in English.
