About the job
We are seeking an experienced Data Engineer with strong AWS and banking sector experience to join its engineering team in the UAE. The candidate will be responsible for designing, developing, and optimising scalable Lakehouse data architectures and pipelines that support enterprise analytics and machine learning applications. This role requires strong expertise in AWS data services, Databricks, distributed data processing, and modern data engineering practices, alongside an understanding of financial datasets, banking regulations, and enterprise data governance.
Ready to apply for roles like this?
Unlock the company name and direct application link. Subscribers get instant access to fresh jobs across Dubai, Abu Dhabi and Riyadh, many with visa support.
Unlock employer & apply directly
Key Responsibilities
- Data Architecture: Design, develop, and maintain scalable Lakehouse architectures using Databricks and Delta Lake on AWS.
- Data Pipeline Development: Build and optimise batch and real-time ETL/ELT pipelines using Python, PySpark, SQL, and Apache Kafka to process financial and transactional data.
- AWS Infrastructure: Develop and manage data solutions using AWS services, including S3, Glue, EMR, Redshift, and Athena.
- Performance Optimisation: Improve data processing efficiency, query performance, and storage utilisation through partitioning, indexing, and distributed processing techniques.
- Machine Learning Integration: Collaborate with data science and MLOps teams to develop feature engineering pipelines, manage experiments using MLflow, and support model deployment through AWS SageMaker.
- Data Governance and Security: Implement data access controls, encryption, monitoring, and audit mechanisms in line with banking security, privacy, and regulatory requirements.
- Infrastructure Automation: Apply Infrastructure as Code practices using Terraform and manage CI/CD pipelines through GitHub Actions or AWS CodePipeline.
- Software Engineering Practices: Develop clean, maintainable, and well-tested Python code while following version control and development best practices.
- Client Collaboration: Engage with client stakeholders and cross-functional teams to understand technical requirements, present proposed solutions, and communicate architectural decisions effectively.
Required Qualifications and Experience
- Proven experience in data engineering within banking, financial services, or fintech environments.
- Strong understanding of financial datasets, transactional systems, and banking data governance and regulatory requirements.
- Extensive hands-on experience with AWS data services, particularly S3, Glue, EMR, and Redshift.
- Strong experience with Apache Spark, PySpark, and Apache Kafka for large-scale data processing.
- Demonstrated experience designing and implementing Lakehouse architectures using Databricks and Delta Lake.
- Solid understanding of data modelling, schema evolution, ACID transactions, and data warehousing principles.
- Proficiency in Python and SQL. Experience with Scala is an advantage.
- Practical experience with Docker, Kubernetes (EKS), Terraform, and Git-based workflows.
- Strong analytical and problem-solving skills.
- Excellent communication and stakeholder management skills, with the ability to explain technical concepts to both technical and non-technical audiences.
Preferred Qualifications
- AWS certifications such as AWS Certified Data Engineer or AWS Solutions Architect.
- Databricks Certified Data Engineer certification.
- Experience with real-time data processing frameworks such as Apache Flink or Spark Structured Streaming.
- Exposure to streaming analytics use cases, including fraud detection and trade processing.
- Experience with workflow orchestration tools such as Apache Airflow or Prefect.
- Experience deploying and maintaining production-grade data and machine learning solutions.
Core Competencies
In addition to technical expertise, the successful candidate should demonstrate:
- Client-Facing Communication: Confidence in engaging with senior stakeholders and presenting technical solutions.
- Business Acumen: Ability to understand business challenges and connect technical decisions to measurable business outcomes.
- Solution Ownership: Ability to take ownership of technical deliverables and advocate for effective engineering approaches.
- Collaboration: Strong ability to work across engineering, data science, and business teams.
- Presentation Skills: Confidence in explaining, demonstrating, and articulating the value of proposed technology solutions.
