We are seeking a Data Engineer with 5-8 years of experience in designing and building data pipelines utilizing Apache Spark, Databricks, or equivalent big data frameworks. The ideal candidate should have hands-on expertise with streaming and messaging systems such as Apache Kafka, Confluent Cloud, RabbitMQ, or Azure Event Hub.
Ready to apply for roles like this?
Unlock the company name and direct application link. Subscribers get instant access to fresh jobs across Dubai, Abu Dhabi and Riyadh, many with visa support.
Unlock employer & apply directly
Key responsibilities include:
- Creating producers, consumers, and topics, integrating them into downstream processing.
- A deep understanding of relational databases and Change Data Capture (CDC) processes.
- Proficiency in SQL Server, Oracle, or other RDBMSs; with experience capturing change events using Debezium or native CDC tools.
- Strong skills in programming languages such as Python, Scala, or Java and solid knowledge of SQL for data manipulation and transformation.
- Familiarity with cloud platforms, including Azure or AWS services for data storage, compute, and orchestration (e.g., ADLS, S3, Azure Data Factory, AWS Glue, Airflow, DBX, DLT).
- Experience in data modeling and warehousing, with knowledge of Lakehouse architectures, Delta Lake, partitioning strategies, and performance optimization.
Additional requirements:
- Understanding of version control and DevOps practices with familiarity in Git and CI/CD pipelines, alongside the ability to automate deployment and manage infrastructure as code.
- Strong problem-solving and communication skills, with the capability to work collaboratively with cross-functional teams and articulate complex technical concepts to non-technical stakeholders.
Preferred/Bonus Skills:
- Experience with event-driven architectures and microservices integration.
- Exposure to NiFi, Flume, or other ingestion frameworks for connecting heterogeneous sources.
- Knowledge of graph processing or machine learning pipelines on Spark.
Location: Dubai, Dubai, United Arab Emirates
Work Conditions: Hybrid, Full-time
Language Requirements: Not specified.
