Data Engineer

We are seeking a Data Engineer with 5-8 years of experience in designing and building data pipelines utilizing Apache Spark, Databricks, or equivalent big data frameworks. The ideal candidate should have hands-on expertise with streaming and messaging systems such as Apache Kafka, Confluent Cloud, RabbitMQ, or Azure Event Hub.

Key responsibilities include:

• Creating producers, consumers, and topics, integrating them into downstream processing.
• A deep understanding of relational databases and Change Data Capture (CDC) processes.
• Proficiency in SQL Server, Oracle, or other RDBMSs; with experience capturing change events using Debezium or native CDC tools.
• Strong skills in programming languages such as Python, Scala, or Java and solid knowledge of SQL for data manipulation and transformation.
• Familiarity with cloud platforms, including Azure or AWS services for data storage, compute, and orchestration (e.g., ADLS, S3, Azure Data Factory, AWS Glue, Airflow, DBX, DLT).
• Experience in data modeling and warehousing, with knowledge of Lakehouse architectures, Delta Lake, partitioning strategies, and performance optimization.

Additional requirements:

• Understanding of version control and DevOps practices with familiarity in Git and CI/CD pipelines, alongside the ability to automate deployment and manage infrastructure as code.
• Strong problem-solving and communication skills, with the capability to work collaboratively with cross-functional teams and articulate complex technical concepts to non-technical stakeholders.

Preferred/Bonus Skills:

• Experience with event-driven architectures and microservices integration.
• Exposure to NiFi, Flume, or other ingestion frameworks for connecting heterogeneous sources.
• Knowledge of graph processing or machine learning pipelines on Spark.

Location: Dubai, Dubai, United Arab Emirates
Work Conditions: Hybrid, Full-time
Language Requirements: Not specified.