Senior Data Engineer

About the Job:
As a Senior Data Engineer, you will lead and manage a backend/distributed systems team while working with third-party resources. Your key responsibilities will include building and optimizing Java, MapReduce, Hive, and Spark jobs, and utilizing a wide range of Hadoop tools such as HDFS, Pig, Hive, HBase, Sqoop, and Flume. You will be tasked with implementing and managing real-time stream processing systems using Spark Streaming and Storm, as well as developing dimensional data models and performing advanced SQL tuning. Ensuring source data integrity and accurate data ingestion will be part of your role, in addition to building dashboards and BI solutions following best practices. You will collaborate with internal teams and vendors to prioritize and deliver critical data initiatives and will also deploy, monitor, and audit big data models and workflows.

Key Responsibilities:

• Lead and manage a backend/distributed systems team and third-party resources.
• Build and optimize Java, MapReduce, Hive, and Spark jobs.
• Work with a wide range of Hadoop tools: HDFS, Pig, Hive, HBase, Sqoop, Flume.
• Implement and manage real-time stream processing using Spark Streaming, Storm.
• Develop dimensional data models and perform advanced SQL tuning.
• Analyze source data integrity and ensure accurate data ingestion.
• Build dashboards and BI solutions using best practices.
• Collaborate with internal teams and vendors to prioritize and deliver data initiatives.
• Deploy, monitor, and audit big data models and workflows.

Technical Skills:

• Strong hands-on experience with Hadoop (Cloudera), Hive, Pig, Impala, Spark.
• Proficient in programming with Java, Python, Scala, and scripting (Linux, Ruby, PHP).
• Experience with NoSQL and SQL databases: Cassandra, Postgres.
• Familiarity with cloud services (Azure preferred).
• Exposure to machine learning/data science tools is a plus.

Work Conditions:

• On-site, Full-time

Location:

• Dubai, United Arab Emirates

Language Requirements:

• Not specified.