Data Engineer (PySpark)

Unlock employer Dubai, United Arab Emirates Direct to Company 10 Apr 2025

Apply Direct

Financial

Estimate: $80k - $120k*
Zero income tax location

Accessibility

Hybrid
Apply from abroad
Visa Provided

Requirements

Experience: Intermediate

Explore similar roles:

View Data Engineer jobs in Dubai · View all Data Engineer jobs

Position

About the Job:
As a Data Engineer, you will play a pivotal role in designing, developing, and maintaining highly scalable ETL pipelines using PySpark on the Cloudera Data Platform. You will ensure data integrity and manage data ingestion processes from various sources into our data lake or warehouse. Your expertise will support analytical needs and business requirements through the transformation and processing of large datasets, while also optimizing performance and ensuring data quality.

Ready to apply for roles like this?

Unlock the company name and direct application link. Subscribers get instant access to fresh jobs across Dubai, Abu Dhabi and Riyadh, many with visa support.

Unlock employer & apply directly

Key Skills: PySpark, Cloudera Data Platform, Big Data - Hadoop, Hive, Kafka

Responsibilities:

Design, develop, and maintain scalable ETL pipelines using PySpark.
Implement and manage data ingestion processes from diverse sources.
Process, cleanse, and transform large datasets to meet analytical needs.
Optimize performance of PySpark code and Cloudera components.
Implement data quality checks to ensure accuracy throughout the pipeline.
Automate workflows using orchestration tools like Apache Oozie or Airflow.

Technical Skills:

3+ years as a Data Engineer, focusing on PySpark and Cloudera Data Platform.
Proficient in PySpark, including RDDs and DataFrames.
Experience with Cloudera Manager, Hive, Impala, HDFS, and HBase.
Knowledge of data warehousing concepts and SQL-based tools.
Familiarity with Big Data technologies such as Hadoop and Kafka.
Experience with orchestration frameworks like Apache Oozie and Airflow.
Strong scripting skills in Linux.

Location: Dubai, Dubai, United Arab Emirates

Work Conditions: Hybrid, Full-time

About Virtusa:
Virtusa is one of the fastest growing IT services companies in the Middle East, with a client base in the UAE, KSA, Qatar, and Oman. The company collaborates with leading firms across various sectors and has received recognition for its work from industry leaders like Gartner and IDC. Virtusa values teamwork, quality of life, and professional development, offering growth opportunities and exciting projects for a global workforce of over 30,000.

Apply Direct