Pyspark - Data Architect

Unlock employer Dubai, United Arab Emirates Direct to Company 19 Feb 2025

Apply Direct

Financial

Estimate: $80k - $120k*
Zero income tax location

Accessibility

Hybrid
Apply from abroad
Visa Provided

Requirements

Experience: Intermediate

Explore similar roles:

View Data Architect jobs in Dubai · View all Data Architect jobs

Position

Location: Dubai, Dubai, United Arab Emirates
Work Conditions: Hybrid, Full-time

Ready to apply for roles like this?

Unlock the company name and direct application link. Subscribers get instant access to fresh jobs across Dubai, Abu Dhabi and Riyadh, many with visa support.

Unlock employer & apply directly

About the Job:
As a Pyspark - Data Architect, you will be responsible for designing, developing, and maintaining scalable and optimized ETL pipelines, managing data ingestion, transforming and processing large datasets, and ensuring data quality. You will tune performance, automate workflows, and collaborate with teams to support data-driven initiatives while maintaining comprehensive documentation.

Role: Associate Architect
Key Skills: Pyspark, Big Data – Hadoop, Hive, Impala, Kafka

Qualifications:

Education: Bachelor’s or Master’s degree in Computer Science, Data Engineering, Information Systems, or a related field.
Experience: 3+ years as a Data Engineer, focusing on PySpark and the Cloudera Data Platform.
Technical Skills:
- Advanced proficiency in PySpark, including RDDs, DataFrames, and optimization techniques.
- Strong experience with Cloudera Data Platform (CDP) components, including Cloudera Manager, Hive, Impala, HDFS, and HBase.
- Knowledge of data warehousing concepts, ETL best practices, and experience with SQL-based tools (e.g., Hive, Impala).
- Familiarity with Hadoop, Kafka, and distributed computing tools.
- Experience with orchestration frameworks like Apache Oozie and Airflow.
- Strong scripting skills in Linux.