Pyspark Data Engineer

We have an immediate job opportunity for a Pyspark Data Engineer position in Dubai, United Arab Emirates, requiring on-site, full-time commitment. The successful candidate will possess 6 to 14 years of commercial experience in a data-driven role, with at least 5+ years specifically in building data pipelines in production environments. Experience with both structured and unstructured data is essential.

Key responsibilities include:

• Developing and maintaining data pipelines.
• Working with ML pipelines for streaming and batch workflows.
• Writing clean, maintainable, and robust code in Python.
• Utilizing analytics techniques such as descriptive, predictive, and exploratory data analysis.
• Applying feature engineering and anomaly detection methods.
• Utilizing Python visualization libraries (e.g., matplotlib, seaborn).
• Engaging with querying languages and database technologies, as well as CI/CD practices for testing and validation.

Required skills:

• Python
• Spark (PySpark)
• Jupyter
• SQL and No-SQL databases
• Git (for source code versioning and CI/CD)
• Exploratory Data Analysis (EDA)
• Data Imputation Techniques
• Data Linking/Cleansing
• Feature Engineering
• Apache Airflow/Jenkins for scheduling and automation

Qualifications:

• Bachelor’s or Master’s degree in computer science, Information Systems, or a related field.

Soft skills:

• Excellent communication and leadership skills.
• Strong interpersonal and collaboration abilities.
• Ability to work under pressure and meet tight deadlines.
• Positive attitude and strong work ethic.

Language requirements:

• English proficiency is assumed as a base requirement for this role.

The company embraces diversity and inclusion and is an equal opportunity workplace. We encourage applicants from diverse backgrounds to apply.