As a Pyspark Data Engineer, you will be responsible for designing, developing, and maintaining scalable data pipelines using PySpark for processing large volumes of structured and unstructured data. Key responsibilities include:
- Developing and managing ETL/ELT workflows using Informatica BDM.
- Collaborating with data architects, analysts, and business stakeholders to understand data requirements and translate them into technical solutions.
- Optimizing data processing jobs to ensure high performance and reliability.
- Implementing data quality, governance, and validation processes.
- Integrating data from multiple sources, including databases, APIs, and enterprise systems.
- Supporting data warehouse and data lake initiatives.
- Ensuring compliance with banking regulatory and security standards.
- Troubleshooting and resolving data pipeline issues and performance bottlenecks.
- Participating in code reviews, testing, and deployment processes.
Ready to apply for roles like this?
Unlock the company name and direct application link. Subscribers get instant access to fresh jobs across Dubai, Abu Dhabi and Riyadh, many with visa support.
Unlock employer & apply directly
Required Skills & Qualifications
- 6+ years of experience in Data Engineering or related roles.
- Strong hands-on experience with PySpark for large-scale data processing.
- Solid experience with Informatica BDM (Big Data Management).
- Experience with Big Data technologies and distributed data processing frameworks.
- Strong knowledge of SQL and data modeling concepts.
- Experience with data integration, ETL development, and workflow orchestration.
- Familiarity with cloud platforms or big data ecosystems (Hadoop/Spark environments) is a plus.
- Good understanding of data governance, security, and compliance requirements in banking.
- Strong analytical, problem-solving, and communication skills.
Domain Requirement
- Mandatory experience in the Banking or Financial Services domain, preferably with enterprise data platforms, regulatory reporting systems, or large-scale financial datasets.
Preferred Qualifications
- Experience with data lake architectures and data warehouse platforms.
- Exposure to Agile development methodologies.
- Knowledge of Python-based data engineering frameworks and automation.
Work Conditions: Hybrid, Full-time
Location: Dubai, Dubai, United Arab Emirates
