Pyspark Data Engineer

As a Pyspark Data Engineer, you will be responsible for designing, developing, and maintaining scalable data pipelines using PySpark for processing large volumes of structured and unstructured data. Key responsibilities include:

• Developing and managing ETL/ELT workflows using Informatica BDM.
• Collaborating with data architects, analysts, and business stakeholders to understand data requirements and translate them into technical solutions.
• Optimizing data processing jobs to ensure high performance and reliability.
• Implementing data quality, governance, and validation processes.
• Integrating data from multiple sources, including databases, APIs, and enterprise systems.
• Supporting data warehouse and data lake initiatives.
• Ensuring compliance with banking regulatory and security standards.
• Troubleshooting and resolving data pipeline issues and performance bottlenecks.
• Participating in code reviews, testing, and deployment processes.

Required Skills & Qualifications

• 6+ years of experience in Data Engineering or related roles.
• Strong hands-on experience with PySpark for large-scale data processing.
• Solid experience with Informatica BDM (Big Data Management).
• Experience with Big Data technologies and distributed data processing frameworks.
• Strong knowledge of SQL and data modeling concepts.
• Experience with data integration, ETL development, and workflow orchestration.
• Familiarity with cloud platforms or big data ecosystems (Hadoop/Spark environments) is a plus.
• Good understanding of data governance, security, and compliance requirements in banking.
• Strong analytical, problem-solving, and communication skills.

Domain Requirement

• Mandatory experience in the Banking or Financial Services domain, preferably with enterprise data platforms, regulatory reporting systems, or large-scale financial datasets.

Preferred Qualifications

• Experience with data lake architectures and data warehouse platforms.
• Exposure to Agile development methodologies.
• Knowledge of Python-based data engineering frameworks and automation.

Work Conditions: Hybrid, Full-time
Location: Dubai, Dubai, United Arab Emirates