Lead Software Engineer

About the Job:
Lead Software Engineers in Dubai, UAE are expected to design, develop, and maintain scalable data pipelines using technologies like PySpark for processing large volumes of structured and unstructured data. This role involves developing and managing ETL/ELT workflows using Informatica BDM and collaborating with data architects, analysts, and business stakeholders to translate data requirements into technical solutions.

Key responsibilities include optimizing data processing jobs for high performance, implementing data quality and governance processes, integrating data from multiple sources, and supporting data warehouse and data lake initiatives. Ensuring compliance with banking regulatory and security standards, troubleshooting pipeline issues, and participating in code reviews and testing are also part of the role.

Required Skills & Qualifications:

• 6+ years of experience in Data Engineering or related roles.
• Strong hands-on experience with PySpark for large-scale data processing.
• Proficiency with Informatica BDM (Big Data Management).
• Experience with Big Data technologies and distributed data processing frameworks.
• Strong knowledge of SQL and data modeling concepts.
• ETL development and workflow orchestration experience.
• Familiarity with cloud platforms or big data ecosystems (Hadoop/Spark environments) is a plus.
• Good understanding of data governance, security, and compliance in banking.
• Strong analytical, problem-solving, and communication skills.

Domain Requirement:

• Mandatory experience in the Banking or Financial Services domain, preferably with enterprise data platforms or regulatory reporting systems.

Preferred Qualifications:

• Experience with data lake architectures and data warehouse platforms.
• Exposure to Agile development methodologies.
• Knowledge of Python-based data engineering frameworks and automation.