Pyspark Data Engineer

We are seeking a Pyspark Data Engineer with expertise in Data Architecture, Data Strategy, and Roadmap for large and complex organizations and systems. The role involves implementing large-scale end-to-end Data Management & Analytics solutions. Candidates should have experience in transforming traditional Data Warehousing approaches to Big Data-based methodologies and managing risks and data security.

Key responsibilities include:

• Expertise in Data Quality, Data Profiling, Data Governance, Data Security, Metadata Management, and Data Archival.
• Definition of workload migration strategies using appropriate tools.
• Driving delivery in a matrixed environment collaborating with various internal IT partners.
• Working independently while managing multiple tasks/projects and adhering to short deadlines.
• Demonstrating strong problem-solving, influencing, communication, and presentation skills.
• Hands-on experience with data processing frameworks and platforms (Hadoop, Presto, Tez, Hive, Spark, etc.).
• Proficiency in programming languages and tools (Linux, Python, PySpark, GIT, Jenkins).

Desired Skills and Experience:

• Expertise in PySpark.
• Experience in cloud-native principles, designs, and deployments.
• Comprehensive knowledge of DW Dimensional modeling techniques, including Star and Snowflake schemas, managing slowly changing dimensions, and classification.