Data Engineer

About the Role / About the Job
You will be responsible for assessing complex new data sources and quickly turning these into business insights. You also will support the implementation and integration of these new data sources into our Azure Data platform.

Responsibilities

• Collect progress updates from Squad on a regular basis.
• Consolidate updates into weekly project status reports and share with the Squad Leads for their review.
• Participate in early state design and feature definition activities.
• Implement robust data pipelines using Microsoft and the Databricks Stack.
• Create reusable and scalable data pipelines.
• Collaborate with team members across multiple engineering teams to support the integration of proven prototypes into core intelligence products.
• Convey complex data insights to non-technical stakeholders effectively through strong communication skills.
• Develop and manage L3-level project plans with sufficient details for tasks, milestones, and dependencies.

Requirements

Skills

• Knowledge and experience with relational and non-relational databases.
• Strong analytic skills related to working with structured and unstructured datasets.
• Hands-on experience in Azure Databricks utilizing Spark to develop ETL pipelines.
• Strong proficiency in data analysis, manipulation, and statistical modeling using tools like Spark, Python, Scala, SQL, or similar languages.
• Strong proficiency in building JIRA dashboards based on tasks.
• Strong experience in Project Management will be considered an additional advantage.
• Experience with Azure DevOps to deploy data pipelines through CI/CD.

Qualifications

• Minimum 5-7 years of practical experience.
• Bachelor’s degree in computer science, software engineering, information technology, or a related field.
• Azure cloud stack in-production experience.