Data Engineer

About the job

Technical Lead

Qualification

Bachelors in science, engineering or equivalent

Responsibility

• Project Planning and Setup:

  • Understand the project scope, identify activities/tasks, task level estimates, schedule, dependencies, risks and provide inputs to Module Lead for review.
  • Provide inputs to testing strategy, configuration, deployment, hardware/software requirement etc.
  • Review plan and provide feedback on gaps, timeline and execution feasibility etc as required in the project.
  • Participate in KT sessions conducted by customer/ other business teams and provide feedback on requirements.

• Requirement Understanding And Analysis:

  • Analyze functional/non functional requirements and seek clarifications for better understanding of requirements.
  • Based on understanding of system upstream & downstream, provide feedback and inputs on gaps in requirements and technical feasibility of requirements.

• Design:

  • Prepare the LLD/detailed design documents based on HLD and briefing from Module Lead.
  • Seek inputs from the developers on specific modules as applicable.
  • Consolidate all modules and provide to Module Lead/ Architects/ Designers for review.
  • Suggest changes in design on technical grounds.

• Develop components inventory for the code:

  • To be developed tying it to the nonfunctional requirements.
  • Perform sampling of data to understand the character/ quality of the data (project dependent in the absence of data analyst or designer).
  • Identify tools and technologies to be used in the project as well as reusable objects that could be customized for the project.

• Coding:

  • Follow coding standards and best practices to develop code and check code quality.
  • Share developed code with supervisor for review.
  • Rework on the code based on inputs if required.
  • Perform complex integration.
  • Guide the developers in identifying, preparing and conducting unit test cases and fixing defects based on results.
  • Consolidate the test results and share with supervisor.
  • Provide periodic status update to supervisor and highlight/recommend any changes in design based on challenges faced.
  • Anticipate unreported defects and raise the same to supervisor.
  • Conduct technical troubleshooting.
  • Conduct reviews for codes created by team.

• Testing Management:

  • Develop unit test case for each module.
  • Conduct/guide conducting of unit and integration testing and fix defects.
  • Review/approve code to be moved to testing environment.
  • Provide support to the QA team and coordinate for various phases of testing.
  • Address queries raised by QA within defined timelines.
  • Investigate critical defects and establish need for fixing.
  • Raise issues to leads/QA.
  • Report defect status as per project standard process within agreed timelines.
  • Share revised code with supervisor for review.
  • Assist team lead and project manager on estimates around defect fixes.

• Configuration Management:

  • Maintain versions of the code or consolidate version maintained by the Developers.
  • Provide support as required to the Administrators during configuration, code backups, deployment etc.

• Deployment:

  • Assess and create deployment/ roll back plan.
  • Validate if all the components have been migrated and the right version is checked in.
  • Maintain deployment tracker.
  • Perform sanity checks post deployment to ensure smooth production.
  • Share activity status with supervisor and highlight concerns if any.

• Project Execution Monitoring & closure (Support to Project Management activities):

  • Monitor work of the developers and share work achieved with them.
  • Provide guidance through SDLC.
  • Provide status of progress to leads.
  • In case of change requests, provide inputs on the plan.

• Service Support And MaintenanceSpecific to production and maintenance support:

  • Provide support for 1 week and hand over to production team.
  • Identify if the incoming request is a service request/ defect during the warranty or an incident.
  • If it is a code defect in the warranty, highlight to Lead and initiate defect fix process.
  • Post warranty, support in transition to maintenance team.

• Knowledge Management:

  • Post release participate in project review call and discuss points on what went well and what didn't.
  • Create and update knowledge articles (case studies, lessons learnt) in the knowledge management repository.
  • Guide developers in creating such documents.
  • Publish white papers/blogs/articles (if required).

• People Management:

  • Conduct training through academy or internally within the team.
  • Conduct technical, face to face interviews for internal transfer or external hiring.
  • Provide feedback on Developers form technical/domain standpoint to the module lead.

Must Have Skills:

• Spark in Scala
• SQL
• Azure DevOps
• Python
• New Prod Development-T&H

Good To Have Skills:

• Synapse Pipelines
• Azure CLI
• Azure PowerShell
• Azure Synapse SQL Pool
• Azure Synapse Spark Pool
• Structured Streaming
• Kafka
• Azure Monitor
• Azure Data Lake Store
• Azure Key Vault
• PySpark
• Azure Event Hub