Senior Data Engineer

About the job:

Client Introduction:
In this role, you will have the opportunity to work closely with one of our esteemed clients, known for its commitment to quality and innovation. They have chosen Dautom as their trusted partner for their upcoming projects.

Responsibilities:

• Identifying and extracting data from various sources (such as databases, APIs, logs, files, and external systems).
• Ensuring data quality during extraction and handling any anomalies.
• Transforming raw data into a structured format suitable for analysis and reporting.
• Cleaning, aggregating, and enriching data using ETL tools and techniques.
• Implementing business rules and data validation logic.
• Loading processed data into data warehouses, data lakes, or other storage systems.
• Designing and maintaining ETL pipelines to move data efficiently.
• Collaborating with database administrators to optimize data loading processes.
• Designing, developing, and maintaining ETL workflows and pipelines.
• Writing efficient ETL code (Informatica PowerCenter & Informatica BDM) to automate data movement.
• Ensuring scalability, reliability, and performance of ETL processes.
• Profiling and tuning ETL processes for optimal performance.
• Identifying bottlenecks and addressing them to improve data processing speed.
• Monitoring ETL jobs, identifying failures, and resolving issues promptly.
• Implementing error handling and logging mechanisms.
• Communicating effectively with cross-functional teams to understand data needs and ensure data consistency and accuracy.

Requirements:

• Strong ETL Tool Proficiency: Experience with ETL tools such as Informatica PowerCenter, Informatica BDM, and Apache NiFi.
• Database Knowledge: Proficiency in SQL and understanding of relational databases (e.g., PostgreSQL, MySQL, Oracle).
• Data Warehousing: Familiarity with data warehousing concepts and technologies.
• Big Data Technologies: Exposure to big data platforms (e.g., Hadoop, Spark) and NoSQL databases (e.g., Hbase, MongoDB).
• Data Governance and Security: Understanding of data governance practices and security protocols.
• Problem-Solving Abilities: Ability to troubleshoot complex data issues and optimize ETL processes.
• Excellent communication skills for technical and non-technical stakeholders.

Benefits and Perks:

• Competitive salary and bonus structure.
• Comprehensive health and wellness benefits.
• Opportunities for professional development and growth.
• Flexible work arrangements, including remote work options.
• Employee recognition programs and a collaborative team environment.