About the job
Oversee and manage various aspects of data integration and data warehouse processes. This includes implementing data quality measures, designing and deploying integration solutions, collaborating with team members, troubleshooting production issues, refining best practices, educating end users, participating in code reviews, exploring, and adopting new technologies, leading prototype development, and ensuring adherence to industry standards. The role involves working within Agile teams, reporting on project status, and partnering with IT Solution Architects to optimize data integration designs. The goal is to contribute to efficient and effective data analytics integration solutions while staying informed about technological trends and mentoring team members.
Job Responsibilities
Operational Work
- Implement Data Warehouse data quality processes
- Provide solutions where issues are identified
- Design, develop, test, and deploy data integration solutions based on business requirements
- Collaborate with team members on data integration designs and alternatives
- Troubleshoot and resolve production issues
- Continually refine and document data integration best practices
- Educate end users (where applicable) in the use of metadata and understanding where data originates
- Participate in code reviews
- Identify, evaluate, and refine leading-edge data integration tools and methods
- Build prototypes and POCs
- Bring a passion to stay on top of tech trends, experiment with and learn new technologies, participate in internal & external technology communities, and mentor other members of the engineering community
- Own the development of cross-functional, multi-platform prototypes
- Work within and across Agile teams to design, develop, test, implement, and support data analytics integration solutions
- Document and support deployment activities
- Report on status of IT Data Operations products and projects on a regular and timely basis
- Ensure the timely resolution of issues
- Follow standards and guidelines according to industry best practices
- Escalate anticipated risks to management promptly and properly
- Partner with IT Solution Architects regarding data integration design options
Strategic Work
- Educate and inform yourself and team members about data integration technology solutions and emerging trends
- Continually participate in visioning exercises and contribute to the data strategy
Candidate Requirements
Required years of relevant experience:
5+ years of experience designing, developing, and maintaining data integration solutions
Desired Qualifications:
Education - B.S. in Computer Science or related discipline. Advanced degree a plus
Specialized Skills:
Required:
Data integration development experience with the following skill sets:
- Extract, Transformation, and Load (ETL) of disparate data sources to targets including databases, files, and API’s. Talend, Matillion, Glue Data Studio preferred
- Ability to detect, identify, and resolve source to target performance/load issues
- Expert level proficiency in structured query language (SQL)
- Thorough understanding of relational data models including third normal form and fact/dimension designs
- Change data capture (CDC) design options and application
- Data marts
- Oracle database platform features including indexing, partitioning, and PL/SQL
- AWS Data Migration System (DMS), Amazon Redshift, Aurora Postgres Expertise including but not limited to performance tuning
- Linux O/S commands and scripting
- Secure File Transfer Protocol (SFTP)
- Desire to work in the data integration space (data warehouse, data lake, AWS, data API’s) while continuing to grow skill sets
Desired:
Modern development practices – e.g. DevOps
Experience with open source database platforms: PostgreSQL, MySQL, MariaDB
Data Lake integration design and development (ETL & ELT Methodologies)
Experience with data quality tools
Experience with AWS platform: DMS, Glue, Redshift, S3, etc
AWS Certifications
Experience in API development
Master data management integration and concepts
Experience with job scheduling solutions such as Airflow
