As a Data Engineer at G42, you will be responsible for building and maintaining the infrastructure that supports data collection, processing, and storage. In this role, you will report to the Lead Data Engineer or Principal Data Engineering, collaborating closely with data scientists, analysts, and stakeholders to ensure data systems are reliable, scalable, and secure. Your contributions will play a vital role in enabling data-driven decision-making across the organization.
The Data Engineer position is a key technical role focused on developing and optimizing the company's data infrastructure. Responsibilities include designing and implementing data pipelines, ensuring data quality, and working with cross-functional teams to support various data initiatives.
Key Responsibilities:
- Design, construct, install, test, and maintain highly scalable data pipelines focused on machine learning models and analytics.
- Work closely with data scientists and ML engineers to ensure data accessibility, consistency, and reliability.
- Develop and maintain APIs for data access and manipulate data, integrating with external services as necessary.
- Manage and optimize data storage solutions, including relational databases and NoSQL databases.
- Implement processes to monitor data quality and ensure the accuracy of production data for stakeholders.
- Collaborate with ML engineers to address data-related technical issues and provide architectural guidance.
- Ensure compliance with data security and privacy policies.
- Maintain clear documentation including data dictionaries and architectural diagrams.
Requirements:
- Bachelor’s degree in Computer Science, Engineering, Mathematics, or a related field, or equivalent work experience.
- 3+ years of experience in a Data Engineering role.
- Proficiency in SQL and programming languages such as Python, Java, and Scala.
- Hands-on experience with big data technologies like Hadoop, Spark, and Flink.
- Familiarity with machine learning frameworks like TensorFlow or PyTorch.
- Strong understanding of data warehousing concepts and ETL processes.
- Experience with API development and cloud platforms like AWS or GCP.
- Knowledge of DevOps practices and containerization technologies like Docker or Kubernetes.
- Experience with real-time data processing.
Skills and Technical Stack:
- Programming Languages: Python, Java, Scala, SQL, Bash
- Big Data Technologies: Hadoop, Spark, Flink
- Databases: MySQL, PostgreSQL, MongoDB, Cassandra
- Cloud Platforms: Azure
- API Development: RESTful APIs, GraphQL
- Data Services: Kafka, RabbitMQ
- Containers: Docker, Kubernetes
If you are a performance-driven individual with a passion for data and a desire to build meaningful collaborations, you will thrive in this role. G42 offers a competitive remuneration package with numerous perks, a supportive work culture, and opportunities for personal growth and professional development.
