As a Senior Data Engineer, you will be responsible for building and maintaining the infrastructure that supports data collection, processing, and storage. Reporting to the Lead Data Engineer or Principal Data Engineering, you will work closely with data scientists, analysts, and other stakeholders to ensure that data systems are reliable, scalable, and secure. Your work will be crucial in enabling data-driven decision-making across the organization.
The Data Engineer is a key technical role focused on developing and optimizing the company's data infrastructure. This involves designing and implementing data pipelines, ensuring data quality, and collaborating with cross-functional teams to support various data initiatives. The ideal candidate will have strong technical skills in data engineering, experience with data architectures, and a passion for working with data.
Key Responsibilities:
- Design, construct, install, test, and maintain highly scalable data pipelines with a focus on machine learning models and analytics.
- Work closely with data scientists, ML engineers, and stakeholders to ensure that data is accessible, consistent, and reliable.
- Develop and maintain APIs for data access and manipulation, and integrate with external data services as needed.
- Manage and optimize data storage solutions for both structured and unstructured data.
- Implement processes to monitor data quality and ensure production data is always accurate and available for key stakeholders.
- Collaborate with ML engineers to assist in data-related technical issues and provide architectural guidance.
- Ensure compliance with data security and privacy policies.
- Maintain clear and up-to-date documentation including data dictionaries, metadata, and architectural diagrams.
Qualifications:
- Bachelor’s degree in Computer Science, Engineering, Mathematics, or a related field; or equivalent work experience.
- 7+ years of experience in a Data Engineering role.
- Proficiency in programming languages such as Python, Java, and Scala.
- Hands-on experience with big data technologies like Hadoop, Spark, and Flink.
- Familiarity with machine learning frameworks such as TensorFlow or PyTorch.
- Strong understanding of data warehousing concepts, ETL processes, and data modeling.
- Experience with API development and integration with data services.
- Experience with cloud platforms like Azure.
- Knowledge of DevOps, CI/CD methods, and containerization technologies like Docker or Kubernetes.
- Experience with real-time or streaming data processing.
Technical Stack:
- Programming Languages: Python, Java, Scala, SQL, Bash
- Big Data Technologies: Hadoop, Spark, Flink
- Databases: MySQL, PostgreSQL, MongoDB, Cassandra, HBase, Redis
- Cloud Platforms: Azure
- API Development: RESTful APIs, GraphQL, OpenAPI
- Data Services: Kafka, RabbitMQ
- Containers: Docker, Kubernetes
What We Offer:
- A diverse and inclusive environment with a global vision that encourages personal growth and innovation.
- Outstanding learning and development opportunities via structured training programs and high-tech projects.
- A hybrid work policy to balance office and home life.
- A competitive remuneration package with numerous perks, including healthcare, education support, leave benefits, and more.
If you meet the criteria and are eager to thrive in an innovative environment, please contact us as soon as possible.
