As a Senior Data Engineer at the company, you will be responsible for building and maintaining the infrastructure that supports data collection, processing, and storage. You will work closely with data scientists, analysts, and other stakeholders to ensure that data systems are reliable, scalable, and secure, enabling data-driven decision-making across the organization. This key technical role focuses on developing and optimizing the company's data infrastructure. Your responsibilities will include designing and implementing data pipelines, ensuring data quality, and collaborating with cross-functional teams to support various data initiatives.
Ready to apply for roles like this?
Unlock the company name and direct application link. Subscribers get instant access to fresh jobs across Dubai, Abu Dhabi and Riyadh, many with visa support.
Unlock employer & apply directly
Key Responsibilities:
- Data Pipeline Architecture and Development: Design, construct, install, test, and maintain highly scalable data pipelines focused on machine learning models and analytics.
- Data Integration: Collaborate with data scientists, ML engineers, and stakeholders to ensure data accessibility, consistency, and reliability.
- API and Data Services: Develop and maintain APIs for data access and manipulation, integrating with external data services as needed.
- Data Storage: Manage and optimize data storage solutions for both structured (relational databases) and unstructured data (Text, Image, Audio, Video, Search Engines like Elasticsearch, and NoSQL databases).
- Data Quality and Governance: Implement processes to monitor data quality and ensure production data is accurate and available.
- Collaboration and Support: Assist ML engineers with data-related technical issues and provide architectural guidance.
- Security and Compliance: Ensure compliance with data security and privacy policies.
- Documentation: Maintain clear documentation including data dictionaries, metadata, and architectural diagrams.
Qualifications:
- Bachelor’s degree in Computer Science, Engineering, Mathematics, or a related field; or equivalent work experience.
- 7+ years of experience in a Data Engineering role.
- Proficiency in programming languages like Python and SQL with experience in managing large-scale data (Terabyte to Petabyte).
- Hands-on experience with big data technologies like Spark (using PySpark/Scala) and Flink.
- Familiarity with machine learning frameworks such as TensorFlow, PyTorch, or similar.
- Strong understanding of data warehousing or Lake-house concepts, ETL processes, and data modeling.
- Experience with API development and integration with data services.
- Experience with cloud platforms like Azure.
- Knowledge of DevOps, CI/CD methods, and containerization technologies like Docker or Kubernetes.
- Experience with real-time/streaming data processing.
Technical Stack:
- Programming Languages: Python, SQL
- Query Engine: Trino
- Big Data Technologies: Spark, Flink
- Unstructured Data: Text, Image, Audio & Video
- Databases: Clickhouse, MySQL, PostgreSQL, MongoDB, Cassandra, HBase, Redis
- Cloud Platforms: Azure
- API Development: RESTful APIs, GraphQL, OpenAPI
- Data Services: Kafka, RabbitMQ
- Containers: Docker, Kubernetes
Language Requirements: (Not mentioned)
