We are seeking an experienced Senior Data Engineer / Data Architect with a strong background in data streaming, Apache Kafka, and Airflow to join our dynamic data engineering team. This role will involve managing and enhancing a large-scale infrastructure designed for extensive social media data scraping, integration into a data lake house, and coordination across multiple data-centric teams.
Ready to apply for roles like this?
Unlock the company name and direct application link. Subscribers get instant access to fresh jobs across Dubai, Abu Dhabi and Riyadh, many with visa support.
Unlock employer & apply directly
Key Responsibilities:
- Design, develop, and maintain scalable data pipelines to support AI model development.
- Build and maintain efficient and reliable data pipelines using Apache Kafka, streaming services, and Airflow.
- Coordinate with the data acquisition team to ensure seamless data flow while addressing and resolving any security vulnerabilities in the Airflow and Kubernetes setup.
- Implement data solutions for large volumes of structured and unstructured data, including videos, audio, images, and text.
- Collaborate with AI researchers, machine learning engineers, and software engineers to ensure data is ready for model training.
- Ensure data quality, integrity, and security throughout the data lifecycle.
- Optimize data processing workflows for performance and scalability.
- Validate and manage multilingual data, focusing on Arabic and English datasets, especially YouTube data.
- Oversee data scraping projects from social media platforms under API constraints.
Requirements:
- Bachelor's or master's degree in computer science, Data Engineering, or a related field.
- 8+ years of experience in data engineering with a focus on building data pipelines.
- Proficiency in data processing technologies such as Apache Spark, Airflow, Kafka, Hadoop, and cloud platforms (AWS, GCP, Azure).
- Strong programming skills in Python, Java, Go, or Scala.
- Experience with SQL and NoSQL databases.
- Excellent problem-solving skills and attention to detail.
- Strong communication and teamwork skills.
Language Requirements:
- Proficiency in Arabic and English is preferred, especially for managing multilingual datasets.
What We Offer:
- An international workforce with diverse, multicultural values.
- A fair compensation package and a substantial annual leave of 25 days.
- Additional holidays for partners with a newborn.
- Medical insurance depending on the agreement.
- Opportunities for growth and development through our internal learning and development program.
- Team building activities and events.
Location: Riyadh, Saudi Arabia (On-site)
Work Conditions: Full-time
