We are seeking a Senior QA Engineer to lead the design, testing, validation, and automation of QA strategies across our Data & AI programs. The role involves testing data pipelines, AI/ML models, Generative AI (LLM) applications, and multi-agent (Agentic AI) workflows, ensuring that solutions meet the highest standards of accuracy, reliability, fairness, security, and compliance.
Ready to apply for roles like this?
Unlock the company name and direct application link. Subscribers get instant access to fresh jobs across Dubai, Abu Dhabi and Riyadh, many with visa support.
Unlock employer & apply directly
Location: Riyadh, Saudi Arabia (On-site)
Work Conditions: Contract
Key Responsibilities:
- Define and enforce QA frameworks, processes, and standards for data pipelines, AI/ML models, GenAI/LLM applications, and agent-based AI systems.
- Develop and automate data validation tests (integrity, consistency, lineage, schema drift).
- Design and execute ML model validation tests, including performance, fairness, bias, and reproducibility.
- Implement automated prompt-response validation for GenAI/LLM applications (semantic similarity, factuality, hallucination detection).
- Integrate QA processes into CI/CD, MLOps, and LLMOps workflows.
- Develop monitoring and alerting systems for data drift, model decay, and pipeline failures.
- Define, track, and report on AI evaluation metrics.
- Ensure compliance with AI governance, ethics, security, and privacy requirements.
- Document QA processes, maintain reusable test cases, and support audits.
Skills & Qualifications:
- 5+ years of experience in Quality Assurance, with at least 2+ years focused on Data, AI/ML, GenAI, or Agentic AI testing.
- Proven experience testing LLM/GenAI applications (prompt testing, RAG pipelines, grounding validation, hallucination detection).
- Experience testing Agentic AI workflows (multi-agent orchestration, decision-making validation, safety guardrails).
- Strong Python skills and proficiency with testing frameworks (PyTest, unittest).
- Knowledge of ML evaluation metrics (precision, recall, F1, ROC, fairness, bias testing).
- Familiarity with AI governance, compliance, and privacy testing.
- Strong analytical skills, with attention to detail and quality.
- Excellent communication and collaboration skills for cross-functional work.
- Excellent writing and verbal communication skills in Arabic and English.
