Software Quality Engineer

We are seeking a Senior QA Engineer to lead the design, testing, validation, and automation of QA strategies across our Data & AI programs. The role involves testing data pipelines, AI/ML models, Generative AI (LLM) applications, and multi-agent (Agentic AI) workflows, ensuring that solutions meet the highest standards of accuracy, reliability, fairness, security, and compliance.

Location: Riyadh, Saudi Arabia (On-site)
Work Conditions: Contract

Key Responsibilities:

• Define and enforce QA frameworks, processes, and standards for data pipelines, AI/ML models, GenAI/LLM applications, and agent-based AI systems.
• Develop and automate data validation tests (integrity, consistency, lineage, schema drift).
• Design and execute ML model validation tests, including performance, fairness, bias, and reproducibility.
• Implement automated prompt-response validation for GenAI/LLM applications (semantic similarity, factuality, hallucination detection).
• Integrate QA processes into CI/CD, MLOps, and LLMOps workflows.
• Develop monitoring and alerting systems for data drift, model decay, and pipeline failures.
• Define, track, and report on AI evaluation metrics.
• Ensure compliance with AI governance, ethics, security, and privacy requirements.
• Document QA processes, maintain reusable test cases, and support audits.

Skills & Qualifications:

• 5+ years of experience in Quality Assurance, with at least 2+ years focused on Data, AI/ML, GenAI, or Agentic AI testing.
• Proven experience testing LLM/GenAI applications (prompt testing, RAG pipelines, grounding validation, hallucination detection).
• Experience testing Agentic AI workflows (multi-agent orchestration, decision-making validation, safety guardrails).
• Strong Python skills and proficiency with testing frameworks (PyTest, unittest).
• Knowledge of ML evaluation metrics (precision, recall, F1, ROC, fairness, bias testing).
• Familiarity with AI governance, compliance, and privacy testing.
• Strong analytical skills, with attention to detail and quality.
• Excellent communication and collaboration skills for cross-functional work.
• Excellent writing and verbal communication skills in Arabic and English.