As an AI Validation Engineer within a cross-functional team of Engineers, Data Scientists, and Product Owners, you will primarily focus on testing AI-driven chat applications and mobile apps, ensuring their reliability, performance, and seamless functionality. If you are passionate about seeing your ideas evolve from initial concept to shaping engaging AI-powered experiences for millions of users, Avrioc is the perfect place for you!
Ready to apply for roles like this?
Unlock the company name and direct application link. Subscribers get instant access to fresh jobs across Dubai, Abu Dhabi and Riyadh, many with visa support.
Unlock employer & apply directly
Requirements:
- Proven experience as an AI Validation Engineer with at least 2+ years dedicated to testing AI/ML systems.
- Bachelor's degree in computer science, Engineering, or a related technical field.
- Basic hands-on experience with Python coding.
- Proven expertise in AI/ML testing methodologies, including validation of models like NLP (specifically LLM), CV, and predictive systems.
- Hands-on experience in automation testing frameworks such as PyTest, Selenium, Robot Framework, or equivalent.
- Strong background in prompt testing – ability to craft both functional and adversarial prompts to stress-test generative AI models.
- Solid understanding of VAPT (Vulnerability Assessment & Penetration Testing) techniques in AI/ML contexts.
- Experience in testing LLMs and jailbreak of LLMs required.
- Experience with data validation, bias/fairness audits, model drift detection, and performance benchmarking.
- Knowledge of SQL, or Shell scripting is a plus (minimal knowledge is fine).
- Willingness to participate in an on-call rotation, including handling high-priority, time-sensitive issues that may arise during off-hours.
- Ability to function under pressure and support real-time production environments.
- Clear communication skills to report issues, document findings, and coordinate with developers and AI researchers.
- Must be able to generate reports based on testing results.
Responsibilities:
- Design and execute comprehensive test plans for AI-driven applications, including edge cases, performance stress, and failure modes.
- Perform rigorous prompt testing on LLMs and generative AI models to identify weaknesses and ensure behavior aligns with design intent.
- Conduct automated and manual testing to validate model inputs, outputs, and user interactions across platforms.
- Test security, data privacy, and compliance readiness of AI systems through VAPT and audit mechanisms.
- Continuously monitor live AI product performance, identify anomalies, and report bugs in a structured format.
- Support the AI engineering and data science teams by validating models, assisting in debugging, and ensuring stability in staging and production.
- Provide on-call support for critical issues, particularly during non-standard hours, with compensatory time off as per company policy.
- Collaborate in model explainability, fairness, and accountability testing, ensuring all ethical and regulatory requirements are met.
For interested candidates, please apply with your resume to [email protected] with the subject line: "Your Name | Years of Experience | Notice Period, Location" along with details like your current and expected salary. Only shortlisted candidates will be contacted.
Work Conditions:
- On-site
- Full-time
- Participation in on-call rotation required
