As an AI Validation Engineer within a cross-functional team of Engineers, Data Scientists, and Product Owners, you will primarily focus on testing AI-driven chat applications and mobile apps, ensuring their reliability, performance, and seamless functionality. If you are passionate about seeing your ideas evolve from initial concept to shaping engaging AI-powered experiences for millions of users, Avrioc is the perfect place for you!
Location: Abu Dhabi, Abu Dhabi Emirate, United Arab Emirates
Work Conditions: On-site, Full-time
Requirements:
- Proven experience as an AI Validation Engineer with at least 2+ years dedicated to testing AI/ML systems.
- Bachelor's degree in computer science, Engineering, or a related technical field.
- Basic hands-on experience with Python coding.
- Proven expertise in AI/ML testing methodologies, including validation of models like NLP (especially LLM), CV, and predictive systems.
- Hands-on experience in automation testing frameworks such as PyTest, Selenium, Robot Framework, or equivalent.
- Strong background in prompt testing – ability to craft both functional and adversarial prompts to stress-test generative AI models.
- Solid understanding of VAPT (Vulnerability Assessment & Penetration Testing) techniques in AI/ML contexts.
- Experience in testing LLMs, including jailbreak of LLMs.
- Experience with data validation, bias/fairness audits, model drift detection, and performance benchmarking; familiarity with PyTest and Robot Framework.
- Knowledge of SQL or Shell scripting is a plus (minimal knowledge is acceptable).
- Willingness to participate in an on-call rotation, including handling high-priority, time-sensitive issues that may arise during off-hours.
- Ability to function under pressure and support real-time production environments.
- Clear communication skills to report issues, document findings, and coordinate with developers and AI researchers.
- Ability to generate reports based on testing results.
Nice to Have:
- Good appreciation for the Agile/Scrum methodology.
- Developer background would be advantageous.
Responsibilities & Authorities:
- Design and execute comprehensive test plans for AI-driven applications, including edge cases, performance stress, and failure modes.
- Perform rigorous prompt testing on LLMs and generative AI models to identify weaknesses and ensure behavior aligns with design intent.
- Conduct automated and manual testing to validate model inputs, outputs, and user interactions across platforms.
- Test security, data privacy, and compliance readiness of AI systems through VAPT and audit mechanisms.
- Continuously monitor live AI product performance, identify anomalies, and report bugs in a structured format.
- Support the AI engineering and data science teams by validating models, assisting in debugging, and ensuring stability in staging and production.
- Provide on-call support for critical issues, particularly during non-standard hours, with compensatory time off as per company policy.
- Collaborate in model explainability, fairness, and accountability testing, ensuring that all ethical and regulatory requirements are met.
If you meet the qualifications and are interested in this position, please apply or share your resume with us at careers@avrioc.com with the subject line: "Your Name | Years of Experience | Notice Period, Location" along with details like your current salary, expected salary, etc. Please note that only shortlisted candidates will be contacted.
Elevate your career and embark on a transformative journey with us.
