About the Role
In this builder's role, you will develop and enhance internal AI systems that will significantly improve the workflow of our engineering teams. This position focuses on creating agents that automate testing processes, triage failures, expedite code reviews, and develop evaluation infrastructures necessary for testing AI features. Your performance will be assessed based on the adoption and usability of the systems you deliver.
Ready to apply for roles like this?
Unlock the company name and direct application link. Subscribers get instant access to fresh jobs across Dubai, Abu Dhabi and Riyadh, many with visa support.
Unlock employer & apply directly
What Will You Do
Test automation across the stack
- Backend: API and contract testing, service-level and integration coverage, maintaining data setups, and creating robust test designs that withstand schema changes.
- Frontend: Implement web E2E and component-level coverage with Playwright, conduct visual and RTL regression, ensuring testing suites are fast enough for merge gating rather than nightly runs.
- Mobile: Cover native and cross-platform applications using Appium or Maestro, strategize device farms, test offline and sync behavior, and address payment-peripheral paths that require real hardware.
- The connective tissue: Manage shared fixtures, test data, environment management, parallelization, and CI pipelines where build failures are actionable.
- Conduct performance and load testing as required.
The AI layer on top of it
- Generate tests from specs, code, and production traffic, ensuring a solid maintenance strategy is in place.
- Implement failure triage to classify build failures as real bugs, flakes, environment issues, or test rot before human intervention.
- Develop self-healing locators and suite health tools, including flake detection and coverage-gap analysis.
- Build evaluation infrastructure for AI features, focusing on datasets, scoring, and regression detection related to prompt or model modifications.
- Evaluate market-specific behavior for Arabic and English, particularly regarding RTL interfaces and local POS, tax, and payment regulations, ensuring correctness goes beyond mere string matches.
Agentic AI and orchestration
- Foster agentic AI that performs actionable tasks within pipelines, like reading diffs, running relevant suites, reproducing failures, proposing fixes, and opening pull requests.
- Develop agents responsible for quality workflows, including exploratory testing against live builds, coverage-gap hunting, and assessing release risks while knowing when to escalate to a human.
- Ensure orchestration can withstand high loads, featuring multi-step planning, tool utilization, retries, sustained state management, sandbox execution, and clean handoffs between agentic and deterministic processes.
- Integrate seamlessly with existing stack elements (CI, Jira, observability) instead of creating parallel systems.
- Discern when a typical pipeline is preferable to agentic approaches and articulate that judgment clearly.
The technical ground
- Stay informed on contemporary practices in test automation, including framework design, the test pyramid's limitations, flake economics, and CI/CD gating, with experience using frameworks like Playwright, Appium, or Maestro.
- Engage deeply with agentic AI concepts, orchestration, multi-step planning, memory/state handling, and tool integration standards.
- Master context engineering principles, including retrieval strategies and managing long-context behavior degradation.
- Conduct evaluations using offline and online methods, understanding LLM functions and their limitations, human-in-the-loop reviews, and statistical significance checks on small samples.
- Ensure reliability through structured outputs, guardrails, fallback design, and managing non-determinism in critical systems.
- Maintain operations knowledge related to tracing and observability in LLM systems, prompt management, and model routing, alongside awareness of when to prefer fine-tuning over improved prompts.
What Are We Looking For
- An engineer who has a proven track record of shipping production software and possesses recent hands-on experience with LLM-backed systems that actual users rely on.
- Strong skills in Python; familiarity with at least one of .NET, Java, or TypeScript. Experience in maintaining tested production code outside notebooks is essential.
- Comprehensive automation experience across multiple layers. You should have been responsible for suites that gate releases both on the backend and on user interfaces (web or mobile) and demonstrate how you ensured suite reliability.
- Demonstrated experience in building evaluation systems, capable of explaining how you measure system improvement through quantifiable metrics.
- In-depth familiarity with the modern LLM toolkit, including prompting, structured output, tool utilization, retrieval, and orchestration of agentic AI, along with an understanding of the associated trade-offs.
- Solid testing fundamentals; while a QA title isn’t necessary, you should be conversant with test design, automation frameworks, and CI/CD principles.
- A noteworthy inclination towards adoption; your success should be gauged by how much engineers utilize your contributions rather than how well you can present them.
What We Offer You
- An inclusive and diverse culture promoting innovation and offering flexibility with both in-office and hybrid work options.
- Highly competitive compensation packages, including bonuses and potential stock options.
- A commitment to personal development, featuring regular training sessions and an annual learning stipend to take on new challenges in a rapidly growing environment.
- Join an accomplished team comprising over 30 nationalities across 14 countries, gaining invaluable experience in an exciting industry.
- Opportunities for autonomy, mentorship, and challenging goals that present exceptional growth possibilities for you and the company.
