Test Job- Head of Product Engineering
Design and implement QA foundations and evaluation frameworks for assessing AI agents, their workflows and processes.
Design and run experiments to assess AI-driven QA processes, including prompt strategies, automation workflows, and agent performance.
Benchmark systems across multiple engineering streams to determine accuracy, reliability, cost, and consistency of outputs.
Develop approaches for creating synthetic datasets and structured test data to support reliable evaluation.
Identify gaps in existing testing approaches and design AI-based solutions or agents to address them.
Build proof-of-concepts (POCs) to explore new testing approaches and validate ideas.
Structure and document product knowledge, including user journeys, APIs, and business rules, to support AI-assisted testing systems.
Evaluate the cost, ROI, and effectiveness of different QA approaches and automation strategies.
Collaborate closely with the QA Lead and engineering teams to integrate AI-assisted QA into development workflows.
Ensure AI-driven testing processes produce consistent, reliable, and high-quality results.
Remain hands-on in building tooling, experiments, and testing infrastructure.
Strong background in SDET, test automation.
Experience building and maintaining automated testing frameworks and QA tooling.
Hands-on experience building proof-of-concepts or experimental systems.
Experience evaluating or working with AI systems, LLMs, or agent-based workflows.
Ability to design structured experiments and evaluation methodologies.
Strong understanding of testing fundamentals, including UI and API automation tools such as Cypress, Playwright, or similar.
Experience analysing systems and identifying opportunities to improve processes through automation or AI.
Ability to work hands-on while also contributing to process design and strategic improvements.
Experience working in AI-focused companies or startups.
Hands-on experience building or experimenting with AI agents or AI-driven automation workflows in Quality Assurance.
Experience designing evaluation frameworks for AI systems.
Experience working with synthetic data generation or test data modelling.
Familiarity with knowledge graph systems or structured product knowledge mapping.
Experience evaluating cost efficiency and ROI of engineering or QA initiatives.
Experience working in fast-moving startup environments.
Builder mindset with a strong desire to experiment and create new solutions.
Proactive and hands-on, comfortable turning ideas into working prototypes.
Strong analytical thinking and problem-solving skills.
Comfortable working in rapidly evolving technical environments.
Curious about emerging AI technologies and their practical applications.
Able to think creatively about testing strategies and data generation.
Collaborative and comfortable working across engineering and QA teams.
Maintains a can-do attitude and flexible approach to tools and technologies.