AI Evaluation Engineer - Business & Operations Domain
Resumo da vaga pela JobGrid
AI Evaluation Engineer - Business & Operations Domain at Gramian Consulting Group: Remoto, Brasil; Contrato; Sénior; Operações e gestão de projetos. This listing is part of JobGrid's Empregos remotos de IA em páginas de empresas. JobGrid adds normalized role facts, source context, and a path to the employer application page so candidates can compare the listing before applying.
- Location and workplace: Remoto, Brasil
- Role classification: Operações e gestão de projetos, Contrato, Sénior
- Source freshness: checked by JobGrid on 2026-05-29.
- Application path: candidates continue to the employer application page with non-personal referral tags.
Gramian Consultancy is a boutique consultancy specializing in IT professional services and engineering talent solutions. With a strong background in software engineering and leadership, we help companies build high-performing teams by matching them with professionals who truly fit their needs.
Role Overview
We are looking for highly analytical business and operations professionals to contribute to advanced AI evaluation projects focused on realistic operational workflows, business processes, and multi-step decision-making scenarios. In this role, you will help design and evaluate complex tasks that test how effectively AI systems handle real-world operational challenges across business environments.
The ideal candidate has hands-on experience working with business operations, analytics, workflow automation, reporting, CRM systems, financial processes, or cross-functional operational workflows. You will contribute realistic scenarios involving process optimization, business reasoning, operational failures, reporting workflows, and structured decision-making.
This role is particularly well suited for professionals with backgrounds in business operations, management consulting, business analytics, finance operations, CRM and sales operations, supply chain and logistics, operational excellence, process automation, reporting and BI, strategy consulting, or workflow optimization.
CONTRACT: Contractor assignment (5 weeks)
COMMITMENT: Full-time (40h/week) or Part-time (20h/week) with minimum 4h PST overlap
LOCATION: Remote — Bangladesh, Brazil, Colombia, Egypt, Ghana, India, Pakistan, Indonesia, Kenya, Nigeria, Turkey, Vietnam
PROCESS: One technical assessment/interview (~45 min)
Responsibilities:
- Design realistic business and operational workflow scenarios for AI evaluation systems
- Create structured tasks involving analytics, reporting, operational reasoning, and process optimization
- Develop clear task specifications, expected outcomes, and validation logic
- Identify operational edge cases, bottlenecks, and workflow failure scenarios
- Evaluate AI-generated outputs for reasoning quality, usefulness, and accuracy
- Contribute expertise across business operations, analytics, automation, or operational systems
- Review and improve workflow complexity, clarity, and evaluation quality
- Collaborate with reviewers and researchers to refine AI benchmark scenarios
- Help create realistic multi-step business and operational problem-solving tasks