AI Evaluator - POLISH
Podsumowanie roli od JobGrid
AI Evaluator - POLISH at Gramian Consulting Group is a remote contract role in Poland for mid-level candidates in IT / Analytics. JobGrid normalizes the role facts, keeps the employer description separate, and sends candidates to the original public application page with non-personal referral parameters. This listing is part of JobGrid's Zdalne oferty pracy AI z publicznych stron firm.
- Remote, Poland; workplace: Remote; employment type: Contract; seniority: Mid; category: IT / Analytics.
- Source content is in English, and JobGrid preserves that original-language boundary while presenting the listing in English.
- Source freshness is current: posted on 2026-06-01 and last checked on 2026-06-01.
- No salary was provided in the payload, so JobGrid does not add salary context or estimates here.
About Us
Gramian Consultancy is a boutique consultancy specializing in IT professional services and engineering talent solutions. With a strong background in software engineering and leadership, we help companies build high-performing teams by matching them with professionals who truly fit their needs.
This opening is on behalf of one of our clients, and we’ll work closely with you to make the process clear and straightforward.
Role Overview
We are looking for Polish-speaking AI Content Analysts to support evaluation of a new personalization capability within a leading AI assistant platform.
In this role, you will design realistic conversational prompts based on your own context and experiences, then rigorously evaluate how effectively the AI uses personal signals (such as prior conversations or activity context) to produce relevant, grounded, and helpful responses.
This position combines creative prompt design, analytical evaluation, and structured quality assessment, making it ideal for candidates with experience in AI evaluation, annotation, content review, or analytical research roles.
Responsibilities
- Design and run short multi-turn conversations (typically 1–5 turns) intended to test AI personalization behavior
- Create prompts grounded in realistic personal scenarios to evaluate contextual understanding
- Review AI responses to determine whether personalization is correctly applied
- Check grounding quality to ensure the model does not invent unsupported claims about the user
- Evaluate integration quality — confirming personal signals are used naturally (not forced or robotic)
- Compare two responses side-by-side and determine which is more helpful, natural, and relevant
- Write clear, structured rationales explaining rankings and referencing specific conversation turns
- Verify debug information showing whether correct data sources were used
- Maintain strict workflow hygiene (including deleting evaluation conversations when required)
Notes:
- The role is 100% remote, working hours within your local time zone (this is a MUST)
- 30-40 hours/week commitment. Paid by hours logged and approved.
- Contracting Model
- Duration: 1 month (possible extension)