Role Description & Scope
Executive Role Brief & Overview
Step into a high-priority remote AI evaluation workstream built for rigorous analytical practitioners. Contractors earn $25–$45/hr with guaranteed weekly payouts and priority access to bonus calibration batches. Specialists selected for AI Agent Tester execute structured RLHF & LLM Rubric Evaluation audits, ensuring model outputs meet production-grade domain standards. Contributors enjoy complete geographic freedom and asynchronous task queues that fit seamlessly around existing commitments. Rolling intake is currently live—complete your brief technical screening via the verified apply route to lock in your batch allocation.
- Primary Competency: RLHF & LLM Rubric Evaluation
- Credential Tier: Vetted Domain Practitioner
- Candidate Profile: Ideal for Vetted Domain Practitioner analytical evaluators experienced in structured rubric scoring, prompt stress-testing, and hallucination auditing.
- Verified Compensation: $25 - $45 / hr
What is the role overview and compensation benchmark for this position?
- Role Scope & Domain: Verified remote contract for AI Agent Tester within the AI Training track on verified partner platforms.
- Direct Compensation: Rates range from $25 to $45 per hour (category benchmark average: $73/hr) with direct escrow disbursements.
- Work Structure: Fully remote, flexible scheduling with independent task execution.
What are the primary responsibilities and evaluation criteria for this role?
- Core Evaluation Tasks: Review and benchmark generative AI completions for factual accuracy, reasoning integrity, and compliance with project rubrics.
- Domain Focus: Apply specialized knowledge in ChatGPT, Muse, AI agents to identify hallucinations, logic gaps, and edge-case errors.
- Grading Standards: Calibrate outputs against structured rubric criteria and maintain high inter-annotator agreement on partner evaluation batches.
Detailed Role Specifications & Project Scope
Role Title: AI Agent Tester
Role Type: Contractor
Location: Remote — Europe
Opportunity Snapshot
Pay: $25-45/hour
Time per task: ~1.5 hours per run (evidence and scoring included)
Onboarding: Once you are presented to the hiring manager, onboarding typically takes 24–48 hours before you can start tasking.
You’ll need: An active personal popular AI account (Free, Go, or Plus, Pro), an active personal and a smartphone that can screen record
Your tasks: Run everyday errands with two AI agents, approve key steps, screen record, and score the results
About the role
AI agents are starting to handle real-life errands: paying bills, booking trips, doing the groceries. We want to know how well they actually do it, from the point of view of a real user. That’s you.
You’ll test two AI agents side by side, using your own accounts on your phone, and tell us what worked, what didn’t, and how it felt.
What you’ll do
- Run real-life scenarios: bills, returns, email follow-ups, bookings, shopping, groceries, travel, family logistics, outings, and forms
- Approve or stop the agent at key steps
- Screen record every attempt
- Score each run on completion, quality, control issues, satisfaction, and time
- Share clear written feedback on the agent experience
Who we’re looking for
- Based in Europe, with a bachelor’s degree or local equivalent.
- Someone who already uses AI in daily life
- 2-3 years of experience, preferably in engineering, business management, or office/operations
- Detail-oriented, great at following instructions and documenting each step
- Clear written communication
- Generalists welcome, domain experts preferred
- No coding or AI background needed
Why you’ll like it
- Get paid to use the AI tools you already use
- Work entirely from your phone, on your schedule
- Have a direct say in how next-generation AI agents get built and improved
How the selection process works
- Apply
- Complete the screening questions
- Interview: 15-30 minutes max
- Profile review: if everything looks good, we present you to the client
- Onboarding: our Talent Ops team will contact you to get you started