Home > Jobs > Design & Creative > Generalist (US Only)
✓ Verified Design & Creative 🔥 High Volume

Generalist (US Only)

Compensation $21 - $45 / hr
Location Remote
Available Openings 50+ openings
TIER 2/5 High Volume • Fast-Track
High Acceptance Rate
Interview Format 15-min Async / AI Screen
Insider Preparation Tip Ensure clear microphone audio and highlight direct language or evaluation experience for instant fast-tracking.

Key Skills & Domain Expertise

image data annotation attention to detail written and verbal communication remote collaboration annotation tools quality assurance documentation adaptability time management
SPONSORED (728x90) Accelerate your AI benchmarks: Explore verified developer tooling & APIs

Role Description & Scope

Executive Role Brief & Overview

This vetted remote AI contract connects high-caliber specialists directly with frontier model evaluation teams. Specialists are compensated at $21–$45/hr via direct weekly or bi-weekly settlements for all verified task submissions. As a Generalist (US Only), your primary focus involves RLHF & LLM Rubric Evaluation, synthesizing gold-standard responses, and flagging subtle model hallucinations. Work entirely on your own schedule with 24/7 cloud benchmark access and flexible weekly hour commitments. Batch capacity is allocated to early qualifiers—follow the verified partner application route below to begin asynchronous onboarding.

  • Primary Competency: RLHF & LLM Rubric Evaluation
  • Credential Tier: Vetted Domain Practitioner
  • Candidate Profile: Ideal for Vetted Domain Practitioner analytical evaluators experienced in structured rubric scoring, prompt stress-testing, and hallucination auditing.
  • Verified Compensation: $21 - $45 / hr

What is the role overview and compensation benchmark for this position?

  • Role Scope & Domain: Verified remote contract for Generalist (US Only) within the Design & Creative track on verified partner platforms.
  • Direct Compensation: Rates range from $21 to $45 per hour (category benchmark average: $54/hr) with direct escrow disbursements.
  • Work Structure: Fully remote, flexible scheduling with independent task execution.

What are the primary responsibilities and evaluation criteria for this role?

  • Core Evaluation Tasks: Review and benchmark generative AI completions for factual accuracy, reasoning integrity, and compliance with project rubrics.
  • Domain Focus: Apply specialized knowledge in image data annotation, attention to detail, written and verbal communication to identify hallucinations, logic gaps, and edge-case errors.
  • Grading Standards: Calibrate outputs against structured rubric criteria and maintain high inter-annotator agreement on partner evaluation batches.

Detailed Role Specifications & Project Scope

AI Image Evaluation Specialist

Role type: Contractor, part-time, project-based

Location: Remote, United States only

About the role

our verified AI client is looking for detail-oriented individuals to support an AI image evaluation project. You'll upload your own original images and compare them with AI-generated versions, identifying visual differences, flaws, and inconsistencies.

This role is ideal for someone with a sharp eye for detail, strong writing skills, and the patience to evaluate images thoroughly. Basic familiarity with AI-generated images is required, but specialized AI experience is not.

What you'll do

Upload your own original images for evaluation tasks.

Compare original images with AI-generated versions to identify differences and visual artifacts.

Spot inconsistencies in details such as lighting, textures, proportions, and overall image quality.

Document findings through clear, accurate written feedback.

Follow evaluation guidelines carefully, prioritizing accuracy and thoroughness.

Requirements

Must be based in the United States.

Excellent written English, with strong grammar and spelling.

Exceptional attention to detail and the ability to identify subtle visual differences.

Patience and a meticulous approach to repetitive evaluation tasks.

Basic understanding of AI image generation and common AI-generated image flaws.

Ability to provide clear, precise, and well-structured written feedback.

Application process

Candidates will complete an AI interview as part of the selection process.

About our verified AI client

our verified AI client is a data lab that connects professionals with opportunities to apply their expertise to customer projects. We work with specialists across fields such as engineering, finance, healthcare, and law to support AI labs and enterprises. Through our extended network, contributors bring their knowledge and judgment to projects with their own scope, requirements, and engagement terms.

ROLE REALITY CHECK Standard Rubric Calibration • Multi-Turn Prompt Audit

Field Inspection & Rubric Realities

1. Technical Task Profile

Daily production batches for Generalist (US Only) center on image data annotation, attention to detail, written and verbal communication, remote collaboration. Practitioners execute structured side-by-side (SxS) model evaluations, author adversarial prompts, and score multi-step chain-of-thought traces against strict rubric calibration guidelines. Every submitted batch is audited against a reference golden set to maintain high inter-annotator agreement and Cohen's kappa, while voice and multimodal queues also log S2S audio latency and barge-in interruption handling.

2. Screening Pitfalls

Automated intake for Tier 2/5 (High Volume • Fast-Track — 15-min Async / AI Screen) enforces deterministic halting criteria. During Micro1 Zara automated technical interviews, candidates are flagged if spoken or coded explanations stall beyond the S2S audio latency window, or if barge-in interruption recovery fails during live follow-up probes. On Mercor and partner asynchronous video screens, recording constraints require concise, evidence-backed walkthroughs without reading off external scripts. Across written rubric calibration gates, the #1 disqualification trigger is missing a P0 factual or spec error on a seeded golden set item, which immediately drops your Cohen's kappa and inter-annotator agreement score below cohort onboarding thresholds.

3. Benchmark Verification

Cross-referencing 52 verified active Design & Creative contracts in our local index shows hourly compensation spanning $12/hr to $150/hr. At $21 - $45 / hr, this Generalist (US Only) contract is positioned as a high-volume entry allocation relative to the $75/hr category median. It carries a Vetting Tier 2/5 (High Volume • Fast-Track) classification, meaning applicants who maintain consistent inter-annotator agreement on the initial golden set and respect all halting criteria during rubric calibration qualify for weekly escrow payouts and priority batch allocation.

DESIGN & CREATIVE • PRE-SCREEN CHECK 0 / 3 Answered

Pre-Screen Readiness Check

Quick 3-step pre-screen check: Test your instincts against real partner grading rules for this specific role before launching the official application.

SKILL CHECK: IMAGE DATA ANNOTATION True / False • Golden Set Compliance

1. Ambiguous Edge Cases vs. Guideline Halting Criteria

True or False: When annotating an ambiguous edge-case sample where the guidelines offer two conflicting interpretations, the annotator should guess the most likely option rather than tagging the item for supervisor escalation.

SKILL CHECK: ATTENTION TO DETAIL True / False • Explicit Rubric Gate

2. Explicit Halting Criteria vs. Overall Impression

True or False: If a candidate submission exhibits exceptional overall effort and creativity but explicitly fails one P0 mandatory constraint defined in the project instructions, an evaluator is permitted to give it a passing score based on holistic impression.

SKILL CHECK: MICRO1 INTERVIEW Single-Choice • Micro1 Screening Protocol

3. Micro1 Automated Voice & Technical Screen

This role routes through an automated Micro1 technical screen. How should you pace your spoken explanations and live problem-solving to pass the automated evaluation gate?