Role Description & Scope
Executive Role Brief & Overview
Contribute your subject-matter mastery to next-generation foundation model alignment in this verified remote engagement. Verified remuneration is structured at $50–$85/hr, rewarding high-precision rubric adherence and analytical depth. Contributors in the Instruction Following & Constraint Adherence Grader role leverage RLHF & LLM Rubric Evaluation to benchmark model performance across high-difficulty domain scenarios. The workflow is 100% remote and asynchronous, enabling you to allocate 10 to 40 flexible hours per week from anywhere. Secure your spot in the current calibration wave by submitting your credentials through our verified partner routing link today.
- Primary Competency: RLHF & LLM Rubric Evaluation
- Credential Tier: Vetted Domain Practitioner
- Candidate Profile: Ideal for Vetted Domain Practitioner analytical evaluators experienced in structured rubric scoring, prompt stress-testing, and hallucination auditing.
- Verified Compensation: $50 - $85 / hr
What is the role overview and compensation benchmark for this position?
- Role Scope & Domain: Verified remote contract for Instruction Following & Constraint Adherence Grader within the AI Training track on verified partner platforms.
- Direct Compensation: Rates range from $50 to $85 per hour (category benchmark average: $73/hr) with direct escrow disbursements.
- Work Structure: Fully remote, flexible scheduling with independent task execution.
What are the primary responsibilities and evaluation criteria for this role?
- Core Evaluation Tasks: Review and benchmark generative AI completions for factual accuracy, reasoning integrity, and compliance with project rubrics.
- Domain Focus: Apply specialized knowledge in analytical evaluation and rubric design to identify hallucinations, logic gaps, and edge-case errors.
- Grading Standards: Calibrate outputs against structured rubric criteria and maintain high inter-annotator agreement on partner evaluation batches.
Detailed Role Specifications & Project Scope
Benchmark model adherence to complex negative constraints and precise formatting rules.\nResponsibilities:\n- Test complex prompt constraints (word count limits, prohibited words, specific JSON structures).\n- Audit multi-layered instructions containing conditional clauses and ordering requirements.\n- Quantify constraint satisfaction rate (CSR) across automated evaluation runs.\nRequirements:\n- Methodical, detail-obsessed personality.\n- Familiarity with IFEval (Instruction Following Evaluation) benchmarks.