Role Description & Scope
Executive Role Brief & Overview
Accelerate frontier AI safety and domain accuracy by joining this verified remote specialist contract. Specialists are compensated at $65–$115/hr via direct weekly or bi-weekly settlements for all verified task submissions. As a RLAIF Benchmark & Synthetic Feedback Architect, your primary focus involves RLHF & LLM Rubric Evaluation, synthesizing gold-standard responses, and flagging subtle model hallucinations. The platform provides round-the-clock task availability, allowing vetted specialists to contribute asynchronously across any time zone. Because cohort seats fill on a rolling basis, submit your profile early via our verified partner routing link to secure priority review.
- Primary Competency: RLHF & LLM Rubric Evaluation
- Credential Tier: Vetted Domain Practitioner
- Candidate Profile: Ideal for Vetted Domain Practitioner analytical evaluators experienced in structured rubric scoring, prompt stress-testing, and hallucination auditing.
- Verified Compensation: $65 - $115 / hr
What is the role overview and compensation benchmark for this position?
- Role Scope & Domain: Verified remote contract for RLAIF Benchmark & Synthetic Feedback Architect within the AI Training track on verified partner platforms.
- Direct Compensation: Rates range from $65 to $115 per hour (category benchmark average: $73/hr) with direct escrow disbursements.
- Work Structure: Fully remote, flexible scheduling with independent task execution.
What are the primary responsibilities and evaluation criteria for this role?
- Core Evaluation Tasks: Review and benchmark generative AI completions for factual accuracy, reasoning integrity, and compliance with project rubrics.
- Domain Focus: Apply specialized knowledge in LLM, Prompt Engineering to identify hallucinations, logic gaps, and edge-case errors.
- Grading Standards: Calibrate outputs against structured rubric criteria and maintain high inter-annotator agreement on partner evaluation batches.
Detailed Role Specifications & Project Scope
Design judge prompts and scoring rubrics for Reinforcement Learning from AI Feedback (RLAIF).\nResponsibilities:\n- Author meta-prompts that instruct LLM judges to evaluate peer model outputs accurately.\n- Benchmark judge model alignment with human expert consensus.\n- Mitigate LLM judge biases (position bias, verbosity bias, self-enhancement bias).\nRequirements:\n- 2+ years working with LLM evaluation frameworks and prompt engineering.\n- Strong quantitative background in statistical correlation and error analysis.