Role Description & Scope
Executive Role Brief & Overview
This vetted remote AI contract connects high-caliber specialists directly with frontier model evaluation teams. Verified remuneration is structured at $50–$85/hr, rewarding high-precision rubric adherence and analytical depth. In this Git Version Control & Pull Request Synthesis Evaluator workstream, practitioners apply RLHF & LLM Rubric Evaluation to stress-test frontier model outputs against strict scoring rubrics. The platform provides round-the-clock task availability, allowing vetted specialists to contribute asynchronously across any time zone. Rolling intake is currently live—complete your brief technical screening via the verified apply route to lock in your batch allocation.
- Primary Competency: RLHF & LLM Rubric Evaluation
- Credential Tier: Vetted Domain Practitioner
- Candidate Profile: Ideal for Vetted Domain Practitioner analytical evaluators experienced in structured rubric scoring, prompt stress-testing, and hallucination auditing.
- Verified Compensation: $50 - $85 / hr
What is the role overview and compensation benchmark for this position?
- Role Scope & Domain: Verified remote contract for Git Version Control & Pull Request Synthesis Evaluator within the AI Training track on verified partner platforms.
- Direct Compensation: Rates range from $50 to $85 per hour (category benchmark average: $73/hr) with direct escrow disbursements.
- Work Structure: Fully remote, flexible scheduling with independent task execution.
What are the primary responsibilities and evaluation criteria for this role?
- Core Evaluation Tasks: Review and benchmark generative AI completions for factual accuracy, reasoning integrity, and compliance with project rubrics.
- Domain Focus: Apply specialized knowledge in analytical evaluation and rubric design to identify hallucinations, logic gaps, and edge-case errors.
- Grading Standards: Calibrate outputs against structured rubric criteria and maintain high inter-annotator agreement on partner evaluation batches.
Detailed Role Specifications & Project Scope
Benchmark AI capabilities in generating Git commits, rebase instructions, and PR descriptions.\nResponsibilities:\n- Evaluate AI code review comments for actionable critique and tone.\n- Benchmark multi-file diff understanding and complex merge conflict resolution.\n- Score PR summary accuracy against actual code changes.\nRequirements:\n- Strong command of Git CLI (interactive rebases, bisect, reflog, merge strategies).\n- Experience conducting rigorous code reviews in collaborative engineering teams.