Role Description & Scope
Executive Role Brief & Overview
Step into a high-priority remote AI evaluation workstream built for rigorous analytical practitioners. The contract features transparent hourly remuneration at $55–$100/hr with zero platform commission deductions. Specialists selected for AI Evaluation Rubric Architect - Standard Operating Procedures execute structured RLHF & LLM Rubric Evaluation audits, ensuring model outputs meet production-grade domain standards. The platform provides round-the-clock task availability, allowing vetted specialists to contribute asynchronously across any time zone. Immediate start dates are available for vetted practitioners; use our verified referral link to enter the priority review pipeline.
- Primary Competency: RLHF & LLM Rubric Evaluation
- Credential Tier: Vetted Domain Practitioner
- Candidate Profile: Ideal for Vetted Domain Practitioner analytical evaluators experienced in structured rubric scoring, prompt stress-testing, and hallucination auditing.
- Verified Compensation: $55 - $100 / hr
What is the role overview and compensation benchmark for this position?
- Role Scope & Domain: Verified remote contract for AI Evaluation Rubric Architect - Standard Operating Procedures within the Finance & Accounting track on verified partner platforms.
- Direct Compensation: Rates range from $55 to $100 per hour (category benchmark average: $104/hr) with direct escrow disbursements.
- Work Structure: Fully remote, flexible scheduling with independent task execution.
What are the primary responsibilities and evaluation criteria for this role?
- Core Evaluation Tasks: Review and benchmark generative AI completions for factual accuracy, reasoning integrity, and compliance with project rubrics.
- Domain Focus: Apply specialized knowledge in LLM to identify hallucinations, logic gaps, and edge-case errors.
- Grading Standards: Calibrate outputs against structured rubric criteria and maintain high inter-annotator agreement on partner evaluation batches.
Detailed Role Specifications & Project Scope
Author clear, unambiguous evaluation rubrics and standard operating procedures (SOPs) for LLM evaluation.\nResponsibilities:\n- Translate client model objectives into multi-tiered, objective evaluation criteria.\n- Write comprehensive guideline documents with positive and negative anchor examples.\n- Continuously iterate on rubrics based on annotator edge case inquiries.\nRequirements:\n- 3+ years in technical writing, legal drafting, or curriculum design.\n- Exceptional ability to create structured, systematic guidelines for complex subjective topics.