Role Description & Scope
Executive Role Brief & Overview
Contribute your subject-matter mastery to next-generation foundation model alignment in this verified remote engagement. The contract features transparent hourly remuneration at $45–$80/hr with zero platform commission deductions. Core deliverables for the AI Workflow Automation Evaluator - Zapier & API Connectors role center on RLHF & LLM Rubric Evaluation, multi-step reasoning verification, and factual edge-case auditing. Flexible weekly allocations and asynchronous review pipelines make this remote contract ideal for high-performing specialists. High-performing evaluators are onboarded daily; initiate your application through our verified referral channel for direct queue placement.
- Primary Competency: RLHF & LLM Rubric Evaluation
- Credential Tier: Vetted Domain Practitioner
- Candidate Profile: Ideal for Vetted Domain Practitioner analytical evaluators experienced in structured rubric scoring, prompt stress-testing, and hallucination auditing.
- Verified Compensation: $45 - $80 / hr
What is the role overview and compensation benchmark for this position?
- Role Scope & Domain: Verified remote contract for AI Workflow Automation Evaluator - Zapier & API Connectors within the AI Training track on verified partner platforms.
- Direct Compensation: Rates range from $45 to $80 per hour (category benchmark average: $73/hr) with direct escrow disbursements.
- Work Structure: Fully remote, flexible scheduling with independent task execution.
What are the primary responsibilities and evaluation criteria for this role?
- Core Evaluation Tasks: Review and benchmark generative AI completions for factual accuracy, reasoning integrity, and compliance with project rubrics.
- Domain Focus: Apply specialized knowledge in analytical evaluation and rubric design to identify hallucinations, logic gaps, and edge-case errors.
- Grading Standards: Calibrate outputs against structured rubric criteria and maintain high inter-annotator agreement on partner evaluation batches.
Detailed Role Specifications & Project Scope
Benchmark AI models orchestrating business workflow automation and third-party SaaS integrations.\nResponsibilities:\n- Evaluate AI-generated webhook payloads, OAuth flow instructions, and data transformation scripts.\n- Test integrations across Salesforce, Slack, Google Workspace, and Jira.\n- Rate output robustness against API rate limits and unexpected schema changes.\nRequirements:\n- Proficiency with REST APIs, webhooks, and automation tools (Zapier, Make, n8n).\n- Strong technical documentation skills.