Role Description & Scope
Executive Role Brief & Overview
Join a high-impact reinforcement learning and benchmark evaluation program designed for experienced practitioners. Compensation is benchmarked at $100/hr, reflecting top-tier market rates for specialized AI evaluation work. Core deliverables for the FP&A Expert role center on RLHF & LLM Rubric Evaluation, multi-step reasoning verification, and factual edge-case auditing. Work entirely on your own schedule with 24/7 cloud benchmark access and flexible weekly hour commitments. Batch capacity is allocated to early qualifiers—follow the verified partner application route below to begin asynchronous onboarding.
- Primary Competency: RLHF & LLM Rubric Evaluation
- Credential Tier: Master's
- Candidate Profile: Ideal for Master's analytical evaluators experienced in structured rubric scoring, prompt stress-testing, and hallucination auditing.
- Verified Compensation: $100 / hr
What is the role overview and compensation benchmark for this position?
- Role Scope & Domain: Verified remote contract for FP&A Expert within the AI Training track on verified partner platforms.
- Direct Compensation: Rates range from $100 per hour (category benchmark average: $73/hr) with direct escrow disbursements.
- Work Structure: Fully remote, flexible scheduling with independent task execution.
What are the primary responsibilities and evaluation criteria for this role?
- Core Evaluation Tasks: Review and benchmark generative AI completions for factual accuracy, reasoning integrity, and compliance with project rubrics.
- Domain Focus: Apply specialized knowledge in analytical evaluation and rubric design to identify hallucinations, logic gaps, and edge-case errors.
- Grading Standards: Calibrate outputs against structured rubric criteria and maintain high inter-annotator agreement on partner evaluation batches.
Detailed Role Specifications & Project Scope
Role Overview
- our verified AI client is hiring experienced FP&A professionals to build and review evaluation tasks for AI agents.
- Each task is set in a realistic simulated company, with the budgets, forecasts, models, emails and business apps a finance team really uses.
- You make sure each task reads like a real request, the reference answer holds up against the source data, and the grading rubric checks what a senior FP&A lead would check.
Key Responsibilities
- Refine AI-drafted task prompts so they read like a real request from a CFO or business partner, without giving the answer away.
- Check reference models and outputs against the world's files and app records, and correct what is wrong.
- Write and tighten grading rubrics: pass/fail criteria with exact values, plus the judgment calls a generalist would miss.
- Read how leading AI models attempt each task and confirm they fail for fair, defensible reasons.
- Work across annual budgeting and long-range planning, rolling, revenue, cash-flow and headcount forecasts, variance analysis and KPI reporting, board and investor reporting, three-statement and scenario modeling, and business partnering on pricing and cost decisions.
Ideal Qualifications
- 3+ years of hands-on FP&A experience (Analyst, Senior Analyst, Manager or above). Time in the field matters more than title.
- Bachelor's degree in finance, accounting, economics or a related field.
- Ownership of budgeting or forecasting cycles, strong driver-based and scenario modeling, and fluency in a planning tool such as Anaplan, Adaptive or Planful.
- CFA, MBA or CPA is a plus, as is prior experience on a our verified AI client RL project.
Details
- Pay: $100/hour.
- You start with a short onboarding session and a paid skills assessment. Once you pass, you can claim your first task right away.
- Strong performers can move into reviewer and world lead roles.