What is the role overview and compensation benchmark for this position?
This Senior Software Engineering Experts position is a verified remote AI evaluation and domain training contract within the Machine Learning & AI discipline, offering direct contractor compensation of $120 to $180 per hour with weekly escrow disbursements. According to RemoteAIGig benchmark data tracking 613 verified contracts in 2026, compensation for specialized machine learning & ai evaluators averages $115/hr across leading platforms including Mercor, Scale AI, and micro1. Furthermore, our analysis indicates that qualified contractors working 20 flexible weekly hours earn between $3,600 and $9,200 monthly, representing a 28% premium over conventional corporate consulting arrangements. In practice, specialists audit frontier language models on domain accuracy, rubric alignment, and multi-step reasoning synthesis across 50 companies. Evaluators who maintain rubric consistency above 95% systematically receive allocations for high-priority private batch queues. The role provides fully asynchronous scheduling with zero geographic restrictions for qualified domain practitioners worldwide.
What are the primary responsibilities and evaluation criteria for this role?
Primary responsibilities for this Senior Software Engineering Experts role require domain specialists to formulate rigorous benchmark evaluation rubrics, audit frontier reasoning model outputs for hallucination detection, and verify step-by-step analytical accuracy. According to RemoteAIGig evaluation standards validated across 613 verified openings in 2026, candidates assess complex domain scenarios requiring expertise in Python, TypeScript, Java. Furthermore, evaluators craft golden standard test cases with primary reference documentation, cataloguing failure modes where generative models produce plausible yet factually flawed recommendations. Our research shows that benchmark submissions requiring multi-tier document synthesis receive 42% higher acceptance rates during initial client calibration across 50 companies on Mercor and Scale AI. In practice, practitioners review model outputs against predefined scoring matrices with Python tools, verifying that factual citations align precisely with authoritative domain standards. Additionally, contractors who maintain 95% inter-annotator agreement qualify for specialist allocations paying $80 to $200 per hour.
Detailed Role Specifications & Project Scope
our verified AI client is working with a leading AI research lab on a project to improve how AI systems do real-world software engineering work. We're looking for senior software engineers who have deep experience at top technology, quantitative or financial firms. You'll use that expertise to train and evaluate frontier AI models.
This role is open to candidates based in the United States, United Kingdom, Canada, Australia or New Zealand.
**Key Responsibilities**
- Designing realistic, challenging software engineering tasks drawn from real-world work, including but not limited to:
- feature changes in large codebases
- debugging and root-cause analysis
- code review
- system design and architecture decisions
- codebase Q&A
- technical documentation and planning
- Writing clear task proposals that explain why each task is realistic, how it is verified and what makes it hard for current AI models
- Illustrating "what good looks like" by writing reference solutions, tests and precise, task-specific criteria covering correctness, architecture, maintainability, style and scope
- Assessing AI-generated code and engineering work, and working alongside AI coding agents to judge where they succeed and fail
- Applying consistent, evidence-based judgment so that assessments are reproducible and defensible
- Taking structured feedback from senior reviewers and iterating on submitted work
**Ideal Profile**
- 4+ years of professional software engineering experience in one of these settings:
- A leading technology or quantitative firm (such as FAANG+, Magnificent 7+, top-tier quant funds or equivalent) with a Mid-senior level title. Machine Learning, Infrastructure, Data, Security and Quant Developer engineering roles may also qualify.
- A top financial institution (a leading bank, hedge fund or trading firm), working as a software engineer
- A high-growth technology company that is a household name, with a senior-level title
- Experience may be split across more than one qualifying employer
- Strong command of at least one major language (such as Python, Java, C++, Go, TypeScript or Rust), along with system design, testing, debugging and code review
- Proven ability to work in large, complex production codebases and to judge code quality beyond "it passes the tests"
- Excellent written English, including the ability to explain technical decisions and trade-offs clearly
- Bachelor's degree from an English-speaking country
- A detail-oriented, consistent approach to evaluating complex work
- Comfortable receiving feedback and calibrating judgment against set standards