Role Description & Scope
Executive Role Brief & Overview
This vetted remote AI contract connects high-caliber specialists directly with frontier model evaluation teams. Approved contributors receive direct contractor compensation of $70–$140/hr, disbursed on a reliable weekly escrow schedule. Core deliverables for the NSA / Signals Intelligence (SIGINT) Expert role center on RLHF & LLM Rubric Evaluation, multi-step reasoning verification, and factual edge-case auditing. Built for remote autonomy, this contract lets you scale your weekly hours up or down based on batch availability. Client calibration batches are actively reviewing applicants—use the verified partner portal link to fast-track your credential screening.
- Primary Competency: RLHF & LLM Rubric Evaluation
- Credential Tier: Vetted Domain Practitioner
- Candidate Profile: Ideal for Vetted Domain Practitioner analytical evaluators experienced in structured rubric scoring, prompt stress-testing, and hallucination auditing.
- Verified Compensation: $70 - $140 / hr
What is the role overview and compensation benchmark for this position?
- Role Scope & Domain: Verified remote contract for NSA / Signals Intelligence (SIGINT) Expert within the Finance & Accounting track on verified partner platforms.
- Direct Compensation: Rates range from $70 to $140 per hour (category benchmark average: $104/hr) with direct escrow disbursements.
- Work Structure: Fully remote, flexible scheduling with independent task execution.
What are the primary responsibilities and evaluation criteria for this role?
- Core Evaluation Tasks: Review and benchmark generative AI completions for factual accuracy, reasoning integrity, and compliance with project rubrics.
- Domain Focus: Apply specialized knowledge in signals intelligence (sigint) analysis, intelligence scenario development, analytical reasoning to identify hallucinations, logic gaps, and edge-case errors.
- Grading Standards: Calibrate outputs against structured rubric criteria and maintain high inter-annotator agreement on partner evaluation batches.
Detailed Role Specifications & Project Scope
Role type: Contractor
Location: Remote
Commitment: Flexible, project or task-based
Compensation: Typically $50–$100 USD/hour, negotiable based on experience
About the opportunity
our verified AI client is looking for experienced Signals Intelligence (SIGINT) professionals with backgrounds in the National Security Agency (NSA), military intelligence, or allied government intelligence organizations to support customer projects focused on AI training and evaluation. In this role, you'll apply your intelligence expertise to assess AI-generated outputs, develop realistic scenarios, and help establish standards for analytical accuracy and reasoning.
Your expertise will help improve how AI systems interpret intelligence information, understand operational workflows, and approach complex national security scenarios.
What you'll do
- Develop realistic, unclassified SIGINT scenarios and reference materials for AI evaluation tasks.
- Review AI-generated intelligence assessments for technical accuracy, relevance, and analytical quality.
- Create and refine evaluation rubrics to assess AI-generated outputs against established intelligence methodologies.
- Identify gaps, inconsistencies, and errors in SIGINT-related reasoning and operational scenarios.
- Provide structured feedback and collaborate with project teams to improve model performance.
Required qualifications
- Professional experience in government or military Signals Intelligence (SIGINT) operations, collection, or analysis.
- Strong understanding of SIGINT methodologies, intelligence workflows, and analytical practices.
- Ability to assess complex intelligence scenarios and communicate findings clearly.
Preferred qualifications
- Previous experience with the NSA, US intelligence community, or allied government intelligence agencies.
- Expertise in COMINT, ELINT, signals analysis, or cyber intelligence operations.
- Experience preparing intelligence assessments, operational reports, or technical documentation.
- Familiarity with intelligence training, quality evaluation, or analytical review processes.
- Exposure to AI evaluation, model testing, or rubric development is a plus.
Application process
- Candidates will complete an AI interview as part of the application process.
Learn More About Our Government & Defense Work
At our verified AI client, we work with government and defense experts to evaluate how advanced AI models perform on real-world tasks.
Our MissionReport benchmark assesses AI-generated military and government reports, highlighting the importance of expert judgment, accuracy, and operational understanding.
Learn more: https://remoteaigig.com/jobs