Role Description & Scope
Executive Role Brief & Overview
An active model-alignment batch is currently seeking domain-vetted remote contributors for this role. Verified remuneration is structured at $45–$65/hr, rewarding high-precision rubric adherence and analytical depth. Contributors in the Codeforces Problem Setter role leverage Python & Backend Engineering to benchmark model performance across high-difficulty domain scenarios. Work entirely on your own schedule with 24/7 cloud benchmark access and flexible weekly hour commitments. Client calibration batches are actively reviewing applicants—use the verified partner portal link to fast-track your credential screening.
- Primary Competency: Python & Backend Engineering
- Credential Tier: Vetted Domain Practitioner
- Candidate Profile: Ideal for Vetted Domain Practitioner software engineers and systems developers skilled in production code review, debugging, and algorithmic synthesis.
- Verified Compensation: $45 - $65 / hr
What is the role overview and compensation benchmark for this position?
- Role Scope & Domain: Verified remote contract for Codeforces Problem Setter within the Software Engineering track on verified partner platforms.
- Direct Compensation: Rates range from $45 to $65 per hour (category benchmark average: $80/hr) with direct escrow disbursements.
- Work Structure: Fully remote, flexible scheduling with independent task execution.
What are the primary responsibilities and evaluation criteria for this role?
- Core Evaluation Tasks: Review and benchmark generative AI completions for factual accuracy, reasoning integrity, and compliance with project rubrics.
- Domain Focus: Apply specialized knowledge in Basics C++, C++, Competitive Programming to identify hallucinations, logic gaps, and edge-case errors.
- Grading Standards: Calibrate outputs against structured rubric criteria and maintain high inter-annotator agreement on partner evaluation batches.
Detailed Role Specifications & Project Scope
Job Title:Codeforces Problem Setter
Job Type: Contractor (15hrs+ a week)
Location: Remote
Opportunity Snapshot
Pay: $30 - $100
Commitment: 15+ hours per week
Location: Fully remote
You’ll need: Strong competitive programming background and advanced C++
Your tasks: Create original competitive programming problems with reference solutions and robust test suites
You’ll create Reinforcement Learning Environments that test a model’s ability to solve hard algorithmic problems. Each task means writing an original problem that doesn’t exist on any online judge, a correct reference solution, and a test suite strong enough that a wrong solution fails.
What you’ll do
- Design original competitive programming problems, absent from Codeforces, AtCoder, and all other judges.
- Write a correct reference solution and verify it against the constraints.
- Build a robust test suite: edge cases, maximum constraints, and tests that break plausible wrong approaches.
- Implement checkers for problems with multiple valid outputs.
- Document the problem’s intended solution, complexity, and the reasoning behind your test cases.
- Review problems from other contributors for correctness and test coverage.
- Flag problems that turn out to be ambiguous or have unintended solutions.
What we’re looking for
- Strong competitive programming record — Codeforces, AtCoder, ICPC, IOI, or equivalent.
- Advanced C++ for reference solutions and checkers.
- Deep algorithmic knowledge and the ability to reason about constraints, complexity, and edge cases.
- Meticulous attention to detail in validation and output consistency.
- Clear written and verbal English for documenting problems and edge cases.
- Comfortable working independently and meeting deadlines.
- Prior experience as a problem setter, tester, or checker on a major platform is a strong plus.
Compensation Structure
Compensation is output-based; experts are paid per task that meets the project specifications. The time required may vary depending on your experience and workflow. Minimum submission requirements apply.
Start Timeline & Availability
We typically fill roles within 48 hours and are looking for experts ready to jump in right away. If selected, we expect you to start your first tasks within 24–48 hours of completing onboarding.
How the selection process works
- Apply and complete the screening questions
- Complete the AI interview for this role — it’s how your expertise gets in front of the client
- Profile review
- Onboarding — our Talent Ops team will get you set up