Role Description & Scope
Executive Role Brief & Overview
An active model-alignment batch is currently seeking domain-vetted remote contributors for this role. Compensation is benchmarked at $50–$65/hr, reflecting top-tier market rates for specialized AI evaluation work. Core deliverables for the Kotlin Team Lead role center on RLHF & LLM Rubric Evaluation, multi-step reasoning verification, and factual edge-case auditing. Enjoy frictionless remote execution with asynchronous rubric queues, zero commute, and flexible part-time or full-time bandwidth. Client calibration batches are actively reviewing applicants—use the verified partner portal link to fast-track your credential screening.
- Primary Competency: RLHF & LLM Rubric Evaluation
- Credential Tier: Senior
- Candidate Profile: Ideal for Senior analytical evaluators experienced in structured rubric scoring, prompt stress-testing, and hallucination auditing.
- Verified Compensation: $50 - $65 / hr
What is the role overview and compensation benchmark for this position?
- Role Scope & Domain: Verified remote contract for Kotlin Team Lead within the Finance & Accounting track on verified partner platforms.
- Direct Compensation: Rates range from $50 to $65 per hour (category benchmark average: $104/hr) with direct escrow disbursements.
- Work Structure: Fully remote, flexible scheduling with independent task execution.
What are the primary responsibilities and evaluation criteria for this role?
- Core Evaluation Tasks: Review and benchmark generative AI completions for factual accuracy, reasoning integrity, and compliance with project rubrics.
- Domain Focus: Apply specialized knowledge in LLM, LLM Evaluation, Code Review to identify hallucinations, logic gaps, and edge-case errors.
- Grading Standards: Calibrate outputs against structured rubric criteria and maintain high inter-annotator agreement on partner evaluation batches.
Detailed Role Specifications & Project Scope
In this hourly, remote contractor role, you will work as a Kotlin Quality Assurance Lead (QAL) to oversee quality, consistency, and trainer performance across Kotlin AI training projects. You will review AI-generated Kotlin code and trainer/QA work, evaluate output quality against project guidelines, provide precise written feedback, and ensure contributors follow expected standards.
You will assess work for code correctness, type safety, coroutine behavior, Android/JVM awareness, debugging accuracy, readability, maintainability, performance, security, test coverage, formatting, instruction-following, and rubric adherence.
This role is with SME Careers, a fast-growing AI Data Services company and subsidiary of SuperAnnotate, delivering training data for many of the world’s largest AI companies and foundation-model labs. Your Kotlin quality leadership will help ensure Kotlin training data is accurate, executable, idiomatic, clearly explained, and aligned with client expectations.
Selection process involves an AI interview, a domain-specific task, and an interview with a recruiter.
Important: There is no immediate project for this role; however, if qualified, you will be among the first experts we reach out to when relevant opportunities arise. This will also provide you with access to future projects available through our expert network.