Math Reasoning RLHF Annotator

Description

You'll spend your time evaluating model-generated solutions to math problems — ranging from algebra and calculus to combinatorics and proof-based reasoning. Your core task is to compare two or more responses side by side, rank them by correctness and clarity, and flag specific reasoning errors (wrong setup, arithmetic mistakes, skipped logical steps) using a structured rubric. You won't just say "this one is better" — you'll annotate exactly where and why.

The workflow runs inside a browser-based annotation tool. Each task presents a problem statement and two to four model responses. You'll read each solution, verify the math independently, score it across dimensions like factual accuracy, step completeness, and explanation quality, then submit a ranked preference with written rationale. Tasks average 8–15 minutes each depending on problem complexity. You'll work asynchronously across flexible hours with a weekly throughput target.

Strong candidates hold at least an undergraduate background in math, engineering, physics, or a related quantitative field. You should be comfortable doing manual verification — not just pattern-matching for "looks right." Familiarity with LaTeX notation is a plus since many prompts and responses are rendered that way. You won't need to write code, but you will need to show your own working when the rubric asks for it.

Skills & requirements

Mathematical reasoningStep-by-step solution verificationPreference rankingRubric-based annotationLaTeX reading comprehensionError classificationApply Now

Frequently asked questions

Do I need a graduate degree in math to qualify?How long are typical tasks and how many can I complete per hour?Will I see the same problem types repeatedly, or does the work vary?// QUICK INFO

Type

Contractor

Compensation

$35–55/hr

Location

Remote

Category

RLHF

Posted

July 20, 2026

Apply Now Back to all positions

// RELATED POSITIONS

More RLHF rolesRLHFAI Prompt Quality Rater$35–55/hrViewRLHFLegal Reasoning AI Response Evaluator$55–100/hrViewRLHFPhysics Problem Evaluator$55–100/hrViewRLHFHuman Feedback Annotator$35–40/hrViewView all open positions

// READY TO GET STARTED?

Apply in minutes

Create your profile, select your areas of expertise, and start working on frontier AI projects.

Apply Now