Senior Evaluation Expert

Surge AI

Salary range

$75–$175/hr

Benefits

Similar Jobs

See all

Role Overview:

  • Lead benchmark design and validation for AI research projects.
  • Guide grading operations including calibrating graders and resolving disputes.

Key Responsibilities:

  • Design scope, question types, difficulty levels, and scoring methods for benchmarks.
  • Write and review rubrics to ensure consistent grading.
  • Develop gold-standard answers and test sets.

Qualifications:

  • At least 5 years of experience in evaluation or domain expertise.
  • Experience designing assessments or benchmarks used by others.
  • Proficiency in Python and evaluation tools.

Surge AI

Surge AI connects senior evaluation experts to support AI research and model testing. The company focuses on rigorous benchmark design and grading for AI systems, with a network of specialized contractors working on diverse projects.

Apply for This Position