Similar Jobs
See allSenior Evaluation Expert
Surge AI
Python
Real Questions You'd Ask an AI
Kled
Global
Auditor
Anyone AI
Global
Auditing
Accounting
Financial Analysis
Senior Software Engineer (OpenAI Codex, Claude Code or Cursor)
Not Specified
Netherlands
TypeScript
JavaScript
Python
AI Agent Evaluation Engineer
Cresta
Canada
Python
LLM APIs
Data Analysis
Role:
- Write questions and tasks that isolate and test specific model capabilities.
- Develop rubrics that define strong, partial, and poor answers.
- Create reference answers and edge cases to challenge AI models.
Requirements:
- Experience in QA or test engineering, assessment design.
- Experience writing test cases, exam questions, rubrics.
- Comfort using spreadsheets or Python to organize and analyze results.
Compensation:
- $30 to $60 per hour.
- Fully remote.
- Projects typically last from one week to several months.