Similar Jobs
See allSTEM PhDs and Technical Domain Experts
Jobgether
Canada
AI
Research
Analytical
Finance Professionals - AI Training
Prolific
US
Financial Analysis
Critical Thinking
Data Interpretation
Humanities / Arts / Culture Evaluator
Partner Company
Canada
Critical Thinking
Analytical Skills
Written Communication
Document / Deck Production QA Evaluator
Jobgether
Canada
Quality Assurance
Microsoft Office
Google Workspace
Data Science & Analysis - AI Training
Prolific
Global
Python
R
SQL
Accountabilities:
- Review and advise on evaluation criteria and scoring rubrics to determine whether AI-generated outputs meet professional standards.
- Create, edit, and validate high-quality benchmark tasks, prompts, and responses that serve as trusted reference data.
- Analyze model failures, including hallucinations, flawed reasoning, and logical inconsistencies, and provide expert explanations.
Requirements:
- Demonstrated elite-level expertise in a specialized professional or academic domain, supported by an advanced degree.
- Strong ability to distinguish high-quality from flawed reasoning and to explain complex concepts clearly.
- Excellent written communication skills, with the ability to articulate nuanced technical reasoning.
Benefits:
- Project-based flexibility with defined compensation and non-exclusive engagements.
- Remote work with intellectual challenge and meaningful impact on AI safety.
- Global engagement with additional revenue opportunity to supplement existing income.
Partner Company
The company specializes in AI evaluation and training, helping define standards for next-generation AI systems. They operate as a partner company that manages applications and next steps, valuing autonomy and expertise in their consultants.