Evaluate user requests and AI model responses involving violence, weapons, threats, and dark fiction within the full conversation context.
Distinguish fictional, educational, historical, and defensive violence from requests that seek real-world uplift or express real intent to harm.
Assess whether a model's response gives meaningful real-world capability, and write concise rationales citing policy language and conversation details.
Evaluate user requests and AI model responses against detailed customer policies with precise reasoning.
Distinguish subtle differences in context and intent to classify ambiguous cases accurately.
Participate in calibration discussions and contribute to improving evaluation frameworks.
Handshake powers a platform connecting 25 million job seekers with employers and educational institutions. Through Handshake AI, they provide data to frontier AI labs, having grown to a ~$1B run rate and paying over 30K individuals monthly.
Evaluate generated images against prompts for adherence, composition, realism, and technical defects.
Compare images side by side and select the stronger one with clear, evidence-based rationale.
Classify images against customer content policies covering sexual content, violence, and real-person likeness.
Handshake is a career platform that helps 25 million job seekers connect with employers and 1,600 educational institutions. Handshake AI, a division, works with frontier AI labs on complex data at scale, having grown to a $1B run rate and paying over 30K individuals monthly.
Review and advise on evaluation criteria and scoring rubrics for AI-generated outputs.
Create, edit, and validate high-quality benchmark tasks and reference data for AI training.
Analyze model failures, including hallucinations and flawed reasoning, providing expert explanations.
The company specializes in AI evaluation and training, helping define standards for next-generation AI systems. They operate as a partner company that manages applications and next steps, valuing autonomy and expertise in their consultants.