Source Job

12 jobs similar to Pharmacists: Paid Expert Review for Clinical AI Benchmarks

Jobs ranked by similarity.

  • Review de-identified medical images across multiple modalities
  • Provide expert diagnostic reads and clinical interpretations
  • Apply structured scoring rubrics to clinical cases

Terac is building the world's largest pool of vetted human experts for AI. Researchers, AI labs, and product teams use Terac to recruit, screen, and pay study participants across industries, languages, and skill sets.

Global

  • Review de-identified medical images and associated clinical cases through a secure remote platform.
  • Provide structured expert reads and complete standardized scoring rubrics for diagnostic benchmarking.
  • Articulate diagnostic reasoning and clinical confidence while working asynchronously at your own pace.

Terac is building the world's largest pool of vetted human experts for AI, connecting researchers with study participants across industries and skills. The company enables flexible, paid expert work through a secure remote platform.

Global

  • Audit AI 'Chain-of-Thought' reasoning for medical diagnostics to ensure adherence to international clinical protocols.
  • Rigorously verify AI-generated drug interactions, dosage recommendations, and contraindications against peer-reviewed medical literature.
  • Precisely label complex biomedical datasets—including MRI/CT scans, pathology reports, and genomic data—to improve model visual-spatial reasoning.

TELUS Digital is a technology company that helps businesses collect, enhance, train, translate, and localize content to build better AI models. They have a global AI community of over one million contributors from diverse backgrounds, fostering a culture of flexibility and innovation.

UK US

  • Review anonymised conversations with Ash and judge, as a clinician, whether it handles key moments well.
  • Explain your reasoning on what Ash should do, what could go wrong, and why.
  • Work alongside experienced clinicians without contacting users, helping shape what great looks like.

Slingshot AI is the team behind Ash, the first AI designed for mental health, making support more accessible. It's a well-funded Series A company with $93M raised, offices in NYC and London, and a culture of autonomy, flexibility, and care.

$30–$60/hr
Global

  • Design realistic healthcare and medical tasks that challenge advanced AI agents.
  • Analyze clinical records, research findings, and healthcare data to identify inconsistencies and errors.
  • Develop reference solutions and evaluation criteria for AI-generated medical analyses.

Anyone AI creates high-quality training and evaluation data for frontier AI models. The company offers flexible, remote contract work focused on expert-driven AI evaluation and improvement.

Global

  • Review transcripts of short AI conversations focused on mental health topics.
  • Evaluate the safety, tone, and clinical appropriateness of generated responses.
  • Write clinically sound example responses and develop structured grading rubrics.

Terac is building the world's largest pool of vetted human experts for AI, helping researchers and labs recruit, screen, and pay study participants across industries. Their platform connects specialized professionals with paid AI research opportunities.

$125–$125/hr
US

  • You'll review AI-generated outputs for clinical accuracy, safety, and appropriateness across Headway's patient and provider products.
  • You'll apply and refine evaluation rubrics, documenting your clinical reasoning and collaborating with peers on ambiguous cases.
  • You'll work with Clinical and Product teams to turn feedback into rubric and model improvements, ensuring high standards before launch.

Headway builds a new mental healthcare system that automates insurance admin and gives therapists tools to run their practice. We are a Series D company with $325M+ in funding, supporting 85,000+ providers and 1M+ patients across all 50 states.

US

  • Evaluate AI-generated clinical interactions for accuracy, safety, and appropriateness.
  • Annotate and score AI responses against evidence-based protocols and structured rubrics.
  • Provide detailed written feedback and participate in calibration sessions to enhance AI reliability.

The company focuses on improving AI-powered clinical solutions by leveraging clinical expertise. It is an early-stage partner organization offering flexible, remote contract opportunities for healthcare professionals.

US

  • Review and annotate AI agent conversations across clinical use cases to assess accuracy, safety, and guideline adherence.
  • Score and annotate agent performance using structured rubrics and evidence-based protocols.
  • Identify clinical safety concerns and provide actionable feedback to refine AI-driven clinical agents.

Hippocratic AI is developing a healthcare-only, safety-focused large language model to improve patient outcomes globally. With $404M in funding, the company brings together experts from top healthcare and AI institutions like Stanford, Google, and NVIDIA.

Global

  • Fact-check AI-generated answers on medical topics to ensure accuracy and up-to-date information.
  • Review medical terminology for proper use of names for bones, muscles, medicines, or conditions.
  • Flag dangerous health advice and simplify complex research for everyday understanding.

TELUS Digital builds better AI models by leveraging a global community of contributors to collect, enhance, train, translate, and localize content. With over one million diverse experts worldwide, it offers a flexible, collaborative environment shaping innovative AI technologies for major brands.

US

  • Audit medical summaries, timelines and record classifications against source records.
  • Write clinical review guidelines and gold-standard examples for research and engineering teams.
  • Flag clinically significant omissions, causation issues and prior-history findings.

Brighterway builds AI-native claims software and administration for casualty claims, starting with workers' compensation medical-legal review. The team is small, technical, and close to customers, with everyone shipping and talking directly to users.

$30–$60/hr
Latin America

  • Design realistic medical and healthcare tasks for evaluating AI systems in clinical, research, and public health settings.
  • Analyze clinical records, research findings, and healthcare data to develop reference solutions and evaluation criteria.
  • Collaborate with technical contributors to assess AI-generated medical content for accuracy, consistency, and completeness.

The hiring partner company develops AI evaluation systems for healthcare, creating realistic medical tasks to assess advanced AI. It is a remote, collaborative team of healthcare experts and technical contributors focused on improving AI reliability.