Annotate or label text, images, or audio to train AI systems
Evaluate search results, ads, or chatbot responses for quality and relevance
Create and test prompts to improve large language models (LLMs)
Welo Data is a global AI data company with 500,000+ contributors delivering high-quality, ethical data to train advanced AI systems. They are building smarter, more human AI with a diverse community in over 100 countries.
Review materials science papers and graphical abstracts generated by LLMs to ensure accuracy.
Identify and flag discrepancies where the abstract doesn't match the original paper.
Apply deep expertise to improve cutting-edge AI models through strategic document audit.
Prolific builds the world's largest pool of quality human data for AI development, connecting over 35,000 researchers with paid study participants. The company focuses on ethical data collection and flexible remote work for experts.
Review scientific papers and LLM-generated graphical abstracts for accuracy.
Fact-check and identify inaccuracies in AI-generated summaries.
Use domain expertise to verify that technical concepts are correctly represented.
Prolific is building the largest pool of quality human data for AI development, serving over 35,000 developers. They focus on ethical data collection and aim to integrate human perspectives into AI.
Collaborate with advanced language models on software engineering tasks in Spanish (Latin America), documenting failure modes to improve model reasoning.
Verify logical accuracy, coding fluency, and naturalness of Spanish language usage in model outputs.
Capture reproducible error traces and suggest improvements to prompt engineering and evaluation metrics.
The company is working on large-scale language models for scientific discovery and software development, aiming to democratize education and streamline engineering. The project is freelance-based and seeks coding specialists fluent in Spanish (Latin America) to train AI, with no specified company size or culture.
Own the knowledge surface, ingestion pipelines, and retrieval quality.
Own the eval harness for benchmarks and regression detection.
Pair with Agent Engineers on the prompt-and-eval iteration cycle.
LTS applies frontier AI to modernize a decades-old legacy system. The small, senior engineering team values innovation, growth, collaboration, and quality.
Review and assess new AI benchmarks at least every two weeks, evaluating their methodology and implications.
Publish and maintain public-facing reports on benchmarks, updating them as versions and models evolve.
Examine individual tasks within benchmarks in detail, using coding agents while maintaining critical oversight.
Epoch AI is a research institute investigating trends in machine learning and the economic consequences of AI. We aim to build a comprehensive knowledge base on AI, serving policymakers and society, with a small, focused team that values rigor and inclusivity.
Create and curate an evaluation suite of real-world tasks for frontier AI models.
Rigorously evaluate AI systems, analyze results, and communicate findings.
Improve evaluation processes and potentially build out standalone benchmarks.
Epoch AI is a research institute that investigates trends in machine learning and the economic consequences of AI. Our mission is to develop a comprehensive, publicly accessible knowledge base on AI that informs policymakers, industry leaders, and society at large.
Evaluate financial documents and verify information accuracy for AI model training.
Respond to prompts with financial expertise to help AI understand complex fiscal concepts.
Provide data validation and expert feedback to bridge human financial knowledge and machine learning.
Prolific is building the largest pool of quality human data in the world, serving over 35,000 AI developers and researchers. They connect researchers with a global pool of participants for ethically sourced human behavioral data, focusing on integrating diverse human perspectives into AI development.
Design, build, and deploy AI-powered solutions connecting Remote’s platform to customer systems and workflows.
Work at the frontier of practical AI, shipping reliable, observable, and secure systems in production.
Own customer outcomes from discovery through production rollout with high autonomy and direct influence on product roadmap.
Remote is a global HR platform that helps businesses recruit, pay, and manage international teams compliantly. The company fosters a future-focused, async culture with team members across 6 continents and emphasizes innovation, automation, and AI.
Build agent harnesses in Python using LangChain and LangGraph, including tool-calling and structured outputs.
Develop evaluation frameworks with golden datasets, LLM-as-judge, and regression suites, wiring them into CI.
Collaborate with data engineers on Snowflake-backed retrieval patterns like Cortex Analyst and Search Services.
Kobie is a loyalty solutions partner that helps brands build emotional connections with their consumers. Named a Top Workplace in the USA, they offer a flexible remote culture with a focus on collaboration and growth.
Design and execute complex role-play evaluation scenarios simulating real customer support interactions in domains like travel and finance.
Evaluate AI-generated audio outputs using structured qualitative and quantitative metrics, focusing on accuracy, fluency, and task completion.
Audit conversational performance and contribute to the creation and refinement of high-quality, representative audio datasets.
Our partner company specializes in AI training and evaluation for conversational models, focusing on audio quality and real-world scenarios. As a freelance platform, they offer flexible remote work opportunities to language and evaluation specialists.
Utilize Automatic Prompt Generation and Automated Prompt Optimization tools to create and refine prompts for LLM autoraters.
Monitor shadowbot runs, ensure sufficient disagreements, and measure autorater quality using accuracy metrics like F1 scores.
Draft technical launch readiness justifications and handle edge-case scenarios by designing manual prompts.
Welo Data provides AI data validation services for global clients. As a freelance and remote employer, it offers flexible project-based roles with a focus on technical AI workflows.
Design and deliver high-quality AI evaluation data initiatives, from proposals through pilot execution and production readiness.
Recruit and manage subject-matter experts across technical domains, ensuring rigorous quality control frameworks.
Act as key interface with AI lab partners, converting pilots into scaled production engagements.
Jobgether uses AI-powered matching to connect candidates with roles quickly and fairly. They are a remote-first company that shares top-fitting candidates with hiring partners.
Design and develop coding benchmarks used to evaluate frontier AI models.
Analyze AI-generated code for correctness, reliability, efficiency, and edge cases.
Build and maintain scalable data pipelines that support AI evaluation workflows.
An enterprise client is a leading AI platform that enables organizations to build intelligent applications through high-quality human feedback, AI evaluation, and model alignment. The selected consultants will work on improving frontier AI models, though company size and culture details are not specified.
Participate in short voice conversations with AI models using a designated platform, following provided scenarios and prompts.
Evaluate AI responses based on relevance, accuracy, clarity, and overall quality, providing objective ratings and feedback.
Complete assigned tasks within established quality and productivity expectations, with a flexible schedule of 10+ hours per week.
RWS is a provider of technology-enabled language, content management, and intellectual property services. As an equal opportunity employer, they embrace diversity and inclusion, offering flexible remote work opportunities for freelancers and private individuals.
Rate the performance of AI models or algorithms based on their output or behavior.
Label elements of content, assign categories, and evaluate quality or appropriateness.
Generate additional training data by transforming original data like text, images, or audio.
Innodata is a global data engineering company enabling responsible AI advancement. With over 36 years of experience, we deliver high-quality data and services to AI builders and adopters.
Evaluate Korean-English translations produced by an AI chatbot for accuracy and adherence to user-specified requirements.
Apply MQM error annotations, verify auto-generated rubric items, and evaluate each item as Pass or Fail.
Ensure consistent and accurate ratings while completing assigned tasks within given timelines.
Appen helps improve AI-powered translation systems through crowd-based evaluation projects. They are a large global company offering project-based independent contractor roles, fostering a culture of flexibility and remote work.
Design and implement multi-agent workflows, including task decomposition, state management, and tool-use.
Implement rigorous evaluation frameworks to measure and improve agent performance at scale.
Develop reusable agents and prompt libraries serving 1,000+ brands with unique voices simultaneously.
Pencil builds the agentic OS for marketing, creating complex multi-agent architectures for brand-safe, scalable generative AI. They are a growing tech company with a distributed team, valuing innovation and flexibility.
Collaborate with advanced AI models on real-world STEM problems to document failure modes and harden reasoning.
Verify factual accuracy and logical soundness of model responses in mathematics, physics, chemistry, and engineering.
Suggest improvements to prompt engineering and evaluation metrics based on reproducible error traces.
This company is building the next generation of AI by training large language models to become powerful engines of scientific discovery and technical problem-solving. They are seeking STEM specialists to provide high-quality training data, and operate as a contractor-based platform.
Develop and improve NLP systems and language model-powered experiences.
Fine-tune and optimize language models for domain-specific use cases and build evaluation frameworks.
Deploy and maintain production-grade ML systems on GPU infrastructure with a focus on scalability and safety.
BetterHelp removes barriers to therapy and makes mental health care accessible globally. Founded in 2013, it is now the world's largest online therapy service with over 30,000 licensed therapists, and it invests deeply in employee well-being and professional development.