Evaluate and improve model safety: Label, rank, audit, and refine human- and model-generated text to improve safety, quality, and policy alignment.
Apply nuanced safety judgment: Assess model outputs against detailed safety guidelines, rubrics, and style standards, making consistent decisions across ambiguous, sensitive, and context-dependent cases.
Create prompts and safety test cases: Write realistic prompts, user scenarios, and adversarial examples that help evaluate model behavior across safety categories and uncover unsafe, evasive, over-refusing, or policy-inconsistent responses.
Interact with generative AI models using project-provided guidelines, safety taxonomies, and attack-vector guidance.
Create and evaluate prompts designed to test model behavior across safety-related categories.
Identify where model responses become unsafe, noncompliant, inconsistent, or otherwise problematic.
Welo Data is an AI services company that specializes in data annotation. They deliver multilingual content transformation services in translation, localization, and adaptation for over 250 languages with a growing network of over 400,000 in-country linguistic resources.
Interact with generative AI models and project guidelines.
Create prompts to test model behavior across safety categories.
Document model breakability and effort level.
Welo Data provides AI services and specializes in data annotation. We foster a collaborative and innovative culture where employees contribute to cutting-edge AI safety evaluation.
Identify and label languages and dialects from model-generated responses.
Review outputs from two different AI models and determine which model correctly identified the proposed language.
Compare model responses and select the appropriate evaluation outcome from predefined options
RWS – TrainAI is looking for Language Data Annotators. They embrace DEI and promotes equal opportunity and prohibits discrimination and harassment of any kind.
Evaluate outputs based on accuracy, relevance, clarity, and instruction-following.
Perform side-by-side (SBS) comparisons of AI-generated responses.
Identify nuances in tone, meaning, and cultural context across French.
Blueprint Technologies is a technology solutions firm headquartered in Bellevue, Washington, with a strong presence across the United States and an expanding footprint across Latin America (LATAM). They are united by a shared passion for solving complex problems and bring diverse perspectives, deep expertise, and real-world experience across industries to help organizations grow, transform, and innovate.
Perform annotation and labeling tasks for generative AI datasets, including text, image, video, and multimodal content.
Create, review, and evaluate prompts and responses across a variety of domains and use cases.
Conduct quality assurance reviews to ensure annotation accuracy, consistency, and adherence to guidelines.
Welo Data delivers multilingual content transformation services in translation, localization, and adaptation for over 250 languages. They drive innovation in language services, delivering high-quality training data transformation solutions for NLP-enabled machine learning, with a network of over 400,000 in-country linguistic resources.
Support the evaluation and labeling of images, search queries and other forms of data.
Understand challenging guidelines and articulate understanding.
Help LLMs learn the intricacies of language and reasoning.
Innodata is a global data engineering company that believes data and Artificial Intelligence (AI) are inextricably linked. Our mission is to enable the responsible advancement of artificial intelligence by providing the data, evaluation frameworks, and human expertise required to build AI systems that can be trusted at scale.
Perform side-by-side (SBS) comparisons of AI-generated responses.
Evaluate outputs based on accuracy, relevance, clarity, and instruction-following.
Apply detailed, scenario-specific annotation guidelines and maintain consistency and high-quality evaluations.
Blueprint Technologies is a technology solutions firm headquartered in Bellevue, Washington, with a strong presence across the United States and an expanding footprint across Latin America (LATAM). Our people bring diverse perspectives, deep expertise, and real-world experience across industries to help organizations grow, transform, and innovate.
Coordinate project workstreams to ensure on-time delivery against execution standards.
Deliver high-quality annotation and QA to establish quality benchmarks.
Analyze datasets to surface patterns and optimization opportunities.
Appen specializes in human-generated data to train, fine-tune, and evaluate models across generative AI, large language models, computer vision, and speech recognition. They have over 1 million contributors in over 200 countries supporting model pre-training, supervised fine-tuning, evaluation and benchmarking, safety and red teaming, and multilingual global expansion.