Source Job

Brazil

  • Design and engineer challenging benchmark tasks for evaluating coding agents in multilingual terminal environments.
  • Create authentic task environments using native language assets and identify model failure points.
  • Participate in rigorous quality assurance processes including calibration and audit of benchmark tasks.

Python Shell Scripting Software Engineering

20 jobs similar to AI Benchmark Engineer | Native Language Specialist

Jobs ranked by similarity.

Global

  • Design and build rigorous, verifiable Terminal-Bench tasks that test multilingual robustness in LLMs across prompt language effects and encoding edge cases.
  • Create realistic task environments with datasets and files in your native language, ensuring assets remain in the target language to genuinely measure multilingual handling.
  • Calibrate task difficulty by analyzing execution logs and participate in a 4-layer human quality control process to ensure benchmark integrity.

LILT is an AI and language technology company whose mission is to make the world's information available to everyone, regardless of language. They operate with a global community of linguists, engineers, and subject matter experts, fostering a culture of innovation and excellence.

Global Europe

  • Contribute to shaping safer, smarter AI by joining a global network of linguists and culturally aware contributors.
  • Work on flexible, remote projects in annotation, evaluation, and prompt creation, always on your terms.
  • Get first access to projects that match your skills, from short tasks to multi-week assignments.

Welo Data, part of Welocalize, is a global AI data company with a network of over 500,000 contributors. They build smarter, more human AI by offering flexible, remote projects to a diverse community in over 100 countries, emphasizing growth and work-life balance.

India 0w PTO

  • Evaluate and edit AI-generated translations between German and English for accuracy and fluency.
  • Translate content between German and English while maintaining cultural relevance and context.
  • Annotate translation errors and provide feedback to improve AI model performance.

This partner company is seeking a German translator for an AI training project. It offers a flexible, remote contract opportunity for language professionals to improve AI translation quality.

US

  • Develop and evaluate AI training data for LLM and AI agent platforms.
  • Create coding tasks and write reference-quality solutions for evaluation.
  • Critically assess AI-generated code for correctness, security, and maintainability.

Toloka is a leading expert human data platform for AI agents and LLMs, providing high-quality training data. The company focuses on improving AI models through human feedback and structured evaluation.

Brazil

  • Adapt, review, and refine scripts for AI-powered dubbing projects to ensure natural, accurate, and engaging localized dialogue.
  • Review AI-generated dubbing outputs, identifying linguistic, performance, and quality issues that require correction.
  • Collaborate with production and language teams to improve localization quality and achieve strong creative outcomes.

Our partner is a media and entertainment localization company that adapts film, television, and digital content for global audiences. They are building a global talent pool of freelancers and focus on integrating AI-powered dubbing technologies into their workflows.

Global

  • Assess software engineering tasks for technical accuracy, realism, and reproducibility.
  • Provide actionable feedback on codebase integration issues and logic errors.
  • Ensure AI training workflows are rigorous and practically applicable.

Project World Wide sources experienced technical specialists for AI training task auditing. This freelance contract opportunity focuses on ensuring technical rigor and accuracy in AI workflows.

Philippines

  • Review and evaluate AI-generated text for linguistic accuracy, grammatical correctness, and cultural appropriateness.
  • Identify issues and provide high-quality rewritten responses to improve language outputs.
  • Develop natural prompts and responses in the target language to enhance conversational datasets.

Welo Data is a global AI data company with over 500,000 contributors delivering high-quality, ethical data to train advanced AI systems. We are building smarter, more human AI with a diverse community in over 100 countries.

Greece

  • Evaluate developer workflow tasks for technical accuracy, realism, solvability, reproducibility, and alignment with reliable testing and evaluation criteria.
  • Audit AI-assisted development scenarios to identify technical inconsistencies, logic errors, workflow inefficiencies, or issues affecting task quality.
  • Provide clear, detailed, and actionable feedback that enables improvements to AI training and evaluation tasks.

Global

  • Translate content between English and Mongolian, preserving cultural nuance and clarity.
  • Edit and refine language content for accuracy, tone, and contextual relevance across digital platforms.
  • Collaborate with cross-functional teams to localize content and support language model quality improvements.

LILT provides multilingual AI and human-verified translation services to enterprises, governments, and AI developers. They foster a global community of linguists and experts dedicated to innovation and excellence.

Brazil 0w PTO

  • Record scripted training material in Welsh with clear pronunciation and natural delivery.
  • Evaluate AI-generated speech for linguistic accuracy, fluency, naturalness, and expression.
  • Collaborate with project teams to refine prompts and voice design guidelines.

Our partner is developing next-generation AI voice technologies. The project offers a flexible freelance environment where your linguistic and performance expertise can have a global impact.

India

  • Evaluate AI-generated documents, spreadsheets, and presentation decks for accuracy and professional quality.
  • Assess visual and aesthetic quality including layout, formatting, and readability.
  • Provide clear, structured written feedback to identify issues and improve AI outputs.

Our partner is a company focused on improving AI systems through quality evaluation. They offer a flexible, remote work environment for independent contractors.

UK

  • Have natural, unscripted conversations in Mandarin over video calls with other participants.
  • Discuss everyday topics to help AI models learn natural speech patterns and cultural nuances.
  • Choose your own schedule with no minimum hours, and get paid per task.

Prolific is building the biggest pool of quality human data in the world, used by over 35,000 AI developers and researchers. The platform connects researchers with a global participant pool for ethically sourced data.

$18–$18/hr
Philippines

  • Evaluate AI-generated responses for accuracy, grammar, and cultural relevance.
  • Create natural prompts and responses in Filipino to improve conversational datasets.
  • Collaborate with global teams to help improve AI language models.

Welo Data, part of Welocalize, is a global AI data company that delivers high-quality, ethical data to train advanced AI systems. They have a community of over 500,000 contributors across 100+ countries, focusing on building smarter, more human AI with limitless opportunities for growth.

Slovakia

  • Support Meridian engineering teams by building, testing, and maintaining AI-assisted development workflows for cloud software.
  • Implement and maintain small AI-assisted engineering utilities for repository indexing, code summarization, and documentation generation.
  • Test open-source coding models and document their strengths, limitations, and practical usage guidance.

Deutsche Telekom IT Solutions Slovakia provides innovative information and communication technology services. It has grown to become the second largest employer in eastern Slovakia with over 3900 employees, fostering a culture of continuous improvement and transformation.

Global

  • Write, rewrite, and rate generated content and conversations for AI training.
  • Translate and review content between Slovenian and English with cultural accuracy.
  • Annotate data and images related to current events, pop culture, and more.

They help businesses build, train, and manage AI systems. They offer flexible remote work with variable workload and weekly pay.

Brazil

  • Review and analyze short-form video content to identify all languages in spoken audio and on-screen text.
  • Validate that language and cultural elements accurately reflect Brazilian Portuguese for machine learning datasets.
  • Perform quality checks and apply classification guidelines to generate accurate ground-truth labels.

Jobgether is an AI-powered job matching platform that connects candidates with partner companies. They use technology to ensure fast, objective candidate reviews and partner with firms in international projects.

Asia United States

  • Review and evaluate annotation and audit work according to project guidelines.
  • Perform quality checks across Simplified and Traditional Chinese content.
  • Identify errors, inconsistencies, and recurring quality issues.

Welo Data provides AI services including data annotation and quality assurance for generative AI systems. They are a global community of AI specialists working on next-generation AI projects.

Europe

  • Review and refine localized Bulgarian content to ensure natural, accurate, and polished quality.
  • Execute test cases, identify localization bugs, and report issues through bug-tracking systems.
  • Translate content from English to Bulgarian as needed and verify fixes.

Welocalize provides AI-enabled multilingual content solutions that help enterprises operate and scale globally. They combine AI, automation, and human expertise, supporting over 300 languages with a global team and seven ISO certifications.

Global

  • Engage in live video conversations in Bengali to help train AI models.
  • Discuss everyday topics naturally with other participants, with no scripts.
  • Choose your own schedule and work remotely with flexible task-based hours.

Prolific builds a platform connecting researchers with a global pool of participants to collect high-quality human data for AI development. With over 35,000 organizations using the platform, they focus on ethical data collection and flexible, remote participation.

Canada

  • Perform quality control and translation tasks from English to French Canadian, ensuring high standards.
  • Manage language materials, resolve queries, and interact with clients on linguistic matters.
  • Train and develop freelancers, and participate in process improvement initiatives.

Welocalize provides AI-enabled multilingual content solutions for global enterprises, combining AI, automation, and human expertise. The company has offices worldwide and fosters a culture of innovation and collaboration, supported by seven ISO certifications.