Source Job

• Assess technical accuracy, realism, and reproducibility of AI-assisted developer workflow tasks.
• Provide actionable feedback on IDE integration faults, AI-assisted coding inefficiencies, and logic errors.
• Apply deep knowledge of modern developer tooling, including AI coding assistants and telemetry/trace analysis.

We are sourcing experienced technical specialists to audit tasks used in training and evaluating AI systems. We focus on ensuring technical rigor and accuracy in developer workflow tasks, operating as a remote freelance project.

• Evaluate AI-generated coding interactions end to end for usefulness, accuracy, and consistency with strong engineering practices.
• Assess whether coding agents demonstrate sound technical reasoning and practical engineering judgment rather than just producing working-looking code.
• Provide actionable feedback, distinguishing between adequate and exceptional AI response quality to shape evaluation standards.

Jobgether uses an AI-powered matching process to ensure your application is reviewed quickly and fairly against the role's core requirements. They are a third-party recruitment platform that partners with companies to fill positions, with a streamlined selection process.

• Assess software engineering tasks for technical accuracy, realism, and reproducibility.
• Provide actionable feedback on codebase integration issues and logic errors.
• Ensure AI training workflows are rigorous and practically applicable.

Project World Wide sources experienced technical specialists for AI training task auditing. This freelance contract opportunity focuses on ensuring technical rigor and accuracy in AI workflows.

• Evaluate Kubernetes tasks for technical accuracy, realism, and reproducibility.
• Provide clear feedback on orchestration issues, configuration bugs, or logic errors.
• Utilize your deep Kubernetes expertise to audit complex technical scenarios.

Greenhouse is a hiring platform that powers recruitment for modern companies. They are an established firm with a distributed team and a culture focused on innovation and flexibility.

• Audit AWS Serverless and Infrastructure as Code tasks for technical accuracy and realism.
• Evaluate deployment scenarios, architecture logic, and testing criteria.
• Provide clear, actionable feedback to improve AI training and evaluation systems.

Jobgether uses AI-powered matching to connect professionals with freelance roles at partner companies. This project offers independent remote work with competitive hourly rates and flexible scheduling.

• Evaluate LLM responses for accuracy, clarity, and completeness.
• Fact-check technical claims using authoritative references.
• Validate code and outputs, and annotate model performance.

Prolific builds the largest pool of high-quality human data for AI development, serving over 35,000 AI developers, researchers, and organizations. They connect researchers with a global community to collect ethically sourced behavioral data.

• Assess technical accuracy and reproducibility of AWS Trainium/NKI tasks.
• Provide actionable feedback on kernel execution bugs and logic errors.
• Evaluate hardware acceleration inefficiencies and compilation issues.

We source experienced technical specialists to audit AI training tasks and evaluation workflows. The project is remote and freelance, with a focus on technical accuracy and efficiency.

• Evaluate AI coding-agent interactions for technical accuracy and engineering judgment.
• Assess explanations and reasoning to ensure they genuinely help developers.
• Provide structured feedback to improve AI-assisted development experience.

Jobgether is an AI-powered job matching platform that connects candidates with hiring companies through objective AI-based review. The role is posted on behalf of a partner company that develops AI coding tools.

• Develop and evaluate AI training data for LLM and AI agent platforms.
• Create coding tasks and write reference-quality solutions for evaluation.
• Critically assess AI-generated code for correctness, security, and maintainability.

Toloka is a leading expert human data platform for AI agents and LLMs, providing high-quality training data. The company focuses on improving AI models through human feedback and structured evaluation.

• Write detailed, honest accounts of complex travel agent workflows for AI training.
• Choose from real tasks like planning multi-destination itineraries or managing group bookings.
• Work fully remote on a freelance basis with flexible hours and competitive pay.

Prolific is building the largest pool of quality human data in the world. Over 35,000 AI developers, researchers, and organizations use the platform to collect high-quality data from diverse participants.

• Evaluate AI-generated documents, spreadsheets, and presentation decks against quality rubrics.
• Identify factual, formatting, visual, and structural issues in professional deliverables.
• Provide clear, structured feedback to enhance AI output quality and consistency.

This partner company specializes in AI training and evaluation, focusing on improving the quality of AI-generated professional content. Operating as a remote and asynchronous team, they value precision, collaboration, and independent work.

• Audit multiple-choice options and correct answers for technical accuracy, eliminating ambiguous distractors.
• Verify coding question prompts and grading rubrics, and write additional edge test cases.
• Format and return the final corrected exam in a valid JSON structure.

Terac is building the world's largest pool of vetted human experts for AI. Researchers, AI labs, and product teams use Terac to recruit, screen, and pay study participants across industries, languages, and skill sets.

• Contribute to shaping safer, smarter AI by joining a global network of linguists and culturally aware contributors.
• Work on flexible, remote projects in annotation, evaluation, and prompt creation, always on your terms.
• Get first access to projects that match your skills, from short tasks to multi-week assignments.

Welo Data, part of Welocalize, is a global AI data company with a network of over 500,000 contributors. They build smarter, more human AI by offering flexible, remote projects to a diverse community in over 100 countries, emphasizing growth and work-life balance.

• Audit multiple-choice and coding questions for technical accuracy in a FastAPI microservices exam.
• Verify code prompts, evaluate grading rubrics, and add edge test cases.
• Submit a corrected JSON file with your expert modifications.

Terac builds the world's largest pool of vetted human experts for AI. Researchers and AI labs use Terac to recruit, screen, and pay study participants across many industries and languages.

• Translate a complex workflow into a demanding AI prompt designed to expose model limitations.
• Test your prompt in ChatGPT, refine it until the AI fails, and write a grading rubric for others to use.
• Submit your prompt, failure notes, rubric, and a screen recording of your thought process.

Terac builds the world's largest pool of vetted human experts for AI research and evaluation. They are a growing platform used by AI labs and researchers to recruit, screen, and pay study participants globally.

• Evaluate AWS Serverless and IaC tasks for technical accuracy and reliability.
• Provide clear, actionable feedback on deployment pipeline errors and architecture issues.
• Ensure tasks are realistic, reproducible, and supported by robust tests.

We source experienced technical specialists to audit tasks used to train AI systems. Our project ensures that AI training workflows are technically rigorous and accurate, with a focus on high-quality deliverables.

• Evaluate AI-generated spreadsheets against quality standards and domain-specific rubrics.
• Identify calculation errors, formatting issues, and inconsistencies in workbooks.
• Provide structured, actionable feedback to improve AI output accuracy and usability.

The partner company focuses on evaluating AI-generated spreadsheets and workbooks. It offers a remote, asynchronous work environment with flexible scheduling.

• Support participants in virtual HR workshops by diagnosing and improving generative AI outputs.
• Provide tool-agnostic guidance to help attendees refine prompts and achieve practical results.
• Ensure responsible AI use and escalate issues as needed while fostering productive learning.

This company is a leading technology firm specializing in internet-related services and products, including search, cloud computing, and AI. It is a large global organization with a culture of innovation and collaboration.

• Design AI agent systems that collaborate with developers in real time.
• Build and implement agentic systems for planning, coding, testing, and deployment.
• Integrate LLMs and design orchestration layers for scalable AI systems on AWS.

Ubiminds connects Latin American tech professionals with software companies in the US and Canada. The company is GPTW-certified and has been supporting hundreds of professionals for 9 years.

• Review and advise on evaluation criteria and scoring rubrics for AI-generated outputs.
• Create, edit, and validate high-quality benchmark tasks and reference data for AI training.
• Analyze model failures, including hallucinations and flawed reasoning, providing expert explanations.

The company specializes in AI evaluation and training, helping define standards for next-generation AI systems. They operate as a partner company that manages applications and next steps, valuing autonomy and expertise in their consultants.