Source Job

Global

  • Evaluate AI-generated code for correctness, efficiency, maintainability, and overall software quality.
  • Develop challenging programming problems, reference solutions, and comprehensive test cases.
  • Provide clear, detailed written technical feedback to support continuous AI model improvement.

Software Engineering Algorithms Data Structures Code Review System Design

20 jobs similar to Junior Software Engineer – AI Code Evaluation

Jobs ranked by similarity.

US

  • Evaluate software engineering tasks for technical accuracy, realism, and reproducibility.
  • Investigate codebases, tests, and integration issues to identify technical weaknesses.
  • Provide clear, actionable feedback that directly improves AI training and evaluation workflows.

Jobgether is an AI-powered job platform that connects candidates to roles through objective, skill-based matching. It focuses on remote and freelance opportunities, with a data-driven recruitment process and a global candidate pool.

US Canada

  • Curate high-quality code examples and datasets for LLM training and evaluation.
  • Develop and assess AI-generated software across multiple programming languages and the full SDLC.
  • Collaborate with research teams to design verification mechanisms and improve coding benchmarks.

This partner company specializes in evaluating large language models and improving AI systems through rigorous engineering benchmarks. It offers a remote, collaborative culture where engineers and researchers advance AI evaluation workflows together.

Global

  • Evaluate AI-generated coding interactions end to end for correctness and engineering judgment.
  • Assess whether outputs reflect strong engineering taste and provide clear, opinionated feedback.
  • Help define what great looks like for AI coding tools like Codex, Claude Code, and Cursor.

G2i Inc. is a technology staffing company that connects software engineers with remote contract opportunities. The company values engineering excellence and provides flexible, ongoing projects for senior-level developers.

US

  • Participate in a remote video interview about your experience with AI coding tools.
  • Discuss how tools like GitHub Copilot and Cursor fit into your development workflow.
  • Share feedback on the strengths and limitations of AI-assisted coding.

Our partner company is a research organization focused on AI-assisted software development. It is currently seeking feedback from practicing software engineers through paid interviews.

United States

  • Evaluate AI model responses across diverse topics and use cases against established quality standards.
  • Write, test, and refine system prompts to influence model behavior and performance.
  • Identify and document model failures and provide clear, actionable feedback to product teams.

This company specializes in evaluating and improving advanced AI systems through content evaluation and prompt engineering. It operates as a partner company managing applications for this role, focusing on quality and consistency in AI behavior.

Global 7w PTO

  • Own product features end-to-end, from problem statement to production.
  • Write specs with AI agents and critically review their code in complex systems.
  • Deliver reliable features handling financial transactions and evolve AI-first development.

Social Discovery Group creates social entertainment platforms that connect people online across cultures, addressing loneliness and disconnection. The company's international remote team works from everywhere, and it earned 'Great Place to Work' recognition in 2024-2025.

Europe

  • Use software engineering expertise and Dutch fluency to evaluate and improve advanced AI systems.
  • Assess model responses for logical accuracy, coding quality, and adherence to best practices across multiple technical domains.
  • Document model failure modes and provide structured feedback to improve AI reasoning and coding performance.

Australia 6w PTO

  • Own product features end-to-end, from problem identification to production.
  • Work with AI agents to create specs and delegate development tasks.
  • Critically review AI-generated code to ensure reliability and quality.

Our partner company is seeking a Product Engineer to own features end-to-end in an AI-native environment. They are a remote, international team with an AI-first culture and a focus on using AI agents as primary development partners.

$88,000–$180,000/yr
US

  • Direct autonomous coding agents across production workstreams, decomposing complex problems and reviewing agent output.
  • Design and maintain evaluation suites to establish measurable quality standards and own end-to-end software quality.
  • Operate within security boundaries such as NIST 800-53, WCAG, and SOC 2 while advancing AI-native development practices.

The company is an organization operating within a FedRAMP-authorized environment, focused on AI-native software development and autonomous coding agents. It maintains a remote-first, globally distributed engineering culture with an emphasis on rigorous evaluation, compliance, and engineering quality.

$88,000–$180,000/yr
US Unlimited PTO

  • Direct the agent array on production workstreams by decomposing problems into tasks and integrating agent output into shipped software.
  • Review agent-generated pull requests at volume and depth, identifying correctness, security, and accessibility defects.
  • Author evaluation suites that make quality measurable using eval-driven development and own end-to-end quality within a FedRAMP-authorized environment.

Granicus provides cloud-based solutions for government communications, website design, meeting management, and records management, serving over 5,500 agencies and 300 million citizens. With a globally distributed team and a culture of transparency and inclusion, Granicus has been recognized on the GovTech 100 list for the past 5 years.

$125,000–$150,000/yr
US

  • Build AI applications across client engagements, products, and reusable tools.
  • Partner with clients and consultants to translate business problems into software.
  • Use agentic coding tools like Claude Code and Codex to build and maintain solutions.

P3 Adaptive is an AI and data consultancy built on Microsoft technologies, helping organizations modernize their relationship with data. The company has small, senior, hands-on teams and a culture that values reliability, adaptability, and direct communication.

US

  • Implement features that improve MDLinx and drive user impact, with ownership growing over time.
  • Apply software engineering best practices including automated testing, clean code, and peer review.
  • Read AI-generated plans and code critically, and identify work for AI-assisted automation.

M3 is a Japanese global leader in healthcare technology and research solutions, operating physician websites with over 5.8 million members. MDLinx, an M3 company, focuses on transforming pharmaceutical brand promotion through omnichannel engagement.

US

  • Build demos and proofs of concept with real or realistic data using AI coding tools.
  • Lead workshops and explain AI in plain terms to non-technical audiences.
  • Own the technical story for deals, from discovery to scope and security reviews.

Stello is an AI transformation firm that builds AI agents to take over operational workflows for mid-market and enterprise companies. It is a small, collaborative team where engineers review each other's work, share learnings, and help teammates, with more scope for those who deliver.

$250,000–$300,000/yr
US Canada Unlimited PTO

  • Own the build out of new agents, skills, and platform capability for teams across TLDR.
  • Build and deploy agents end to end, from design through implementation, evals, and rollout to internal users.
  • Partner with stakeholders across sales, editorial, and people ops to find where an LLM belongs in their process.

TLDR runs the largest network of tech newsletters in the world, with over 8 million subscribers covering startups, software engineering, AI, and more. Our 31-person full-time team is bootstrapped, profitable, and on track for $35M in revenue this year, with a culture of owning functions rather than slices.

Latin America Unlimited PTO

  • Design, develop, and maintain AI-powered software solutions across the full development lifecycle.
  • Work with backend stacks like Python, Node.js, .NET, and Java, plus modern frontend frameworks such as React or Angular.
  • Build and integrate AI agents, RAG pipelines, and LLM-driven products into production environments.

Cresteo is a nearshore tech services company on a mission to be the world leader in people-first and honest software development. Our small, high-trust team values transparency, profit-sharing, and fearless innovation while working with US-based clients and international teams.

$15–$15/hr
Global

  • Evaluate and label AI model outputs to improve performance and alignment with project guidelines.
  • Create prompts, rewrite text, and generate training data for large language models.
  • Work on flexible, remote, project-based tasks while helping shape the future of AI.

Innodata is a global data engineering company that enables the responsible advancement of AI by providing data, evaluation frameworks, and human expertise. With a 36+ year legacy, the company delivers high-quality data and outstanding outcomes for customers.

$180,000–$240,000/yr
Global

  • Design and improve AI systems for extracting structured reasoning signals from technical interviews.
  • Build LLM-powered pipelines for semantic analysis, behavioral feature extraction, and evaluation.
  • Develop scoring, calibration, and benchmarking methods grounded in measurable evidence.

APADCode is an early-stage AI startup building an AI-powered technical interview platform that evaluates how candidates reason and solve problems. The company is a small, early-stage team with a culture of high ownership, direct collaboration, and minimal bureaucracy.

Global Unlimited PTO

  • Create and publish coding benchmarks that challenge state-of-the-art frontier agents.
  • Shape the frontier of coding agents and publish impactful studies of interesting findings.
  • Help top AI labs improve their models for coding through benchmarks, open-source tools, and research.

Handshake's mission is to organize expert human knowledge to advance the AI economy, working directly with frontier labs on data, evaluations, and post-training challenges. It is a rapidly growing startup with a small, high-ownership technical team from top AI companies.

Global

  • Participate in a paid remote video call about your AI-assisted development workflow.
  • Share how you use tools like GitHub Copilot, Cursor, or Claude Code to write and debug code.
  • Discuss strengths, limitations, and prompt strategies of current code generation models.

Terac is building the world's largest pool of vetted human experts for AI. Researchers, AI labs, and product teams use Terac to recruit, screen, and pay study participants across industries, languages, and skill sets.

Global

  • Develop and deploy cutting-edge AI/ML solutions to enhance the platform and improve student learning experiences.
  • Design, develop, and optimize LLM-powered agentic systems and APIs for real product use cases.
  • Collaborate with senior engineers and contribute to evaluation frameworks and MLOps pipelines.

Interview Kickstart specializes in interview preparation and career transitions into high-demand tech fields like AI, ML, and Data Science. Over 17,000 tech professionals have been guided by current and former hiring managers to land coveted positions at companies like Google and Amazon.