Build evaluations to measure and improve AI reasoning in complex, high-stakes decisions.
Collaborate with engineers, product, and domain experts to turn observed failures into testable improvements.
Own the quality of shipped systems, from code reviews to monitoring and integration.
Elicit builds AI systems that help research and business teams make evidence-based decisions on high-stakes questions. They are a remote-friendly, collaborative team that values rigorous thinking, experimentation, and in-person retreats.
Build tools for understanding traces, evaluating agent behavior, and turning results into improvements.
Run experiments on Mastra's primitives to find where they fall short and improve configuration and behavior.
Help turn findings into capabilities developers can use in their own applications.
Mastra is the open-source TypeScript framework for building AI agents. We're a small, fully remote team backed by Y Combinator and Spark Capital with $35M raised.
Lead the engineering strategy and execution for evaluations of AI agents, owning the core evaluation platform.
Design scalable evaluation infrastructure, APIs, workflows, and production systems across software categories.
Mentor and develop a team of engineers as the technical authority on agentic evaluation.
This company focuses on building credible, scalable evaluations of AI agents from software vendors. It operates as a fully remote, inclusive team with a flexible culture and a focus on professional growth.
Curate high-quality code examples and datasets for LLM training and evaluation.
Develop and assess AI-generated software across multiple programming languages and the full SDLC.
Collaborate with research teams to design verification mechanisms and improve coding benchmarks.
This partner company specializes in evaluating large language models and improving AI systems through rigorous engineering benchmarks. It offers a remote, collaborative culture where engineers and researchers advance AI evaluation workflows together.
Evaluate AI-generated coding interactions end to end for correctness and engineering judgment.
Assess whether outputs reflect strong engineering taste and provide clear, opinionated feedback.
Help define what great looks like for AI coding tools like Codex, Claude Code, and Cursor.
G2i Inc. is a technology staffing company that connects software engineers with remote contract opportunities. The company values engineering excellence and provides flexible, ongoing projects for senior-level developers.
Design and build challenging evaluations and executable environments for coding and long-horizon agents.
Develop scoring logic and verifiers to detect reward hacks and trace failures through model outputs and harness code.
Build and scale RL data pipelines for generating and curating challenging coding and agentic tasks.
Prime Intellect is building the open superintelligence stack, providing infrastructure for post-training frontier AI models and open-source models trained for long-horizon tasks. The company has raised $150M from top investors and is a mission-driven team passionate about accelerating science and AI.
Evaluate AI-generated code for correctness, efficiency, maintainability, and overall software quality.
Develop challenging programming problems, reference solutions, and comprehensive test cases.
Provide clear, detailed written technical feedback to support continuous AI model improvement.
Our client is a leading organization in the artificial intelligence industry, collaborating with top technology companies and AI research teams to advance next-generation AI systems. The company offers a remote, contract-based culture focused on evaluating AI-generated code and improving AI model performance alongside highly skilled technical professionals.
Build LLM-based agents on the platform's scaffolding, integrating tool calls, internal APIs, and guardrails.
Take agents to production on AWS with containers, CI/CD, secrets, permissions, and security controls.
Define and run evals, monitor with Langfuse, and document runbooks for independent operation.
Muttdata builds innovative Data Products and Machine Learning solutions to help companies solve complex business challenges. It is a fast-growing, remote-first startup that values collaboration, continuous learning, and a positive, ownership-driven culture.
Lead the engineering foundation behind reliable, measurable, and scalable AI-powered features.
Build evaluation frameworks, observability tooling, and diagnostic infrastructure for AI agents in production.
Combine hands-on software engineering with technical leadership and people management.
The company builds AI-powered products with a focus on reliability, scalability, and evaluation infrastructure. It operates as a fully remote, globally distributed team with an ownership-oriented culture.
Evaluate software engineering tasks for technical accuracy, realism, and reproducibility.
Investigate codebases, tests, and integration issues to identify technical weaknesses.
Provide clear, actionable feedback that directly improves AI training and evaluation workflows.
Jobgether is an AI-powered job platform that connects candidates to roles through objective, skill-based matching. It focuses on remote and freelance opportunities, with a data-driven recruitment process and a global candidate pool.
Own agent quality end-to-end for Bliro's chat and voice agents, including prompting, context, and model evaluation.
Build eval sets, LLM judges, and benchmarks to measure quality, latency, and cost with statistical rigor.
Establish a repeatable process for improving agent quality and set the long-term technology roadmap.
Bliro builds an AI assistant that handles desk work for field sales teams, with over one million customer touchpoints documented. Backed by leading investors and trusted by German Mittelstand companies, it's a small, high-performing team that values ownership and continuous improvement.
Evaluate AI model responses across diverse topics and use cases against established quality standards.
Write, test, and refine system prompts to influence model behavior and performance.
Identify and document model failures and provide clear, actionable feedback to product teams.
This company specializes in evaluating and improving advanced AI systems through content evaluation and prompt engineering. It operates as a partner company managing applications for this role, focusing on quality and consistency in AI behavior.
Develop and deploy cutting-edge AI/ML solutions to enhance the platform and improve student learning experiences.
Design, develop, and optimize LLM-powered agentic systems and APIs for real product use cases.
Collaborate with senior engineers and contribute to evaluation frameworks and MLOps pipelines.
Interview Kickstart specializes in interview preparation and career transitions into high-demand tech fields like AI, ML, and Data Science. Over 17,000 tech professionals have been guided by current and former hiring managers to land coveted positions at companies like Google and Amazon.
Build and operate production-grade agentic AI workflows for customer experience operations.
Develop MCP servers, secure integrations, and shared platform capabilities on AWS.
Establish evaluation, observability, and governance practices for reliable AI systems.
A technology company focused on integrating AI-powered workflows into customer experience operations. They foster a collaborative, inclusive remote culture emphasizing production-grade engineering, rapid prototyping, and mentorship.
Own product features end-to-end, from problem identification to production.
Work with AI agents to create specs and delegate development tasks.
Critically review AI-generated code to ensure reliability and quality.
Our partner company is seeking a Product Engineer to own features end-to-end in an AI-native environment. They are a remote, international team with an AI-first culture and a focus on using AI agents as primary development partners.
Own product features end-to-end, from problem statement to production.
Write specs with AI agents and critically review their code in complex systems.
Deliver reliable features handling financial transactions and evolve AI-first development.
Social Discovery Group creates social entertainment platforms that connect people online across cultures, addressing loneliness and disconnection. The company's international remote team works from everywhere, and it earned 'Great Place to Work' recognition in 2024-2025.
Build and maintain AI agents using large language models to complete multi-step tasks within Cayuse products.
Develop and extend MCP servers and integrate AI features into existing applications through platform APIs.
Write unit and integration tests, participate in code reviews, and ship code through CI/CD pipelines.
Cayuse provides integrated technology solutions that empower organizations to conduct globally connected research, advancing impact in science and discovery. The company is a team of highly skilled, multidisciplinary professionals building a remote-first culture with a focus on customer and employee values.
Design and develop specialized AI agents using Generative AI, LLMs, and agentic frameworks like LangChain and Semantic Kernel.
Build MCP servers and integrate AI applications with event-driven architectures, ensuring secure and traceable decisions.
Implement Human-in-the-Loop workflows and apply PromptOps practices for scalable, secure, and high-impact AI solutions.
The hiring company is a technology organization in Brazil focused on building AI-powered applications and agentic systems. It fosters a collaborative and international culture, with fully remote work and opportunities to work on innovative AI technologies.