Design and build automated evaluation pipelines for AI agents.
Develop testing frameworks for reasoning, multi-turn conversations, and adversarial scenarios.
Own the evaluation architecture and integrate observability tools in a fully remote team.
A company building production-grade AI agents and systems. They operate in a fully remote, globally distributed environment with a culture focused on innovation, autonomy, and continuous improvement.
Build tools for understanding traces, evaluating agent behavior, and turning results into improvements.
Run experiments on Mastra's primitives to find where they fall short and improve configuration and behavior.
Help turn findings into capabilities developers can use in their own applications.
Mastra is the open-source TypeScript framework for building AI agents. We're a small, fully remote team backed by Y Combinator and Spark Capital with $35M raised.
Lead the engineering strategy and execution for evaluations of AI agents, owning the core evaluation platform.
Design scalable evaluation infrastructure, APIs, workflows, and production systems across software categories.
Mentor and develop a team of engineers as the technical authority on agentic evaluation.
This company focuses on building credible, scalable evaluations of AI agents from software vendors. It operates as a fully remote, inclusive team with a flexible culture and a focus on professional growth.
Build and operate production-grade agentic AI workflows for customer experience operations.
Develop MCP servers, secure integrations, and shared platform capabilities on AWS.
Establish evaluation, observability, and governance practices for reliable AI systems.
A technology company focused on integrating AI-powered workflows into customer experience operations. They foster a collaborative, inclusive remote culture emphasizing production-grade engineering, rapid prototyping, and mentorship.
Build evaluations to measure and improve AI reasoning in complex, high-stakes decisions.
Collaborate with engineers, product, and domain experts to turn observed failures into testable improvements.
Own the quality of shipped systems, from code reviews to monitoring and integration.
Elicit builds AI systems that help research and business teams make evidence-based decisions on high-stakes questions. They are a remote-friendly, collaborative team that values rigorous thinking, experimentation, and in-person retreats.
Design, build and operate production AI agents, including an autonomous coding agent that works from Jira and Slack, opening PRs and fixing CI failures.
Build the retrieval and context layer behind our AI brain: embeddings, semantic search, knowledge-graph modeling, and integration with Snowflake and internal docs.
Harden agents against prompt injection and over-broad permissions, and drive cost and latency down using caching and structured outputs.
Weedmaps is a global leader in the cannabis industry, providing a platform for consumers and businesses. The company has a collaborative culture focused on innovation and community, serving the U.S. and worldwide.
Design and deliver production-grade agents that investigate, reason, and act on live observability data.
Own agent work from rough prototype to production, including evals.
Extend the agentic workspace with new canvas capabilities, MCP server, and skills.
Honeycomb provides an observability platform that helps engineers understand and debug complex systems. It is a fully distributed company of over 200 people, with a culture that values impact, autonomy, and inclusivity.
Curate high-quality code examples and datasets for LLM training and evaluation.
Develop and assess AI-generated software across multiple programming languages and the full SDLC.
Collaborate with research teams to design verification mechanisms and improve coding benchmarks.
This partner company specializes in evaluating large language models and improving AI systems through rigorous engineering benchmarks. It offers a remote, collaborative culture where engineers and researchers advance AI evaluation workflows together.
Own agent quality end-to-end for Bliro's chat and voice agents, including prompting, context, and model evaluation.
Build eval sets, LLM judges, and benchmarks to measure quality, latency, and cost with statistical rigor.
Establish a repeatable process for improving agent quality and set the long-term technology roadmap.
Bliro builds an AI assistant that handles desk work for field sales teams, with over one million customer touchpoints documented. Backed by leading investors and trusted by German Mittelstand companies, it's a small, high-performing team that values ownership and continuous improvement.
Build and ship LLM-powered applications and AI agent workflows.
Design systems for reasoning, planning, memory, tool use, and multi-step execution.
Optimize AI systems for quality, latency, and cost, and establish production practices for observability and evaluation.
Gina's Tech Jobs is a technology recruiting firm that connects companies with engineering and IT professionals. They focus on remote and onsite placements, offering personalized recruitment services and career support.
Design and architect next-generation AI agents for observability that transform incident investigation and remediation.
Drive technical direction, architecture, and best practices using AI/ML models to reduce mean time to resolution.
Collaborate with cross-functional teams and mentor engineers to deliver secure, scalable features.
ServiceNow is an AI platform that connects AI, data, and workflows to help enterprises work smarter and faster. The company serves 85% of the Fortune 500 and fosters an AI-native culture where technology and talent are unstoppable.
Build LLM-based agents on the platform's scaffolding, integrating tool calls, internal APIs, and guardrails.
Take agents to production on AWS with containers, CI/CD, secrets, permissions, and security controls.
Define and run evals, monitor with Langfuse, and document runbooks for independent operation.
Muttdata builds innovative Data Products and Machine Learning solutions to help companies solve complex business challenges. It is a fast-growing, remote-first startup that values collaboration, continuous learning, and a positive, ownership-driven culture.
Build RL environments, agentic systems, LLM pipelines, and evaluation frameworks for real-world AI use cases.
Develop benchmarks and evaluation harnesses to assess model quality across accuracy, safety, latency, and cost.
Conduct fine-tuning and model experiments, deploy self-hosted models, and document reproducible methodologies.
The employer is an organization focused on applied AI research, building practical and reusable AI systems for real-world use cases. It values curiosity, accountability, innovation, collaboration, and continuous learning in a remote environment.
Design and build challenging evaluations and executable environments for coding and long-horizon agents.
Develop scoring logic and verifiers to detect reward hacks and trace failures through model outputs and harness code.
Build and scale RL data pipelines for generating and curating challenging coding and agentic tasks.
Prime Intellect is building the open superintelligence stack, providing infrastructure for post-training frontier AI models and open-source models trained for long-horizon tasks. The company has raised $150M from top investors and is a mission-driven team passionate about accelerating science and AI.
Evaluate software engineering tasks for technical accuracy, realism, and reproducibility.
Investigate codebases, tests, and integration issues to identify technical weaknesses.
Provide clear, actionable feedback that directly improves AI training and evaluation workflows.
Jobgether is an AI-powered job platform that connects candidates to roles through objective, skill-based matching. It focuses on remote and freelance opportunities, with a data-driven recruitment process and a global candidate pool.
Own the evaluation framework for AI-assisted systems, building labeled ground truth datasets and designing statistically sound audit and sampling methodologies.
Measure model performance across releases, run drift and override reviews, and produce evidence packages supporting deployment decisions.
Work closely with data scientists and engineers to validate scoring and generative AI capabilities while maintaining independence from build roles.
DEFCON AI leverages artificial intelligence, mathematical optimization, and data analytics to help organizations optimize complex systems and manage disruptions. The company offers a fully remote, results-based environment with unlimited PTO, comprehensive benefits, and a focus on empowering employees to work independently.
Lead the engineering foundation behind reliable, measurable, and scalable AI-powered features.
Build evaluation frameworks, observability tooling, and diagnostic infrastructure for AI agents in production.
Combine hands-on software engineering with technical leadership and people management.
The company builds AI-powered products with a focus on reliability, scalability, and evaluation infrastructure. It operates as a fully remote, globally distributed team with an ownership-oriented culture.
Own end-to-end development of AI observability features, from ingestion to visualization.
Architect data models and infrastructure for observing AI agents across Honeycomb.
Partner with product and design to build intuitive workflows and share best practices.
Honeycomb is an observability platform that helps developers debug complex distributed systems, serving customers like HelloFresh and Slack. The company has surpassed 200 employees, closed Series D funding, and fosters a distributed-first, inclusive, and high-trust culture.
Set the technical direction for AI engineering across the team: agent architecture patterns, evaluation methodology, deployment and monitoring strategies
Design the AI platform layer, including shared agent frameworks, tool integrations, and evaluation infrastructure
Work directly with clients on the most complex engagements, identifying new problem domains and ensuring production quality
Addepar is a global data and AI platform empowering investment professionals to turn complex financial information into actionable intelligence. With over 1,500 firms in 60 countries managing nearly $10 trillion in assets, Addepar fosters a culture of ownership, collaboration, and innovation.
Lead the design and delivery of AI/ML systems, defining architecture and establishing engineering standards for agentic workflows.
Build and maintain RAG pipelines, agentic workflows, and production system prompts with eval and observability.
Mentor engineers and take end-to-end ownership of features, including debugging and hardening production systems.
Robots and Pencils designs AI systems for a human world, pairing engineering with creativity to ship production-ready AI in 30 to 45 days. They are a company where every role contributes directly to integrating AI into enterprise operations, with a culture that values ownership, craft, and measurable impact.