Source Job

Global

  • We are seeking exceptional builders, operators, researchers, designers, and domain experts to help create new data systems for frontier AI.
  • You may be a fit if you excel at building software, designing expert interfaces, scaling human data programs, or developing evaluation frameworks.
  • We look for high-agency, fast-learning individuals who care about quality and want to work on the systems that make more capable AI possible.

Software Engineering Evaluation Frameworks

20 jobs similar to Open Application

Jobs ranked by similarity.

United States

  • Design and deliver high-quality AI evaluation data initiatives, from proposals through pilot execution and production readiness.
  • Recruit and manage subject-matter experts across technical domains, ensuring rigorous quality control frameworks.
  • Act as key interface with AI lab partners, converting pilots into scaled production engagements.

Jobgether uses AI-powered matching to connect candidates with roles quickly and fairly. They are a remote-first company that shares top-fitting candidates with hiring partners.

$236,000–$300,000/yr
US

  • Build the infrastructure for an AI-native design practice, including token systems and component libraries that AI agents can reliably build on.
  • Shape how Mixpanel looks and behaves in an AI-native world by collaborating on conversational and adaptive interfaces.
  • Own the bridge between design and code, ensuring design and code stay in sync and shipping production code.

Mixpanel provides AI-first digital analytics to help teams accelerate adoption and improve retention. Trusted by over 29,000 companies, it fosters a culture of bold bets, collaboration, and customer focus.

Global

  • Own design and implementation of agent harness layer for autonomous app generation.
  • Develop long-horizon agentic workflows with maximal autonomy and robust evaluation loops.
  • Research novel AI approaches and ship them to production with strong observability and debugging.

Sekai is building an AI-driven consumer platform for creating, remixing, and sharing interactive mini apps. The company is Series A, backed by Khosla, a16z, Mayfield, and A*, with $30M raised and a remote-first culture.

Asia

  • Contribute to the design and development of AI agents integrated into live production systems.
  • Build and maintain benchmark datasets to evaluate agent performance, accuracy, and safety.
  • Design and refine prompts, tool integrations, and agent workflows.

Binance is a leading global blockchain ecosystem behind the world's largest cryptocurrency exchange. It is trusted by over 320 million people in 100+ countries and fosters a results-driven, innovative culture with a flat structure.

US Unlimited PTO 16w maternity 16w paternity

  • Design, build, and deploy AI-powered solutions connecting Remote’s platform to customer systems and workflows.
  • Work at the frontier of practical AI, shipping reliable, observable, and secure systems in production.
  • Own customer outcomes from discovery through production rollout with high autonomy and direct influence on product roadmap.

Remote is a global HR platform that helps businesses recruit, pay, and manage international teams compliantly. The company fosters a future-focused, async culture with team members across 6 continents and emphasizes innovation, automation, and AI.

Vietnam

  • Build AI-powered workflows, assistants, agents, and automation systems across customer support, CRM, and finance operations.
  • Work with product and engineering teams to turn manual processes into scalable AI-native systems using LLMs, APIs, and internal data.
  • Prototype quickly, test with users, then productionize what works, ensuring evaluation, monitoring, and reliability.

Bjak believes people deserve smarter ways to plan, save and grow their money, starting with the first mobile-first insurance platform in Southeast Asia. They have teams worldwide with over 20 nationalities working from offices and remotely, seeking talented and driven people.

5w PTO 26w maternity 2w paternity

  • Lead architecture and delivery of distributed cloud-native applications for high concurrency and real-time data needs.
  • Design production-grade data pipelines and integrate ML/GenAI capabilities, including RAG and model serving.
  • Mentor peers, set engineering standards, and evolve legacy systems into microservices for scale.

Outreach, founded in 2014, is a complete agentic AI platform for revenue teams, powering use cases across sales motions with AI automation. The company serves world-leading enterprises like Databricks, SAP, and Siemens, and promotes diversity through employee resource groups.

Global Unlimited PTO

  • Build agentic workflows on Claude for research, account briefs, and content drafting.
  • Stand up a warehouse-backed marketing data model (e.g., Snowflake) as the single source of truth.
  • Ship weekly GTM/Marketing dashboards for the CEO, Head of Marketing, and Head of Sales.

Assured provides large insurers with software solutions for claims processing, from self-service claim-filing to fraud detection. The company is a fast-moving, collaborative environment where employees tackle deep challenges involving AI and digital experiences.

$148,000–$203,500/yr
Global Unlimited PTO

  • Own the design and evolution of core capabilities for the internal AI agent platform to support high-impact business workflows.
  • Partner with cross-functional teams to translate strategic goals into scalable AI agent solutions.
  • Build and maintain shared services, deployment patterns, and observability frameworks for reliable agent deployment.

ActiveCampaign is the autonomous marketing platform that enables teams to automate campaigns with AI agents. It is a global, multicultural company with a fast-growing, collaborative culture.

$173,000–$233,000/yr
Canada

  • Build and ship AI agents, APIs, and applications on Affirm's internal platform, owning the full lifecycle from architecture to production.
  • Turn messy business requirements from People Operations stakeholders into production systems, integrating with tools like Workday and Notion.
  • Design reliability infrastructure for multi-model LLM services, including structured output validation and quality controls.

Affirm is reinventing credit to make it more honest and friendly, giving consumers the flexibility to buy now and pay later without any hidden fees or compounding interest. The People Tech & Analytics team builds and owns the data, AI, and technology infrastructure for Affirm's People function, running like a product engineering group embedded in HR.

Global

  • Design and build AI and agentic systems that analyze organizational data to catch policy violations, compliance risks, and governance issues.
  • Architect and build knowledge graph systems that model organizational structure and relationships for use by agents.
  • Take a fuzzy, undefined problem, propose a real technical approach, defend it, then build it with minimal oversight.

Wand turns AI into labor, enabling humans and AI agents to operate together as a unified hybrid workforce with comprehensive management and oversight. The company is a world-class team with deep research expertise and real-world product execution, operating at scale inside some of the world's largest organizations.

$150,000–$165,000/yr
US

  • Map operational workflows across Sales and broader organization to identify friction and build scalable AI-powered solutions.
  • Partner with Revenue Operations to design and deploy intelligent workflows, automations, and playbooks.
  • Drive adoption by creating documentation, guardrails, and training to ensure systems are embraced and trusted.

1mind builds the next generation of revenue teams with a platform that deploys AI-powered Superhumans to serve buyers across the entire journey. As a rapidly growing startup, the company fosters a high-ownership, collaborative culture where every team member drives impact.

LATAM

  • Build and ship AI workflow solutions including prompt architectures, automation flows, lightweight agents, and API integrations.
  • Embed in business teams across Finance, HR, Legal, Marketing Ops, and Customer Success to understand workflows and create leverage.
  • Support the AI Program Office by building demos, prototypes, and contributing to vendor evaluations.

Backblaze is the object storage leader in the open cloud movement, fueling customer success with cloud storage built purposefully. Founded in 2007, the company generates over $100 million in revenue and manages over three billion gigabytes of data for 500,000+ customers in 175+ countries.

Global

  • Design and ship AI agents that replace high-volume manual marketing workflows like lead routing and campaign execution.
  • Rebuild marketing ops infrastructure with predictive lead scoring, lifecycle management, and agent-forward workflows.
  • Own the systems you ship end-to-end: monitor, iterate, and build fallback logic to ensure reliability.

Monte Carlo provides reliability infrastructure for production AI systems, unifying data and agent observability. Founded in 2019 and backed by leading investors, it is a growing startup with a small, high-trust marketing team.

$184,000–$260,600/yr
US

  • Own and drive Marketing's Agent Center of Excellence, including AI agent strategy and fleet management.
  • Manage ICs overseeing MarTech, Marketing Ops, and campaign systems, evolving MOPs with AI.
  • Lead cross-functional initiatives with full context on marketing strategy, operations, and data.

Airtable is a no-code app platform that empowers people to accelerate critical business processes. With over 500,000 organizations including 80% of the Fortune 100, it fosters a diverse, equal-opportunity culture focused on innovation.

Global Unlimited PTO

  • Design and build the trigger system linking GitLab events to AI-powered Duo Flows.
  • Architect for high availability and throughput in a multi-service environment.
  • Collaborate cross-departmentally to integrate triggers with the broader agent ecosystem.

GitLab is an intelligent orchestration platform for DevSecOps, enabling organizations to increase developer productivity and improve operational efficiency. With more than 50 million registered users and over 50% of the Fortune 100 as customers, GitLab fosters a high-performance, values-driven culture where every voice is valued and innovation flourishes.

  • Build internal tools and coach-facing features on top of LLMs like Claude and OpenAI.
  • Turn unstructured conversational data into structured, reliable signals for daily prioritisation.
  • Own the full stack of what you ship: frontend, backend, model integration, and deployment.

Best10 delivers online, highly personalised diet and lifestyle coaching through WhatsApp and its own platform, working with medical insurers to reduce costs from high-risk members. The company fosters a builder culture where teams ship working AI solutions rapidly and take pride in systems that outlast them.

Global

  • Embed with a brand or central team to map and redesign workflows using AI, ensuring adoption and measurable impact.
  • Build agentic systems, n8n flows, and Claude/Codex skills to orchestrate processes across functions.
  • Measure success via adoption and metrics like time saved, cost reduction, and new capabilities gained.

saas.group is a Software-as-a-Service portfolio company that acquires and grows B2B SaaS businesses. With a fully remote team of nearly 400 colleagues across 50+ countries, they operate over 25 brands.

Global

  • Build internal tools and coach-facing features on top of LLMs to turn unstructured conversational data into structured signals.
  • Design and maintain prompt chains, RAG pipelines, and agent workflows, rapidly prototyping from idea to demo in days.
  • Wire AI into existing systems, own full stack, and sit with coaches to build for their reality.

We deliver highly personalised diet and lifestyle coaching at scale through WhatsApp and our own technology platform. We partner with leading health insurers and employers, and our culture is about using AI to remove administrative work and enhance the personal coach-member relationship.

$258,000–$348,000/yr
United States Unlimited PTO

  • Own AI evaluation methods and operations for Figma's AI-powered experiences, defining quality dimensions and designing measurement frameworks.
  • Build and maintain evaluation frameworks, rubrics, golden datasets, and quality bars using human and automated approaches.
  • Produce clear readouts and dashboards to enable stakeholders to make confident shipping decisions.

Figma is on a mission to make design accessible to all, empowering teams to bring ideas to life through collaborative design and prototyping tools. With a growing team of passionate creatives and builders, Figma fosters a culture of growth and inclusivity.