Software Engineering Director, Agentic Evaluations
Benefits
Unlimited PTO
Similar Jobs
See allVP, AI Platform
Engine
US
LLM
Infrastructure
AI Evaluation
Head of AI Engineering
AIOS
Global
AI Engineering
Software Engineering
Machine Learning
Engineering Manager, Agent Foundations: Agent Execution
GitLab
US
Python
Ruby
Go
Staff Agentic AI Engineer - Marketing
Netomi
North America
Python
LangGraph
RAG
Lead AI Engineer
Finom
Europe
Python
LLMs
RAG
Role Overview:
- Lead the development of credible, repeatable evaluations for live AI agents.
- Own the technology stack supporting the evaluation platform, from APIs to orchestration.
Key Responsibilities:
- Design reusable evaluation primitives applicable across software categories.
- Partner with data science teams to develop proprietary benchmarks.
- Promote the effective use of evaluations across agent-focused products.
Requirements:
- 10+ years of backend or full-stack development experience.
- 2+ years of engineering management experience.
- Hands-on experience evaluating AI agents and using frontier models.
Benefits:
- Total earnings of approximately $260,000-$320,000.
- Unlimited PTO, equity, and comprehensive parental leave.
Partner Company
This company focuses on building credible, scalable evaluations of AI agents from software vendors. It operates as a fully remote, inclusive team with a flexible culture and a focus on professional growth.