AI Evaluations Engineer
Remote regions
North AmericaBenefits
Unlimited PTO
Similar Jobs
See allNew Staff Software Engineer - Customer Facing Applied AI
Addepar
US
Python
AI/LLM
Evaluation
Software Engineering Director, Agentic Evaluations
Partner Company
US
Python
TypeScript
Agentic AI
Applied AI Research Engineer
Appen
India
Machine Learning
Software Engineering
Reinforcement Learning
Senior Product Engineer (Agent Training and Evals)
Prolific
UK
Python
TypeScript
Google Cloud Platform
Forward Deployed Engineer
Stello
US
LLMs
AI Agents
Software Engineering
What you'll do:
- Study how engineers and customers use Archie in the field and turn failures into actionable evaluations.
- Recreate real-world engineering tasks and failure modes in repeatable environments.
- Build and refine evaluation methodologies with automated judges and human evaluation.
About you:
- Technical background in a physical engineering domain such as power systems, mechanical, aerospace, or robotics.
- Experience with AI evaluations and designing reasoning about agentic AI systems.
- Comfortable working with customers, subject matter experts, partners, and internal teams.
Why P-1 AI:
- Mission-driven to build superintelligence for engineering.
- Well-funded with strong backing from leading investors and AI luminaries.
P-1 AI
P-1 AI is building Archie, an AI engineer agent for the physical world that works alongside human engineering teams. The company recently raised a $50 million Series A led by NEA and is driven by the mission of building superintelligence for engineering.