Similar Jobs
See allSenior Technical Product Manager - AI Agents, Evals & Reliability
Confidential
US
AI/ML
Product Management
Large Language Models
Lead AI Engagement Specialist
QAD | Redzone
India
Product Management
Data Literacy
AI Systems
Senior Manager, Data Science
Xsolis
US
Data Science
Machine Learning
LLM Fine-tuning
Senior Machine Learning (ML) Engineer
CCC Intelligent Solutions Inc.
US
Python
SQL
Machine Learning
Forward Deployed AI Engineer
Lunon
US
Fine-tuning
Post-training
Key responsibilities:
- Set the enterprise eval methodology: golden-set design, sampling, labelling and inter-rater agreement, LLM-as-judge validation.
- Build and run churn, payment-recovery, and demand forecasting models, choosing the right tool per task.
- Build evidence behind model routing: accuracy, cost and latency per task.
Qualifications:
- 12+ years in data science or machine learning with 4+ years leading teams.
- Has shipped LLM or agentic systems to production with eval suites.
- Deep experimental design, causal inference, and strong statistics.
Competencies and behaviors:
- Evidence first: brings a number with its error bar.
- Builder, not only researcher: measures success by what reaches customers.
- Pragmatic: picks simplest model that clears the bar; says no to research without BU sponsor.
Xplor
Xplor provides cloud-based technology solutions for small and medium-sized businesses, including payment processing. The company is unified by its purpose of helping people succeed.