Similar Jobs
See allSenior Evaluation Algorithm Engineer
Binance
Global
LLM
Python
Data Analysis
Senior ML Specialist
Jobgether
Brazil
Machine Learning
Python
Statistics
New Staff Software Engineer - Customer Facing Applied AI
Addepar
US
Python
AI/LLM
Evaluation
Technical Training & Quality Manager
Innodata
US
AI/ML
Data Annotation
Quality Management
Research Data Scientist
Innodata
United States
Python
SQL
LLMs
Your Role:
- Own the quality bar for Alpaca's models and agents.
- Define ground truth, metrics, and scoring methods for AI evaluations.
What You'll Do:
- Build repeatable eval loops and catch regressions before release.
- Partner with engineering and analytics to operationalize eval harnesses.
What We're Looking For:
- 6-10 years in quantitative data science or ML with measurement rigor.
- Strong Python, SQL, and statistical skills; LLM eval experience a plus.
Alpaca
Alpaca is a global leader in agent-first brokerage infrastructure, providing APIs for stocks, ETFs, options, crypto, and more. We're a diverse team of 400+ remote-first members backed by $400M in funding, committed to opening financial services to everyone.