Similar Jobs
See allSr AI Architect, Application Security
GuidePoint Security
US
Security Architecture
Python
Threat Modeling
AI Security Engineer
Alpaca
Americas
Security Engineering
Python
Cloud Security
AI Evaluation Engineer
Jobgether
Global
Python
TypeScript
LLM Evaluation
Product Security Engineer
Emergence AI
US
Application Security
Threat Modeling
Kubernetes
Staff AI Engineer
Cobalt
US
LLM
Agent Orchestration
Distributed Systems
About the Role:
- Build trustworthy, public evaluations of AI cybersecurity capabilities as part of the AI Benchmarking Hub.
- Work with the benchmarking team to design, run, and ensure accurate results for new benchmarks.
- Report to the CTO and contribute to strategic input on AI cybersecurity evaluation.
What You’ll Own:
- Cybersecurity benchmarks: design, build, and maintain using Inspect; pitch and prototype new ideas.
- Secure evaluation environments: sandbox and harden against AI escape; stay current on AI agent interactions.
- Evaluation strategy: use threat modelling to decide what to measure and provide strategic input.
Infrastructure and Collaboration:
- Integrate with AI providers to set up benchmarks and ensure they run smoothly.
- Work with researchers, analysts, and engineers to ensure data accuracy and integration.
- Keep the suite running to evaluate new model releases quickly and painlessly.
Epoch AI
Epoch AI investigates trends in machine learning and the economic consequences of AI, developing a comprehensive knowledge base for policymakers. The research institute is committed to inclusive, equitable community and produces rigorous, accessible work on compute trends, data scarcity, and algorithmic progress.