Senior Software Engineer, AI Evaluation Infrastructure

What you will do:

• Design and ship features end to end: data model, API, background workers, and the UI when needed.
• Build and improve how agents and models are evaluated: running trials in sandboxed environments, scoring results, and turning thousands of runs into trusted outcomes.
• Improve benchmark task quality checks, including verifiers, rubric-based and LLM-as-a-judge grading, and task fairness.

What you bring:

• Typically 5+ years building and operating production backend systems, with ownership of at least one service from design through on-call.
• Strong Python, experience with Django or a similar framework, PostgreSQL, and a task queue such as Celery.
• Hands-on experience in LLM evaluation, good tradeoff judgment, ownership, and comfort with TypeScript and React when needed.

How we work:

• Written engineering guidelines applied consistently, with the same review bar for everyone.
• Business logic lives in services, hot paths stay cheap, and lint/tests pass before a PR is ready.
• We prefer deleting code to adding it, and simple designs to clever ones.

Values:

• Client first: putting clients at the center of everything we do.
• Start-up speed: moving fast, staying agile, and favoring action.
• AI forward: helping clients build the future of AI and amplifying productivity with AI in our own workflows.