Sr QA Engineer (AI Systems & Platform)

Partner With Engineers:

• Pair with full-stack and backend engineers on the features they are shipping.
• Reproduce and triage bugs with enough detail that an engineer can fix them without a round-trip.
• Contribute to and help evolve our automated test suites alongside the QA Lead.

AI & Agent Testing:

• Help build and run evaluation pipelines for non-deterministic LLM outputs, prompt regression, model drift detection, and output quality scoring across the LiteLLM routing layer.
• Build and run automated tests for the agent orchestration layer, covering governance audit trail integrity, human-in-the-loop override behavior, and cross-agent handoffs.

Platform & Integration Testing:

• Test the Nango-based integration layer across connectors and the file ingestion pipeline including encryption, formatting edge cases, and audit trail continuity.
• Validate streaming response handling, latency thresholds, and graceful degradation when a model is unavailable or slow.

UX Quality:

• Test the trust-layer UX onboarding flows, progressive disclosure, uncertainty states, agent activity surfacing, and human-in-the-loop governance interfaces and help shape the standards as we go.
• Flag anything that would confuse a non-technical enterprise user.