Evaluate model-generated content across multiple modalities including text, images, audio, and video.
Apply defined quality rubrics such as factuality, consistency, and aesthetics to assess outputs.
Conduct independent research on unfamiliar topics to make well-supported evaluation judgments.
Our client is a global technology company that helps businesses build, train, and manage AI systems. They offer flexible, remote work with variable workload and a focus on high-quality model evaluation.
Define what 'good' means for non-deterministic LLM systems running on bounded local models, building evaluation infrastructure that catches regressions.
Build production prediction and trend capability—forecasting, anomaly detection, and early-warning signals—over operational telemetry that feeds directly into the system.
Work across data scientists, ML/inference engineers, frontend, and product in an enterprise environment with real security and compliance constraints.
ScienceLogic is redefining IT operations for the modern enterprise with an AIOps platform that enables self-healing, self-optimizing systems. The company helps enterprises and service providers achieve unified visibility across hybrid and multi-cloud environments, accelerating digital transformation through automation, AI, and analytics.
Evaluate and rank model outputs, stress-test models for failure modes, and create high-quality datasets with detailed rubrics.
Annotate and correct multimodal data, maintain consistency through calibration exercises, and adapt to evolving task types.
Report on model performance trends and provide clear feedback to cross-functional partners on model successes and failures.
Cohere is a security-first enterprise AI company that builds cutting-edge foundation models and end-to-end products for real-world business problems. It is a global technology company with offices in Toronto, San Francisco, London, New York, Montreal, Seoul, Germany, and Paris, staffed by a team of passionate researchers, engineers, and designers.