Support model launch readiness by running evaluations, monitoring and interpreting results, and surfacing regressions or unexpected behavior changes to relevant stakeholders
Partner closely with policy and domain experts throughout the evaluation lifecycle — from identifying risks and scoping the right evaluation approach, to coordinating creation of new evals and ensuring existing ones remain current with evolving policies, threat vectors, and model capabilities
Work with cross-functional stakeholders to help manage evaluation outcomes, including interpreting results and driving mitigations where needed
Anthropic's mission is to create reliable, interpretable, and steerable AI systems. Their team is a quickly growing group of committed researchers, engineers, policy experts, and business leaders working together to build beneficial AI systems.
Lead safety assessments and threat modeling for new products.
Design and document safety mitigations for novel product risks.
Lead comprehensive red teaming efforts for new products and existing products
Runway is building AI to simulate the world through merging art and science. Their team consists of creative, open minded, caring and ambitious people who are determined to change the world.
Lead investigations, synthesize data, and translate signals into scalable actions.
Ensure timely, unbiased threat assessments and drive seamless transition to enforced solutions.
Establish proactive processes and performance metrics to provide partners time to address threats.
Reddit is a community of communities built on shared interests, passion, and trust, and is home to open and authentic conversations. With 100,000+ active communities and approximately 121 million daily active unique visitors, Reddit is one of the internet’s largest sources of information.
Manage customer inquiries across email, chat, Slack, focusing on complex cases requiring human judgment
Provide technical and creative support to enterprise customers via written communication and calls
Investigate technical issues and create detailed bug reports that save engineering time
Runway is building AI to simulate the world through merging art and science, believing that world models are at the frontier of progress in artificial intelligence. The Runway team consists of creative, open minded, caring, and ambitious people who are determined to change the world and continuously build impossible things.
Define and execute the roadmap for Sayari’s AI infrastructure.
Lead the strategy for model selection, fine-tuning, and deployment.
Establish the "ground truth" for Sayari AI.
Sayari is a venture-backed and founder-led global corporate data provider and commercial intelligence platform that serves financial institutions, legal and advisory service providers, multinationals, journalists, and governments. Their company culture is defined by a dedication to their mission of using open data to prevent illicit commercial and financial activity and they embrace cross-team collaboration.
Evaluate AI-generated responses using a structured safety rubric
Complete two independent evaluations per item
Provide concise, well-structured rationales in English
Welo Data, part of Welocalize, is a global AI data company delivering high-quality, ethical data to train the world’s most advanced AI systems. They have 500,000+ contributors and offer limitless opportunities for their global community to grow, contribute, and work on their terms.
Design and optimise AI-ready tools and APIs that enable LLM platforms to reliably interact with Canva's design capabilities.
Build and maintain evaluation frameworks to systematically measure tool-use accuracy across platforms.
Experiment with LLM orchestration and agent architectures – Develop Canva agents that any 3rd party provider can call to design quickly, efficiently and at scale.
Canva is a platform redefining how the world experiences design. They have a flagship campus in Sydney, with a second campus in Melbourne and co-working spaces in Brisbane, Perth, Adelaide, and Auckland, NZ.
Review contributor evaluations of model-generated responses to ensure adherence to project-specific guidelines.
Verify that contributors consistently apply all instructions and evaluation criteria when assessing model responses.
Confirm that contributors accurately identify factual errors, hallucinations, or missing information in model responses.
Welo Data, part of Welocalize, is a global AI data company delivering high-quality, ethical data to train the world’s most advanced AI systems. Welo Data has a diverse community in 100+ countries building smarter, more human AI, offering limitless opportunities for the global community to grow and contribute.
Plan, draft, and edit practical guidance and secondary content.
Develop and maintain online legal products for attorneys.
Leverage AI to streamline content development and editorial processes.
LexisNexis Legal & Professional provides legal, regulatory, and business information and analytics. As a digital pioneer, they were the first to bring legal and business information online with its Lexis® and Nexis® services.
Define voice, content pillars, channels, cadence, and growth loops.
Draft and edit posts and threads for key spokespeople; develop distinct voiceprints per exec.
Monitor AI narratives; respond quickly to FUD and misinformation; coordinate internally on sensitive topics.
Eigen Labs is building the infrastructure for a more trustworthy internet by making any digital service verifiable. With the fastest-growing developer ecosystem in crypto and backing from top-tier investors, they're at the inflection point where verifiable computing goes mainstream.
Lead data-driven product strategy initiatives supporting fraud detection and prevention systems
Design and deliver insights, reporting, and analytical frameworks to detect and mitigate fraud at scale
Define, track, and evaluate key metrics, including machine learning model performance and business impact
Maleda Tech is focused on protecting platform integrity by embedding measurement, experimentation, and insights into product defenses and user journeys. They are a global technology environment with a fraud & safety analytics team.