Source Job

Europe

  • Own and extend the offline eval suite across AI products, including datasets, judges, and metrics.
  • Build and maintain online quality dashboards tracking resolution rate, CSAT, LLM-as-judge signals, and more.
  • Close the production feedback loop by mining failure patterns and translating data into product decisions.

Python SQL Statistics Data Analysis Machine Learning

20 jobs similar to Product Data Scientist — AI Evaluation & Quality

Jobs ranked by similarity.

United States

  • Design and own the datasets and evaluation specifications for financial-domain LLMs, vision-language models, and AI agents, focusing on unstructured and multimodal financial data.
  • Translate customer goals into concrete dataset specifications, taxonomies, rubrics, and acceptance criteria, ensuring domain validity and statistical defensibility.
  • Develop evaluation methodology beyond surface accuracy, covering numerical consistency, hallucination rates, refusal appropriateness, and fairness across customer segments.

Innodata is a global data engineering company that enables the responsible advancement of artificial intelligence by providing data, evaluation frameworks, and human expertise. With over 36 years of experience, the company delivers high-quality data and solutions for Generative AI builders and adopters.

Global

  • Collaborate with Product, Engineering, and Commercial teams as the analytical backbone of your pillar, blending statistical rigor with commercial judgment.
  • Lead experimentation across your pillar, designing tests, sizing opportunities, and fostering evidence-based iteration.
  • Create self-serve dashboards and standardize metrics using AI-assisted workflows to accelerate output and focus on high-value problems.

Software Mind develops impactful solutions for global companies, working with tech giants and unicorns on transformative projects. The culture embraces openness, respect, grit, and combines employment with enjoyment among cross-functional engineering teams.

US Unlimited PTO

  • Design methodologies to transform raw alternative datasets into AI-ready data products.
  • Own data quality, metric definitions, and methodology standards for an AI-powered product.
  • Partner with engineering and product teams to scale data capabilities and drive customer insights.

YipitData provides alternative data and market intelligence to help clients make better business decisions. They foster a growth-oriented culture where advancement is driven by impact, not tenure, and offer a fully remote-friendly environment within the US.

US 6w PTO

  • Partner with product teams to define KPIs, dashboards, and roadmap planning for AI features.
  • Establish how success is measured for AI-powered product experiences including adoption and retention.
  • Design and interpret experiments to assess the impact of new AI capabilities and inform product decisions.

Grafana Labs builds the open observability cloud platform, Grafana Cloud, which helps organizations see, understand, and act on their data. With over 1,600 team members across 40+ countries, we are a remote-first company with an open-source legacy and a collaborative culture.

$200,000–$250,000/yr
US Unlimited PTO

  • Conduct in-depth analysis to shape product-led growth and B2B strategy.
  • Leverage AI tools to build self-service analytics enabling stakeholders to access reliable metrics.
  • Collaborate with product and engineering teams to drive experimentation culture and practices.

Descript is building a simple, intuitive, fully-powered editing tool for video and audio — an editing tool built for the age of AI. We are a team of 150 and have the backing of some of the world's greatest investors, including OpenAI, Andreessen Horowitz, Redpoint Ventures, and Spark Capital.

United States

  • Serve as a central contributor across customer support, AI quality assurance, and data analytics.
  • Collaborate with Product, Development, Content, and Success teams to ensure quality control and use tools like Tableau and Claude.
  • Drive actionable insights through data and support district clients with onboarding and technical assistance.

Coursemojo is an ed-tech company that has developed a curriculum-aligned, AI-powered assistant teacher named Mojo to support educators in differentiating instruction for diverse learners. It is a fast-paced, entrepreneurial team committed to equitable and transparent compensation.

$135,000–$150,000/yr
US Unlimited PTO

  • Partner with business stakeholders across Operations, Customer Success, Product, Growth, and Learning to drive data-informed decisions.
  • Own data modeling and engineering, ensuring data quality and trust in dashboards and pipelines.
  • Advance AI-native data practices and collaborate with engineering on analytics architecture.

Pathstream transforms frontline employees into strategic contributors through enterprise partnerships. We are a mission-driven team of ~60 employees with a tight-knit, hardworking culture focused on scaling our solution across the United States.

Global

  • Design, configure, and maintain AI agent workflows using Cursor and Claude Code for automated data system architecture.
  • Build and maintain a RAW → Base → Data Marts pipeline using dbt Core and implement business logic at the transformation layer.
  • Build comprehensive test suites using Great Expectations and ensure data quality through manual inspection and automation.

BiOptimizers helps people go from baseline health to peak biological performance with science-backed supplements and wellness tools. As a remote-first company, their globally distributed team focuses on clarity, autonomy, and operational excellence.

United States

  • Design and deliver high-quality AI evaluation data initiatives, from proposals through pilot execution and production readiness.
  • Recruit and manage subject-matter experts across technical domains, ensuring rigorous quality control frameworks.
  • Act as key interface with AI lab partners, converting pilots into scaled production engagements.

Jobgether uses AI-powered matching to connect candidates with roles quickly and fairly. They are a remote-first company that shares top-fitting candidates with hiring partners.

$132,800–$214,000/yr
Global Unlimited PTO

  • Build and maintain measurement infrastructure for outcome metrics to ensure metrics are live and credible.
  • Design and build an AI-native self-serve data layer for product builders to validate their own work.
  • Compile quantitative signal for strategic PMs and own data pipelines end to end.

Camunda is the enterprise platform for agentic orchestration, enabling organizations to coordinate AI agents, people, and systems across complex business processes. Trusted by over 700 organizations worldwide, including 9 of top 10 US banks, we are fully remote and global, and certified as a Great Place to Work.

$97,000–$121,000/yr
Canada

  • Own the development, operationalization and reporting of metrics for platform usability, adoption, and efficiency.
  • Assess data availability and quality, working with data engineers to ensure required datasets are extracted and transformed.
  • Analyze and visualize data to generate actionable insights for business partners.

Addepar is a global data and AI platform empowering investment professionals to turn complex financial information into actionable intelligence. With over 1,400 firms in nearly 60 countries managing nearly $9 trillion in assets, Addepar is a large company with a culture focused on innovation and collaboration.

$130,000–$180,000/yr
US Unlimited PTO

  • Own data products end-to-end, delivering best-in-class outputs with a focus on accuracy, client value, and efficiency.
  • Tackle complex data problems using strong analytical judgment to refine methodologies and solve challenges with creative thinking.
  • Use AI and tooling to drive leverage, automating workflows while maintaining a high bar for correctness and quality.

YipitData is the leading market research and analytics firm for the disruptive economy, analyzing billions of alternative data points for investment funds and Fortune 500 companies. Recognized by Inc. as a Best Workplace for three consecutive years, the company operates globally with offices in the US, APAC, and India, emphasizing transparency, ownership, and continuous mastery.

Global Unlimited PTO 12w maternity 12w paternity

  • Partner with the Product team to scope analytical questions and deliver findings that influence product decisions.
  • Own Mixpanel end-to-end: instrumentation strategy, event taxonomy, analysis, and relationships with PMs and engineers.
  • Surface customer insights from activation patterns and feature adoption to retention drivers, translating them into actionable product narratives.

Help Scout builds software for relationship businesses that earn loyalty through outstanding customer service. We are a fully remote team of 120+ teammates worldwide, with a culture of trust, ownership, and continuous learning.

United States

  • Design and own the quality of clinical datasets for training and evaluating health AI models.
  • Collaborate with cross-functional teams to translate customer goals into dataset specifications and evaluation plans.
  • Ensure clinical realism, statistical defensibility, and compliance in health AI evaluation workflows.

Innodata is a global data engineering company that enables the responsible advancement of artificial intelligence. With over 36 years of legacy, they deliver high-quality data and outstanding outcomes for AI builders and adopters.

$173,000–$233,000/yr
Canada

  • Build and ship AI agents, APIs, and applications on Affirm's internal platform, owning the full lifecycle from architecture to production.
  • Turn messy business requirements from People Operations stakeholders into production systems, integrating with tools like Workday and Notion.
  • Design reliability infrastructure for multi-model LLM services, including structured output validation and quality controls.

Affirm is reinventing credit to make it more honest and friendly, giving consumers the flexibility to buy now and pay later without any hidden fees or compounding interest. The People Tech & Analytics team builds and owns the data, AI, and technology infrastructure for Affirm's People function, running like a product engineering group embedded in HR.

Europe 6w PTO

  • Act as the dedicated analytics partner to product managers, supporting deep analysis of product usage, user behavior, and feature performance.
  • Deliver cross-product and segment-level insights that provide a broader perspective beyond individual product squads, influencing roadmap decisions.
  • Design, evaluate, and interpret A/B tests and experiments, ensuring statistical rigor and actionable outcomes.

Our partner is a leading workflow automation platform that helps users derive value from a highly technical, product-led organization. They offer a fully remote-first environment with a culture of transparency, autonomy, and high ownership, and host regular hackathons and innovation initiatives.

$150,000–$165,000/yr
US

  • Own end-to-end design for major product surfaces, partnering with PMs, engineers, and financial experts to define and ship solutions.
  • Use AI tools like Claude and Cursor to interrogate ideas, build prototypes, and align teams before design work begins.
  • Turn complex financial models and scenarios into intuitive, clear experiences that leave users feeling confident.

Boldin builds software that helps anyone create a comprehensive financial plan and understand how today's decisions shape their future. The company has raised over $20M from top investors, seen exponential growth, and is scaling its team.

Canada Unlimited PTO

  • Set the vision and strategy for Agent Experience, defining what world-class agent tooling looks like at Affirm.
  • Lead and develop a team of PMs, managing and coaching them to sharpen product judgment and improve execution.
  • Drive AI-first transformation in servicing by identifying where AI and automation can improve agent efficiency and reduce customer friction.

Affirm is reinventing credit to make it more honest and friendly, giving consumers the flexibility to buy now and pay later without hidden fees or compounding interest. The company offers competitive benefits including 100% subsidized medical coverage, dental and vision for you and your dependents.

Global 5w PTO

  • Own the metrics and analytical direction for your product area, working directly with Product Directors to influence strategy and roadmap priorities.
  • Manipulate and transform large data-sets to bring new insights, perform root-cause analysis, and guide strategy through metric frameworks.
  • Collaborate with the Product team to test new features and validate hypotheses through experimentation and A/B Testing.

Docplanner empowers patients by giving them access to leave and read reviews and provides doctors with technology to manage bookings easily. With over 2,500 employees globally, we maintain a startup mindset and embrace diversity.

Global

  • Design and analyze A/B tests, applying statistical methods to interpret results and support product decisions.
  • Own measurement frameworks and conduct exploratory analyses to inform product strategy.
  • Translate analytical outputs into actionable recommendations for product managers and engineers.

Tripadvisor connects people to experiences worth sharing, aiming to be the world's most trusted source for travel. It is a publicly traded company with a portfolio of travel brands, including Viator and TheFork, and a culture centered around data-driven decision making.