Own the evaluation framework for AI-assisted systems, building labeled ground truth datasets and designing statistically sound audit and sampling methodologies.
Measure model performance across releases, run drift and override reviews, and produce evidence packages supporting deployment decisions.
Work closely with data scientists and engineers to validate scoring and generative AI capabilities while maintaining independence from build roles.
Design AI evaluations: define ground truth, metrics, and scoring methods for models and agents.
Build repeatable eval loops to track quality over time and catch regressions before release.
Translate eval results into actionable recommendations for system improvements.
Alpaca is a global leader in agent-first brokerage infrastructure, providing APIs for stocks, ETFs, options, crypto, and more. We're a diverse team of 400+ remote-first members backed by $400M in funding, committed to opening financial services to everyone.
Evaluate AI model responses across diverse topics and use cases against established quality standards.
Write, test, and refine system prompts to influence model behavior and performance.
Identify and document model failures and provide clear, actionable feedback to product teams.
This company specializes in evaluating and improving advanced AI systems through content evaluation and prompt engineering. It operates as a partner company managing applications for this role, focusing on quality and consistency in AI behavior.
Build and maintain production-grade machine learning infrastructure for AI governance and trust, including model serving, evaluation, and monitoring.
Develop low-latency inference systems, model registries, and evaluation pipelines for reliable model lifecycle management.
Design and implement red-team testing, drift monitoring, and continuous retraining loops using telemetry and audit data.
The company builds AI governance and trust infrastructure, including production ML systems for AI agents. It offers a remote-first, international engineering organization with a focus on security and reliability.
Own the build out of new agents, skills, and platform capability for teams across TLDR.
Build and deploy agents end to end, from design through implementation, evals, and rollout to internal users.
Partner with stakeholders across sales, editorial, and people ops to find where an LLM belongs in their process.
TLDR runs the largest network of tech newsletters in the world, with over 8 million subscribers covering startups, software engineering, AI, and more. Our 31-person full-time team is bootstrapped, profitable, and on track for $35M in revenue this year, with a culture of owning functions rather than slices.
Translate AI investment and operational decision needs into a phased technical roadmap.
Connect governed source data to enterprise platforms and build analytics-ready fact models.
Lead a matrixed team to deliver trusted reporting products with automated quality controls.
General Dynamics Mission Systems engineers a diverse portfolio of high technology solutions, products and services that enable customers to successfully execute missions across all domains of operation. With a global team of 12,000+ top professionals, we partner with the best in industry and offer a flexible work environment where contributions are recognized and rewarded.
Support the responsible integration of AI into educational and operational business processes.
Design, develop, test, and deploy AI-driven solutions and process automations.
Conduct AI readiness assessments and evaluate emerging AI platforms for organizational applicability.
Rackner is a software consultancy focused on building mission-critical systems for the U.S. government, working across cloud platforms, DevSecOps, AI/ML, and modern software engineering. We are a collaborative team that supports growth and development, offering clear advancement tracks and a creative, forward-thinking environment.
Design, develop, train, test, and deploy AI/ML models and solutions for enterprise environments.
Collect, clean, analyze, and prepare datasets for AI/ML modeling.
Evaluate AI/ML solutions for performance, security, reliability, and enterprise suitability.
PingWind provides cybersecurity, IT infrastructure, supply chain management, and professional services including system design and continuous improvement. It is an SDVOSB with offices in Northern Virginia and Huntsville, AL, and a focus on supporting federal government clients.
Lead the engineering foundation behind reliable, measurable, and scalable AI-powered features.
Build evaluation frameworks, observability tooling, and diagnostic infrastructure for AI agents in production.
Combine hands-on software engineering with technical leadership and people management.
The company builds AI-powered products with a focus on reliability, scalability, and evaluation infrastructure. It operates as a fully remote, globally distributed team with an ownership-oriented culture.
Collaborate with client teams to diagnose operational bottlenecks and develop testable hypotheses.
Build and test AI-powered prototypes using coding agents like Claude Code or Cursor.
Drive adoption by working directly with users and iterating on solutions until they are effective in live workflows.
Jobgether is an AI-powered job matching platform that connects candidates with hiring companies. The platform uses AI to review applications and provide a shortlist to employers, emphasizing efficiency and objectivity.
Test and evaluate AI model responses across diverse topics and conversation types.
Write and refine system prompts to shape model behavior and personality.
Apply detailed rubrics consistently to assess model performance and document findings.
We are a team dedicated to evaluating and improving advanced AI models and products. Our fast-moving environment emphasizes quality, independent judgment, and attention to detail.
Build the foundations of the EdgeRunner Research organization, including data pipelines, model evaluation, and efficient parallelization.
Own codebases in areas like distributed training, quantization, compression, or compute cluster management.
Collaborate with a team of self-starters who operate independently and translate business objectives into technical solutions.
EdgeRunner AI builds state-of-the-art AI models for the tactical edge, enabling warfighters to make faster decisions and interact with robotic platforms. As a Series-A startup, we move quickly, fostering a culture of initiative, ownership, and comfort with ambiguity.
Lead the development and optimization of machine learning models to detect and prevent AI security risks like prompt injection and jailbreaks.
Build reproducible training and evaluation pipelines on Reddit's ML platform, partnering with platform engineers to improve performance and reliability.
Set the technical vision and multi-quarter modeling roadmap, mentoring engineers and establishing best practices for responsible ML development.
Reddit is a community of communities, built on shared interests and authentic conversations, with 100,000+ active communities and 130 million daily active visitors. It is one of the internet's largest sources of information, fostering a culture of openness and trust.
Build the component layer around our layout synthesis engine, including API contracts, services, and evaluation gates.
Turn model retraining into a one-command job with built-in benchmarks and readable results for the whole team.
Own latency and cost budgets for learned capabilities and partner with infrastructure engineers on MLOps and deployment.
Higharc is a VC-backed startup that is changing how new homes are designed and built using spatial AI and generative floor plan technology. The company is fully remote, has raised over $175M, and values flexibility, collaboration, and asynchronous deep work.
Assess current-state service desk and contact center environments and design future-state support models leveraging AI, automation, and intelligent assistance.
Develop modernization strategies, business cases, and ROI models that demonstrate measurable cost savings and operational improvements.
Lead discovery sessions, prototype AI solutions, and support business development through proposals and executive presentations.
Our partner applies AI, machine learning, automation, and analytics to modernize customer and employee support environments for government and commercial clients. They offer a collaborative, innovation-focused environment with opportunities to work alongside experienced technical and mission-focused professionals.
Lead a team producing high-quality training data and evaluations for advanced AI models.
Review engineering-domain work, uphold quality standards, and drive operational improvements.
Coach and develop AI Tutors while collaborating with engineering and human data teams.
Jobgether is an AI-powered job platform that connects candidates with partner companies using objective matching. This role is with a partner company focused on AI training data, offering a collaborative, remote-first environment emphasizing autonomy and continuous learning.
Build and ship AI methods for federal research data with LLMs, NLP, RAG, knowledge graphs.
Benchmark AI against baselines and human reviewers to prove gains in accuracy, reliability, efficiency, cost.
Set standards for trustworthy AI and mentor data scientists through code review and guidance.
ARI builds AI methods for federal research-funding and administrative data, with clients including the NIH. It is a small multidisciplinary team of scientists, analysts, economists, and data scientists experienced with NIH data.
Define end-to-end requirements for AI capabilities, from model behavior to user experience.
Translate model capabilities and technical constraints into product decisions.
Work closely with ML and engineering teams on system design and iteration.
They develop AI-powered email applications and work at the intersection of user needs and model capability. The company is fast-moving and collaborative with a strong focus on technical ownership and innovation.
Own the full journey from raw data to business impact via dbt models on AWS and analytics use cases.
Design clean, scalable data models with strong data quality, testing, and governance.
Partner with Product, Engineering, and Finance to deliver data products and grow into financial reporting.
Sport Alliance provides data and software solutions for the fitness industry through Magicline and Finion, serving thousands of fitness businesses and over 10 million members. The company is an AI-first, data-driven organization that values ownership, curiosity, and flexible working.
Evaluate software engineering tasks for technical accuracy, realism, and reproducibility.
Investigate codebases, tests, and integration issues to identify technical weaknesses.
Provide clear, actionable feedback that directly improves AI training and evaluation workflows.
Jobgether is an AI-powered job platform that connects candidates to roles through objective, skill-based matching. It focuses on remote and freelance opportunities, with a data-driven recruitment process and a global candidate pool.
Owns the architectural direction for Code Metal's AI platform and the team building it.
Builds production-grade inference serving, model gateways, agent harnesses, and context-engineering services.
Instruments the stack end to end and drives experimentation and observability for reliable AI systems.
Code Metal provides automated software engineering using AI and formal verification to produce provably correct code. Founded in 2023 and backed by Accel, Salesforce Ventures, and others, it serves defense and enterprise customers from offices in Boston and San Francisco.