Research Engineer - Inference

Role Impact:

• Build the inference systems that power Prime Intellect’s RL research, scaling toward 1–3T parameter models on NVL72 clusters.
• Improve rollout throughput and make frequent model updates practical.

Key Responsibilities:

• Extend prime-rl’s vLLM integration for new model architectures with small-scale validation before larger runs.
• Optimize multi-node MoE inference, expert parallelism, and prefill/decode separation.
• Reduce trainer-to-inference weight-update overhead through sharding-aware transfer and communication overlap.

Requirements:

• Experience building or optimizing GPU inference systems with understanding of inference-engine internals.
• Strong Python skills and experience with PyTorch or comparable ML frameworks.
• Ability to profile end-to-end workloads and turn measurements into verified performance improvements.

Benefits:

• Competitive cash compensation with equity incentives.
• Flexible remote or in-person work in San Francisco, plus visa sponsorship and relocation assistance.
• Quarterly off-sites, hackathons, conferences, and learning opportunities.