Remote Software engineering Jobs · LLM Inference

Job listings

$175,000–$265,000/yr

  • Own end-to-end performance for workloads on d-Matrix's inference cloud platform.
  • Diagnose performance issues across the full serving path, from orchestration to inference internals.
  • Use a profiling-first method to isolate problems and drive fixes across teams.

d-Matrix is an AI infrastructure company that pioneered a memory-centric architecture for efficient AI inference. The culture values respect, collaboration, and direct communication, with humble experts driven by execution.

Global 5w PTO

  • Own and improve the AI inference serving stack, including prefill/decode disaggregation, continuous batching, KV-cache management, and speculative decoding.
  • Drive GPU performance engineering by identifying and eliminating bottlenecks using CUDA, Triton, profiling, and kernel optimization.
  • Set the serving and measurement roadmap, establish engineering standards, and mentor engineers while remaining hands-on with implementation.

Aleroy builds a unified platform for high-performance AI inference, enabling companies to deploy and run AI models at scale through a single API. We work in small, collaborative teams focused on solving complex engineering challenges in scalable AI infrastructure.

US Unlimited PTO

  • Owns the architectural direction for Code Metal's AI platform and the team building it.
  • Builds production-grade inference serving, model gateways, agent harnesses, and context-engineering services.
  • Instruments the stack end to end and drives experimentation and observability for reliable AI systems.

Code Metal provides automated software engineering using AI and formal verification to produce provably correct code. Founded in 2023 and backed by Accel, Salesforce Ventures, and others, it serves defense and enterprise customers from offices in Boston and San Francisco.