Senior Runtime Engineer, AI Accelerator
About the company
Our client is a well-funded California start-up building custom ASICs for AI inference and training, delivering better performance per watt than general-purpose GPUs.
With first silicon in hand and customer deployments ramping, they are scaling the software team.
The role
You will own the software layer between ML frameworks and the accelerator hardware: the runtime that loads models, manages memory and schedules work across chips.
Working with compiler, kernel and hardware teams, you will decide how much of the silicon’s performance reaches real workloads.
What you’ll do
- Build and own the accelerator runtime in C++, covering memory management, DMA, streams and async scheduling
- Enable multi-chip execution and integrate with PyTorch, JAX and serving stacks such as vLLM
- Optimize end-to-end latency and throughput, and build profiling and debug tools
- Partner with compiler, kernel and hardware teams on interfaces and new silicon bring-up
- Mentor engineers and set technical direction for the runtime
What you’ll bring
- 7+ years in software, with 4+ building GPU or accelerator runtimes or drivers
- Expert C++ (17+), strong Python and solid Linux systems knowledge
- Experience with CUDA, ROCm/HIP, Level Zero or similar, and performance tuning on real hardware
- Understanding of LLM execution, including operator dispatch and KV cache management
- Experience leading technical designs, with a degree in CS, CE, EE or equivalent
Nice to have
- First silicon or pre-silicon bring-up, or Linux kernel driver experience
- Collective communication (NCCL, MPI) and interconnects such as RDMA or InfiniBand
- ML compilers (MLIR, XLA, Triton) or PyTorch device backends
- Prior AI accelerator, GPU or hyperscaler silicon experience
Why join
- Own a core part of a custom AI silicon stack and shape its architecture early
- Work with leading GPU and accelerator engineers and see your code in production
Location and package
- Location: On-site or hybrid in the San Francisco Bay Area, California
- Base salary: $220,000 to $300,000, depending on experience and level
- Equity: Meaningful early-stage stock options
- Benefits: Medical, dental and vision cover, 401(k), flexible PTO and relocation support
Interested? Get in touch for a confidential conversation about the role and the company.
Darwin Recruitment is acting as an Employment Agency in relation to this vacancy.
Vuldyn Naidoo