Senior Runtime Engineer, AI Accelerator

About the company

Our client is a well-funded California start-up building custom ASICs for AI inference and training, delivering better performance per watt than general-purpose GPUs.

With first silicon in hand and customer deployments ramping, they are scaling the software team.

The role

You will own the software layer between ML frameworks and the accelerator hardware: the runtime that loads models, manages memory and schedules work across chips.

Working with compiler, kernel and hardware teams, you will decide how much of the silicon’s performance reaches real workloads.

What you’ll do

• Build and own the accelerator runtime in C++, covering memory management, DMA, streams and async scheduling

• Enable multi-chip execution and integrate with PyTorch, JAX and serving stacks such as vLLM

• Optimize end-to-end latency and throughput, and build profiling and debug tools

• Partner with compiler, kernel and hardware teams on interfaces and new silicon bring-up

• Mentor engineers and set technical direction for the runtime

What you’ll bring

• 7+ years in software, with 4+ building GPU or accelerator runtimes or drivers

• Expert C++ (17+), strong Python and solid Linux systems knowledge

• Experience with CUDA, ROCm/HIP, Level Zero or similar, and performance tuning on real hardware

• Understanding of LLM execution, including operator dispatch and KV cache management

• Experience leading technical designs, with a degree in CS, CE, EE or equivalent

Nice to have

• First silicon or pre-silicon bring-up, or Linux kernel driver experience

• Collective communication (NCCL, MPI) and interconnects such as RDMA or InfiniBand

• ML compilers (MLIR, XLA, Triton) or PyTorch device backends

• Prior AI accelerator, GPU or hyperscaler silicon experience

Why join

• Own a core part of a custom AI silicon stack and shape its architecture early

• Work with leading GPU and accelerator engineers and see your code in production

Location and package

• Location: On-site or hybrid in the San Francisco Bay Area, California

• Base salary: $220,000 to $300,000, depending on experience and level

• Equity: Meaningful early-stage stock options

• Benefits: Medical, dental and vision cover, 401(k), flexible PTO and relocation support

Interested? Get in touch for a confidential conversation about the role and the company.

Darwin Recruitment is acting as an Employment Agency in relation to this vacancy.

Vuldyn Naidoo