Job Description

Salary: $165,200 - 223,600 per year Requirements:

• Were looking for 3+ years of non-internship professional software development experience.
• We need 2+ years of non-internship experience in system design or architecture, including design patterns, reliability, and scaling for new and existing systems.
• We require experience programming in at least one software language.
• Strong experience with C/C++ is required.
• A bachelors degree in computer science or equivalent experience is preferred.
• We value 3+ years of full software development lifecycle experience, including coding standards, code reviews, source control, build processes, testing, and operations.
• Exposure to high-speed networking, HPC interconnects, or GPU/accelerator systems such as RDMA, InfiniBand, libfabric, UCX, NCCL, or MPI is a strong advantage.
• Embedded systems experience is welcome.
• Prior AI/ML experience is not required, but a strong systems engineering background and willingness to learn are important.

• We build and improve low-level data-movement software that moves KV cache and activations across accelerators, servers, and heterogeneous memory over AWSs highest-performance network fabric.
• We profile real workloads, identify the true bottlenecks, and close the gap between functionality and peak performance.
• We work across the full stack, from network transport through inference frameworks, alongside teams building chips, runtimes, and models.
• We deliver features that go into our largest clusters and support our largest customers running production AI models.
• We focus on disaggregated inference, splitting LLM serving into separate prefill and decode pools and moving KV cache between them efficiently.
• We collaborate with infrastructure experts, hardware engineers, RTL engineers, scientists, and architects in a highly international team.
• We mentor junior engineers and also learn from senior and principal engineers through code reviews and knowledge sharing.
• We continuously iterate on performance and ship at scale in a fast-moving AI/ML environment.

• AI
• AWS
• EC2
• Embedded
• Fabric
• Hardware
• InfiniBand
• Support
• LLM
• Machine Learning
• Network
• RDMA
• Cloud
• Linux
• Web

More:

We are Annapurna Labs, an integral part of AWS, building the hardware and software foundations behind EC2. Our team sits at the intersection of AI, silicon, and networking, with a focus on networking solutions for machine learning and high-performance computing workloads on AWS. We offer a fast-paced environment with flexibility in working hours, strong work-life balance, mentorship, career growth opportunities, and a culture that values inclusion, learning, and knowledge sharing. The role is based in Cupertino, California, with a listed salary range of USD 165,200 to 223,600 annually, plus sign-on payments, restricted stock units, and comprehensive benefits including health coverage, 401(k) matching, paid time off, and parental leave.

last updated 37 week of 2026

Job Tags

Similar Jobs