layiq
worthy; deserving; fitting; suitable.
A role, opportunity, or path that merits attention, time, and pursuit.
Loading LAYIQ…Job opportunity
Amazon
Boston, MA; Sunnyvale, CA; Seattle, WA
Source: Amazon careers · View original posting
From Amazon's posting. “We” and “our” refer to the employer.
We are looking for a Senior Inference Engineer to own inference for real-time multimodal conversational AI. This is a full-stack inference role: you will work across the entire path a model takes from research to production — shaping model architecture so it is servable, building the real-time runtime that serves it within hard latency budgets, and building the offline systems that train and reinforce it.
You will operate at the boundary of Science and Inference, taking frontier-scale speech and audio models and making them run within real-time latency budgets on production hardware.
You will co-design architectures with scientists to make them inference-friendly from inception, own the low-latency streaming serving path, and build the training and reinforcement-learning infrastructure that closes the loop. You will have the compute, data, and runway to solve problems that few teams in the world are positioned to tackle.
As a Senior Engineer, you will own a significant area of the inference stack end to end, drive its technical execution, contribute to the team's roadmap, and work closely with scientists and hardware partners to ensure our models run fast enough to feel human in real time — and at a cost that makes them viable at scale. You may go deep in one of the areas below while contributing across the others.
Key job responsibilities
Model Architecture & Inference Co-Design
Partner with research scientists to make model architectures servable from inception — surfacing the latency, memory, and cost implications of architecture choices before they are locked in Implement and optimize the inference path for large-scale multimodal models — attention and KV-cache mechanisms, multimodal/autoregressive decoding, and the compute primitives on the critical path
Apply efficiency techniques across the stack — quantization (per-tensor/per-channel/per- group, INT8/FP8/BF16), speculative decoding, operator fusion, and paged KV-cache — and quantify their quality/latency trade-offs
Develop and tune high-performance kernels for critical operations where off-the-shelf implementations leave performance on the table, integrating them into production serving with minimal overhead
Customize production serving frameworks (e.g., vLLM, PyTorch) for real-time streaming generative models that fall outside standard LLM serving patterns — sustained low-latency output under concurrent session load
Work with the evaluation team to enable offline inference that captures the quality dimensions unique to real-time conversation — latency sensitivity, audio quality, and interaction naturalness
The base salary range for this position is listed below. Your Amazon package will include sign-on payments and restricted stock units (RSUs). Final compensation will be determined based on factors including experience, qualifications, and location.
Amazon also offers comprehensive benefits including health insurance (medical, dental, vision, prescription, Basic Life & AD&D insurance and option for Supplemental life plans, EAP, Mental Health Support, Medical Advice Line, Flexible Spending Accounts, Adoption and Surrogacy Reimbursement coverage), 401(k) matching, paid time off, and parental leave. Learn more about our benefits at https://amazon.jobs/en/benefits.
LAYIQ is an independent job-discovery service. This listing does not imply a partnership with or endorsement by the employer. Review the original posting for current details and availability.
Employer posted: