Member of Technical Staff (AI Inference Engineer)

The role

This is an individual-contributor engineering position on the team that operates the inference stack behind every query the product handles. The work covers bringing many model architectures online under strict latency and cost constraints, using Rust, Python, and GPU kernel languages together. The role also carries ownership of production reliability for the serving path.

What you would do

• Add support for retrieval, generation, and multimodal transformer models, covering weight loading, scheduling, and cache handling
• Move internal GPU kernels onto NVIDIA's CuTe DSL for current and next-generation hardware
• Build out a Rust-based internal inference server to replace Python-bound bottlenecks and absorb rising request volume
• Profile the full request path from ingress through batching and kernel execution to remove bottlenecks
• Create monitoring, alerting, and automated recovery to detect regressions before users notice
• Investigate and learn from production incidents

What they asked for

• 3+ years of professional software engineering, with substantive work on ML inference or performance-heavy systems
• Hands-on GPU programming and performance tuning experience with tools such as CUDA, Triton, or CUTLASS
• Experience operating distributed production systems under real traffic load
• Working knowledge of at least one deep learning framework such as PyTorch, JAX, or TensorFlow
• Understanding of GPU hardware fundamentals including memory hierarchy, warp scheduling, and tensor cores
• Familiarity with modern LLM architectures and inference optimizations like quantization and speculative decoding

Nice to have

• Experience with ML compilers or framework internals such as torch.compile or custom operators
• Knowledge of multi-GPU communication via NCCL, NVLink, InfiniBand, or RDMA
• Work with low-precision formats such as INT8, FP8, or FP4
• Use of GPU profilers and low-level debugging or assembly analysis tools
• Container orchestration and GPU autoscaling experience