Member of Technical Staff (AI Infrastructure Engineer)

The role

This is an individual-contributor infrastructure role on a growing team responsible for the compute layer behind model training and serving. The engineer partners with inference and research groups to stand up and tune large GPU clusters, with a focus on keeping long training runs and production endpoints reliable.

What you would do

• Stand up and maintain Kubernetes environments for model serving and training
• Administer Slurm-managed high-performance computing setups for distributed LLM training
• Create APIs and orchestration tooling spanning training pipelines and inference endpoints
• Schedule and allocate work across mixed hardware pools
• Profile systems, locate performance constraints, and ship fixes for training and serving
• Set up monitoring and alerting tailored to ML jobs on Kubernetes and Slurm
• Handle outages quickly and coordinate with other teams to keep critical workloads up
• Improve cluster usage and configure autoscaling for shifting demand

What they asked for

• Deep hands-on Kubernetes administration, including custom resources and operators
• Practical experience running Slurm for job scheduling and resource allocation
• Background deploying distributed training systems at scale
• Working knowledge of container orchestration and distributed system design
• Familiarity with transformer internals and distributed training approaches
• Experience operating GPU fleets and improving their utilization
• Prior work in SRE, DevOps, or platform roles centered on ML infrastructure
• Experience supporting both long-running batch jobs and highly available serving

Nice to have

• Custom controllers or operators built for ML workloads
• Advanced Slurm setups such as multi-cluster federation
• CUDA tuning and GPU cluster management know-how
• Exposure to TensorFlow or other distributed training libraries
• HPC, parallel computing, or high-speed networking background
• Infrastructure-as-code and GitOps practice
• Container image optimization for ML workloads