AI Software Engineer

About This Role

This role involves building a world-class inference infrastructure to power Zoom's AI services, focusing on high efficiency, scalability, and cost optimization across various AI applications. The AI Software Engineer will design, optimize, and scale runtimes and services for AI models to ensure reliable, high-performance AI experiences for millions of users.

Responsibilities

• Develop and optimize AI runtimes for LLMs, ASR, and MT systems with a focus on performance and cost efficiency
• Apply GPU-level optimization techniques including CUDA, kernel fusion, and memory throughput improvements
• Implement inference optimizations such as TorchCompile, graph optimization, KV cache, and continuous batching
• Build scalable, highly available infrastructure services to support enterprise-grade AI workloads
• Optimize models for edge devices (laptops, PCs and mobile devices) as well as large-scale cloud deployments
• Continuously improve latency, throughput, and efficiency across serving pipelines
• Rapidly integrate and optimize new industry models to stay ahead in AI infrastructure

Requirements

• Track record of building scalable, reliable AI infrastructure under real-world production constraints
• Strong expertise in GPU programming and optimization (CUDA, kernel-level development)
• Deep experience with transformer-based models and inference frameworks (vLLM, TensorRT-LLM, SGLang, ONNX Runtime)
• Proficiency in Python and C++
• Hands-on experience with PyTorch (TorchCompile, graph-level optimization) and/or TensorFlow
• Knowledge of low-level hardware concepts (GPU memory hierarchy, caching, vectorization)
• Familiarity with cloud platforms (AWS, GCP, Azure)
• Familiarity with AI deployment tools (Docker, Kubernetes, MLflow)

Qualifications

• Track record of building scalable, reliable AI infrastructure under real-world production constraints.

Nice to Have

• Java proficiency

Skills

* Required skills

Benefits