Software Engineer – AI Inference Engine
Posted 2 months ago Expired
This job has expired
Looking for a job like Software Engineer – AI Inference Engine in or near San Mateo, CA? Upload your resume and we'll notify you when similar positions become available.
Upload Your ResumeAbout This Role
Optimize the performance and efficiency of FriendliAI's core AI inference engine. This role involves designing, implementing, and optimizing GPU kernels and supporting infrastructure for next-generation generative and agentic AI workloads, directly powering latency-critical and compute-intensive systems.
Responsibilities
- Design and optimize custom GPU kernels for AI (e.g., transformer and diffusion) workloads
- Contribute to the development of FriendliAI’s kernel compiler, memory planner, runtime, and other core components
- Collaborate with cloud and infrastructure engineers to ensure end-to-end inference performance
- Analyze performance bottlenecks across the software and hardware stack, and implement targeted optimizations
- Drive support for new model architectures and tensor compute patterns
- Maintain production-grade performance infrastructure, including profiling, benchmarking, and validation tools
Requirements
- 5+ years of experience in production or high-impact research environments
- Production-level expertise in Python and C++
- Bachelor’s or Master's degree in Computer Science, Computer Engineering, Electrical Engineering, or equivalent
- Experience developing machine learning frameworks or performance-critical runtime systems
- Hands-on experience writing and optimizing GPU kernels
- Hands-on experience profiling GPU kernels
- Experience working with generative AI models such as transformer and diffusion models
Qualifications
- Bachelor’s or Master's degree in Computer Science, Computer Engineering, Electrical Engineering, or equivalent
- 5+ years of experience in production or high-impact research environments
Nice to Have
- Experience developing machine learning compilers or code generation systems
- Familiarity with dynamic shape compilation, memory planning, and kernel fusion
- Contributions to inference engines, compilers, or high-performance numerical libraries
- Understanding of multi-GPU and distributed inference strategies
Skills
* Required skills
Benefits
About FriendliAI
FriendliAI is a Redwood City, CA-based startup building the next-generation AI inference platform that accelerates the deployment of large language and multimodal models with unmatched performance and efficiency.