Software Engineer – AI Inference Engine

About This Role

Optimize the performance and efficiency of FriendliAI's core AI inference engine. This role involves designing, implementing, and optimizing GPU kernels and supporting infrastructure for next-generation generative and agentic AI workloads, directly powering latency-critical and compute-intensive systems.

Responsibilities

• Design and optimize custom GPU kernels for AI (e.g., transformer and diffusion) workloads
• Contribute to the development of FriendliAI’s kernel compiler, memory planner, runtime, and other core components
• Collaborate with cloud and infrastructure engineers to ensure end-to-end inference performance
• Analyze performance bottlenecks across the software and hardware stack, and implement targeted optimizations
• Drive support for new model architectures and tensor compute patterns
• Maintain production-grade performance infrastructure, including profiling, benchmarking, and validation tools

Requirements

• 5+ years of experience in production or high-impact research environments
• Production-level expertise in Python and C++
• Bachelor’s or Master's degree in Computer Science, Computer Engineering, Electrical Engineering, or equivalent
• Experience developing machine learning frameworks or performance-critical runtime systems
• Hands-on experience writing and optimizing GPU kernels
• Hands-on experience profiling GPU kernels
• Experience working with generative AI models such as transformer and diffusion models

Qualifications

• Bachelor’s or Master's degree in Computer Science, Computer Engineering, Electrical Engineering, or equivalent
• 5+ years of experience in production or high-impact research environments

Nice to Have

• Experience developing machine learning compilers or code generation systems
• Familiarity with dynamic shape compilation, memory planning, and kernel fusion
• Contributions to inference engines, compilers, or high-performance numerical libraries
• Understanding of multi-GPU and distributed inference strategies

Skills

* Required skills

Benefits