[Distinguished Lecture Series] RNGD: Design and Implementation of PyTorch-Compatible Programming Interfaces for Tensor Contraction Processors
The future of AI is in its sustainability. As more complex and heavier AI models are being applied to diverse industries, the demand for energy increases exponentially to an unprecedented numbers. Conventional designs may deliver more power yet struggles to deliver sustainable scaling, and hence the need for a new powerful yet efficient solution for today and tomorrow's models.
To make this hardware accessible and practical for end users, we must go beyond providing a compiler and runtime system, a rich set of programming interfaces is essential. In this talk, we share our experiences, insights, and lessons learned from the trials and errors of designing these interfaces, and discuss what it takes to integrate such specialized hardware into widely used software ecosystems.
June Paik is Founder and CEO of FuriosaAI. Before starting the journey of FuriosaAI in 2017, he had accumulated deep experience in developing world-class semiconductor products—from hardware to software—at major chip companies such as AMD and Samsung Electronics. As Founder and CEO of FuriosaAI, he is setting the product vision and orchestrating the engineering development of the world’s best AI chips. He received his master’s degree in electrical engineering from Georgia Tech.
Sol Kim is a Member of the Technical Staff at FuriosaAI and a seasonsed developer with 10+ years of software engineering, of which 6+ years of software engineering were in FuriosaAI. He designs and develops the FuriosaAI HAL (hardware abstraction layer & language) and runtime, a runtime which through software out-of-order achieves 99%+ of NPU utilization on LLM serving.
