​A typical SmartNIC (SNIC) integrates a processor, consisting of Arm CPU and accelerators, with a conventional NIC. The processor is designed to energy-efficiently execute functions frequently used by network-intensive datacenter applications. With such a processor, the SNIC has promised to notably increase overall energy efficiency of datacenter servers. Nevertheless, the recent trend of integrating accelerators into server CPUs for these functions sparks questions on the SNIC processor’s superiority over a host processor (i.e., server CPU with accelerators) in system-wide energy efficiency especially under given tail latency constraints. Answering this pressing question, we first take an Intel Xeon processor, integrated with various accelerators (i.e., Quick Assist Technology), as a host processor, and then compare it to an NVIDIA BlueField-2 SNIC processor. This reveals that (1) the host accelerators, coupled with a more powerful memory subsystem, can outperform the SNIC accel erators and (2) the SNIC processor can improve systemwide energy efficiency over the host processor only at low packet rates for most functions under tail latency constraints. To offer high system-wide energy efficiency without hurting tail latency at any packet rates, we propose HAL consisting of a hardware-based load balancer and an intelligent load balancing policy implemented inside the SNIC. When HAL detects that the SNIC processor cannot efficiently process a given function beyond a specific packet rate, it limits the rate of packets to the SNIC processor and lets the host processor handle the excess. HAL works for stateless functions with conventional PCIe-attached SNICs for now, but we also demonstrate that HAL can work for stateful functions as effectively with a CXL-attached SNIC. We implement HAL with FPGA connected to the BlueField-2 SNIC and show that HAL makes the SNIC processor improve energy efficiency and throughput of the server by 31% and 10%, respectively, withou t notably hurting tail latency.​

​I am the W.J. ‘Jerry’ Sanders III – Advanced Micro Devices, Inc. Endowed Chair Professor at the University of Illinois, Urbana-Champaign and a fellow of ACM, IEEE, and NAI. From 2018 to 2020, I took a leave of absence and as a Sr. Vice President at a major memory manufacturing company I led the development of next-generation DRAM products, including the industry's first HBM-PIM that will play a significant role in shaping the future computing landscape. I have published more than 250 refereed articles to highly-selective conferences and journals in the field of digital circuit, processor architecture, and computer-aided design. The top three most frequently cited papers have more than 5000 citations and the total number of citations of all my papers approaches 16000. I was a recipient of many internationally recognized awards, including ACM/IEEE Most Influential ISCA Paper Award, SIGMICRO 2021 Test of Time Awards, and Intel Outstanding Researcher Award. I am a hall of fame m ember of all three major computer architecture conferences, IEEE HPCA , MICRO, and ISCA.​