Scaling AI Intelligence at the Edge
Event Details
- Time: 4:00 - 5:00 p.m. Eastern Time
- Format: Zoom
Abstract
Deploying frontier large language models directly on battery-powered edge devices such as smartphones and wearables promises sub-10 millisecond latency, strict data privacy, and offline reliability, but it must operate within a rigid 2-5 watt passive thermal and battery envelope. In this environment, adding raw neural processing unit compute does not by itself scale model quality because model capacity, autoregressive decoding bandwidth, and data-movement energy compete with one another. Fetching a single byte from mobile DRAM or NAND flash can consume 100-200 times more energy than performing arithmetic on it.
The talk examines the hardware and algorithmic foundations required to overcome this edge memory and energy wall. Topics include compute-near-memory hardware that reduces bit-movement energy through 3D near-memory computing, bank-level processing-in-memory, and in-memory compute; algorithmic sparsity using sparse mixture-of-experts, INT4 quantization, KV-cache compression, router lookahead, and speculative decoding; and emerging 3D edge-memory candidates evaluated against flash offloading and high-bandwidth flash across package capacity, active-read energy, standby power, latency, endurance, thermal sensitivity, and cost.
Speaker Bio
Parag Agashe is Vice President of Silicon Engineering at Google, leading custom silicon and IP initiatives. A semiconductor and mobile systems veteran with over three decades of industry experience, he previously served as Senior Vice President of Engineering at Qualcomm, where he led large-scale mobile SoC productization for smartphone and PC, wireless system architecture across 3G/4G, and location technologies. An accomplished inventor with numerous issued U.S. patents, Parag conducted his graduate research in electrical engineering at Virginia Tech.