RL Post-Training Unveiled: How Transformers Learn to Reason and Search
Event Details
- Time: 10:00 a.m. - 11:30 a.m. Eastern Time
- Format: In person and Zoom
- Location: Academic Building One, Room 5150
Abstract
Reinforcement learning (RL) has become a central approach to post-training large language models, driving advances in chain-of-thought reasoning and agentic problem solving. A striking feature of this approach is that rewards based only on final outcomes can improve reasoning across many intermediate steps without explicitly supervising those steps. This raises fundamental questions about how sparse feedback guides learning, produces complex behaviors, and supports generalization.
This talk presents recent work toward a theory of reinforcement learning for reasoning in transformers, revealing how training dynamics give rise to multi-step reasoning and search capabilities. It identifies curriculum structure as a key factor in progressing from simple skills to complex behaviors, including an implicit curriculum that emerges when training on mixed-difficulty problems. The work characterizes how the difficulty spectrum governs steady progress versus grokking-like transitions, with prolonged plateaus followed by abrupt improvements. It also explains how RL shapes specialized attention mechanisms that equip transformer-based policies with exploration, memory, and backtracking, yielding depth-first search strategies that generalize from shallow training trees to deeper ones. These findings connect curriculum structure, optimization dynamics, and attention mechanisms, providing new insight into how RL develops reasoning and agentic capabilities from sparse feedback.
Speaker Bio
Yingbin Liang is a Professor in the Department of Electrical and Computer Engineering at The Ohio State University and a core faculty member of the Translational Data Analytics Institute. She serves as Deputy Director of the NSF AI-EDGE Institute and Co-Lead for the Foundational AI Pillar of the OSU AI (X) Hub. She received her Ph.D. in electrical engineering from the University of Illinois Urbana-Champaign in 2005 and held faculty positions at the University of Hawaii and Syracuse University before joining Ohio State.
Her research lies at the intersection of machine learning, large-scale optimization, statistical signal processing, information theory, and wireless networks, with growing applications in other scientific domains. She received the National Science Foundation CAREER Award and the State of Hawaii Governor Innovation Award in 2009, as well as the EURASIP Best Paper Award in 2014. She is an IEEE Fellow and an IEEE Information Theory Society Distinguished Lecturer for 2026-2027.