Education
Reinforcement Learning
Ph.D. Student
The Hong Kong University of Science and Technology (Guangzhou)
Reinforcement Learning Researcher
Reliable learning
with theoretical guarantees.
I develop simple, principled learning objectives for reinforcement learning and the post-training of large foundation models.
Education & Experience
Pushing the Frontiers of Learning
My path spans foundational reinforcement learning and large foundation model post-training.
Research Internship
Foundation Models
Post-training & On-Policy Distillation
LLM Startup
Research Team
Reinforcement Learning
Foundational Reinforcement Learning Team
Hunyuan Qingyun Project
Research Experience
Research
Research Assistant
The Hong Kong University of Science and Technology (Guangzhou)
Education
Undergraduate
Undergraduate Student
Chongqing Normal University
01 / 2026
Trust Region Policy Distillation
TOP-D
Stable on-policy distillation through a dynamically constructed proximal teacher.
TOP-D converts the unbounded learning signal of standard OPD into a smooth, lower-bounded reward, with formal convergence and monotonic improvement guarantees.
Paper↗
Code—
The Impact of TOP-D
Want to make history? I'm waiting for you.
Start a collaboration↗
02 / ICML 2025
Simple Policy Optimization
SPO
A first-order policy optimization objective that preserves corrective gradients beyond the trust region.
By slightly modifying PPO's policy loss, SPO combines PPO's implementation simplicity with TRPO's strong theoretical guarantees, while retaining stable first-order optimization for deep policy networks.
From objective to embodiment
The Impact of SPO
Two independent robotics projects use SPO-style constraints to keep policy updates corrective and stable.
01Physical Intelligence · 2025
π*0.6: a VLA that Learns from Experience
Physical Intelligence. The technical report implements a DPPO/FPO variant with an alternative PPO constraint following SPO for real-robot reinforcement learning.
Technical report↗02Amazon FAR · UC Berkeley · 2026
Flow Policy Gradients for Robot Control
Brent Yi, Hongsuk Choi, et al. FPO++ uses an asymmetric trust region: PPO for positive advantages and SPO for negative advantages.
Paper↗π*0.6 · Official Real-Robot demoLearning from real-world experience
FPO++ · Official Sim-to-Real demosThree physical humanoid behaviors
Official demo excerpts. Click any video to open its original project page.
Zhengpeng Xie