👋 About Me

Hi! I am a PhD student at the Hong Kong University of Science and Technology (HKUST) advised by Prof. Ling Pan. I received my master’s degree from Tsinghua University under the supervision of Prof. Xiu Li, and my bachelor’s degree with honors from Shandong University.

Research Interests: My research centers around large foundation models and post-training, with a particular interest in reinforcement learning (RL) and agents. Specifically:

• Post-Training: RL for improving LLMs/MLLMs after pre-training.
• Agents: Self-evolving agents and personal agents that can adapt to individual preferences and feedback.
• Evaluation: Developing reliable evaluation methods for LLMs/MLLMs, particularly for real-world scenarios.

If you are interested in collaboration, please feel free to reach out via e-mail!

🌟 News

• [2026.09]  🎉 Three papers accepted by NeurIPS 2026
• [2026.01]  🎉 Two papers accepted by ICLR 2026
• [2025.11]  🎉 One paper accepted by AAAI 2026
• [2025.09]  🎉 One paper accepted by NeurIPS 2025
• [2025.09]  🔥 Preprint A Survey of Reinforcement Learning for Large Reasoning Models released at arXiv
• [2025.08]  🎉 Two papers accepted by EMNLP 2025
• [2025.03]  🎉 One paper accepted by Reasoning and Planning for LLMs Workshop @ ICLR 2025
• [2025.01]  🎉 One paper accepted by ICLR 2025
• [2024.12]  🎉 One paper accepted by AAAI 2025 and selected for oral presentation (Top 4.6%)
• [2024.05]  🎉 One paper accepted by ICML 2024
• [2024.01]  🎉 One paper accepted by ICLR 2024
• [2022.09]  🎉 One paper accepted by NeurIPS 2022

📝 Selected Papers

(* denotes equal contribution or core contribution, † denotes project lead)

Preprints

• Sleep-Time Training: From Deployment Experience to Persistent Agent Improvement
  Runze Liu, Haoran He, Jiashun Liu, Yuqian Fu, Zijie Geng, Guanting Dong, Wang You, Chen Hu, Daxin Jiang, Ling Pan
  Preprint, 2026

• Rejuvenating Model Plasticity for Robust SFT-to-RL Handoff
  Runze Liu, Jiashun Liu, Xu Wan, Yuqian Fu, Chen Hu, Daxin Jiang, Ling Pan
  Preprint, 2026

• When Importance Sampling Misallocates Credit: Asymmetric Ratios for Outcome-Supervised RL
  Runze Liu*, Jiakang Wang*, Qingpeng Cai, Lei Lin, Wenping Hu, Xiu Li, Fuzheng Zhang, Guorui Zhou, Kun Gai, Ling Pan
  [GitHub] [QbitAI (量子位)]
  Preprint, 2025

• Can 1B LLM Surpass 405B LLM? Rethinking Compute-Optimal Test-Time Scaling
  Runze Liu, Junqi Gao, Jian Zhao, Kaiyan Zhang, Xiu Li, Biqing Qi, Wanli Ouyang, Bowen Zhou
  [Project Page] [GitHub 280+ Stars] [HuggingFace Daily Papers Top 1] [QbitAI (量子位)] [AI Era (新智元)]
  Preprint, 2025

• A Survey of Reinforcement Learning for Large Reasoning Models
  Kaiyan Zhang*†, Yuxin Zuo*†, Bingxiang He*, Youbang Sun*, Runze Liu*, Che Jiang*, Yuchen Fan*, Kai Tian*, Guoli Jia*, Pengfei Li*, Yu Fu*, Xingtai Lv*, Yuchen Zhang*, Sihang Zeng*, Shang Qu*, Haozhan Li*, Shijie Wang*, Yuru Wang*, Xinwei Long, Fangfu Liu, Xiang Xu, Jiaze Ma, Xuekai Zhu, Ermo Hua, Yihao Liu, Zonglin Li, Huayu Chen, Xiaoye Qu, Yafu Li, Weize Chen, Zhenzhao Yuan, Junqi Gao, Dong Li, Zhiyuan Ma, Ganqu Cui, Zhiyuan Liu, Biqing Qi, Ning Ding, Bowen Zhou
  [GitHub 2.5k+ Stars] [HuggingFace Daily Papers Top 1] [Synced (机器之心)]
  Preprint, 2025

Publications

• Attention as a Compass: Efficient Exploration for Process-Supervised RL in Reasoning Models
  Runze Liu, Jiakang Wang, Yuling Shi, Zhihui Xie, Chenxin An, Kaiyan Zhang, Jian Zhao, Xiaodong Gu, Lei Lin, Wenping Hu, Xiu Li, Fuzheng Zhang, Guorui Zhou, Kun Gai
  [GitHub] [Synced (机器之心)]
  ICLR 2026

• Stabilizing Knowledge, Promoting Reasoning: Dual-Token Constraints for RLVR
  Runze Liu*, Jiakang Wang*, Fuzheng Zhang, Xiu Li, Guorui Zhou, Kun Gai, Ling Pan
  [GitHub] [QbitAI (量子位)]
  NeurIPS 2026

• Reformulate LLM Reinforcement Learning for Efficient Training under Black-box Discrepancy
  Jiashun Liu*, Runze Liu*, Xu Wan*, Jing Liang*, Hongyao Tang, Ling Pan
  NeurIPS 2026

• GenPRM: Scaling Test-Time Compute of Process Reward Models via Generative Reasoning
  Jian Zhao*, Runze Liu*†, Kaiyan Zhang, Zhimu Zhou, Junqi Gao, Dong Li, Jiafei Lyu, Zhouyi Qian, Biqing Qi, Xiu Li, Bowen Zhou
  [Project Page] [GitHub] [Awesome Process Reward Models] [Synced (机器之心)]
  AAAI 2026

• MARTI: A Framework for Multi-Agent LLM Systems Reinforced Training and Inference
  Kaiyan Zhang*†, Kai Tian*, Runze Liu*, Sihang Zeng, Xuekai Zhu, Guoli Jia, Yuchen Fan, Xingtai Lv, Yuxin Zuo, Che Jiang, Yuru Wang, Jianyu Wang, Ermo Hua, Xinwei Long, Junqi Gao, Youbang Sun, Zhiyuan Ma, Ganqu Cui, Ning Ding, Biqing Qi, Bowen Zhou [GitHub 560+ Stars]
  ICLR 2026

• PEARL: Zero-shot Cross-task Preference Alignment and Robust Reward Learning for Robotic Manipulation
  Runze Liu, Yali Du, Fengshuo Bai, Jiafei Lyu, Xiu Li
  ICML 2024

• Meta-Reward-Net: Implicitly Differentiable Reward Learning for Preference-based Reinforcement Learning
  Runze Liu, Fengshuo Bai, Yali Du, Yaodong Yang
  NeurIPS 2022

• VLP: Vision-Language Preference Learning for Embodied Manipulation
  Runze Liu, Chenjia Bai, Jiafei Lyu, Shengjie Sun, Yali Du, Xiu Li
  EMNLP 2025

• A Large Language Model-Driven Reward Design Framework via Dynamic Feedback for Reinforcement Learning
  Shengjie Sun*, Runze Liu*, Jiafei Lyu, Jing-Wen Yang, Liangpeng Zhang, Xiu Li
  Knowledge-Based Systems, 2025

• RAT: Adversarial Attacks on Deep Reinforcement Agents for Targeted Behaviors
  Fengshuo Bai, Runze Liu, Yali Du, Ying Wen, Yaodong Yang
  AAAI 2025 Oral (Top 4.6%)

🎓 Education

• The Hong Kong University of Science and Technology, 2026.09 -
  Ph.D. in Electronic and Computer Engineering
• Tsinghua University, 2023.09 - 2026.06
  M.Eng. in Electronic and Information Engineering (AI)
• Shandong University, 2019.09 - 2023.06
  B.S. in Statistics (Data Science & AI) with honors

🎖 Honors and Awards

• HKUST RedBird PhD Recruitment Award, 2026.01
• National Scholarship (Top 0.2%), 2022.12
• National Scholarship (Top 0.2%), 2021.12
• First Prize in China Undergraduate Mathematical Contest in Modeling (CUMCM) (Top 0.65%), 2021.11
• Outstanding Student of Shandong Province (Top 0.6%), 2022.05

💻 Internships

• Research Intern, ByteDance, 2026.09 - present.
• Research Intern, StepFun, 2025.12 - 2026.09.
• Research Intern, Kuaishou (Kwai Star Plan), 2025.06 - 2025.10.
• Research Intern, Large Model Center, Shanghai AI Laboratory, 2024.10 - 2025.03.
• Research Intern, Intelligent Photonics and Electronics Center (IPEC), Shanghai AI Laboratory, 2024.03 - 2024.09.
• Research Intern, Institute for AI, Peking University, 2022.01 - 2022.09.

🎙 Invited Talks

• Scaling Test-Time Compute of LLMs and PRMs for Mathematical Reasoning. ASAP Seminar. 2025.06.
• Can 1B LLM Surpass 405B LLM? Rethinking Compute-Optimal Test-Time Scaling. Huawei Noah’s Ark Lab. 2025.03.
• Can 1B LLM Surpass 405B LLM? Rethinking Compute-Optimal Test-Time Scaling. Xiaohongshu. 2025.02.

🛠️ Services

• Student Member, MLNLP (2026 - present)
• Conference Reviewer: NeurIPS (2024 - 2026), ICLR (2025 - 2027), ICML (2025 - 2026), COLM (2026), AAAI (2026 - 2027), AAMAS (2024), AISTATS (2025), ECAI (2024)
• Journal Reviewer: IEEE Transactions on Robotics (TRO), IEEE Transactions on Artificial Intelligence (TAI), ACML