👋 About Me

Hi! I am a final-year master’s student at Tsinghua University, under the supervision of Prof. Xiu Li. I received my bachelor’s degree with honors from Shandong University in June 2023.

Previously, I interned at Kuaishou (working with Jiakang Wang and Dr. Fuzheng Zhang), Shanghai AI Laboratory (working with Dr. Biqing Qi and Dr. Chenjia Bai), and Peking University (working with Prof. Yali Du and Prof. Yaodong Yang).

Research Interests: My research centers around Large Language Models (LLMs) and Reinforcement Learning (RL). Specifically, I am interested in:

• Reasoning: Enhancing the reasoning capabilities of LLMs and Multi-modal LLMs (MLLMs).
• Agents: Long-horizon planning agents & LLM agents for real-world workflows.
• LLM4RL: Leveraging the power of LLMs/MLLMs to improve RL algorithms in embodied AI tasks, particularly in the context of reward design and RL from Human/AI Feedback (RLHF/RLAIF).

If you are interested in collaboration, please feel free to reach out via e-mail!

🌟 News

• [2025.09]  🎉 One paper accepted by NeurIPS 2025
• [2025.09]  🔥 Preprint A Survey of Reinforcement Learning for Large Reasoning Models released at arXiv
• [2025.08]  🎉 Two papers accepted by EMNLP 2025
• [2025.05]  🔥 Our multi-agent RL framework for LLM reasoning released (GitHub)!
• [2025.03]  🎉 One paper accepted by Reasoning and Planning for LLMs Workshop @ ICLR 2025
• [2025.01]  🎉 One paper accepted by ICLR 2025
• [2024.12]  🎉 One paper accepted by AAAI 2025 and selected for oral presentation (Top 4.6%)
• [2024.05]  🎉 One paper accepted by ICML 2024
• [2024.01]  🎉 One paper accepted by ICLR 2024
• [2022.09]  🎉 One paper accepted by NeurIPS 2022

📝 Publications

(* denotes equal contribution, † denotes project lead)

Preprints

• Attention as a Compass: Efficient Exploration for Process-Supervised RL in Reasoning Models
  Runze Liu, Jiakang Wang, Yuling Shi, Zhihui Xie, Chenxin An, Kaiyan Zhang, Jian Zhao, Xiaodong Gu, Lei Lin, Wenping Hu, Xiu Li, Fuzheng Zhang, Guorui Zhou, Kun Gai
  [GitHub] [Synced (机器之心)]
  Preprint, 2025

• ASPO: Asymmetric Importance Sampling Policy Optimization
  Jiakang Wang*†, Runze Liu*, Lei Lin, Wenping Hu, Xiu Li, Fuzheng Zhang, Guorui Zhou, Kun Gai
  [GitHub] [QbitAI (量子位)]
  Preprint, 2025

• MARTI: A Framework for Multi-Agent LLM Systems Reinforced Training and Inference
  Kaiyan Zhang*†, Runze Liu*, Xuekai Zhu*, Kai Tian*, Sihang Zeng*, Guoli Jia*, Yuchen Fan*, Xingtai Lv*, Yuxin Zuo*, Che Jiang*, Ziyang Liu, Jianyu Wang, Yuru Wang, Ruotong Zhao, Ermo Hua, Yibo Wang, Shijie Wang, Junqi Gao, Xinwei Long, Youbang Sun, Zhiyuan Ma, Ganqu Cui, Lei Bai, Ning Ding, Biqing Qi, Bowen Zhou
  [GitHub 320+ Stars]
  Preprint, 2025

• A Survey of Reinforcement Learning for Large Reasoning Models
  Kaiyan Zhang*†, Yuxin Zuo*†, Bingxiang He*, Youbang Sun*, Runze Liu* (* denotes core contribution), Che Jiang*, Yuchen Fan*, Kai Tian*, Guoli Jia*, Pengfei Li*, Yu Fu*, Xingtai Lv*, Yuchen Zhang*, Sihang Zeng*, Shang Qu*, Haozhan Li*, Shijie Wang*, Yuru Wang*, Xinwei Long, Fangfu Liu, Xiang Xu, Jiaze Ma, Xuekai Zhu, Ermo Hua, Yihao Liu, Zonglin Li, Huayu Chen, Xiaoye Qu, Yafu Li, Weize Chen, Zhenzhao Yuan, Junqi Gao, Dong Li, Zhiyuan Ma, Ganqu Cui, Zhiyuan Liu, Biqing Qi, Ning Ding, Bowen Zhou
  [GitHub 2k+ Stars] [HuggingFace Daily Papers Top 1] [Synced (机器之心)]
  Preprint, 2025

• Stabilizing Knowledge, Promoting Reasoning: Dual-Token Constraints for RLVR
  Jiakang Wang, Runze Liu, Fuzheng Zhang, Xiu Li, Guorui Zhou, Kun Gai
  [GitHub] [QbitAI (量子位)]
  Preprint, 2025

• Can 1B LLM Surpass 405B LLM? Rethinking Compute-Optimal Test-Time Scaling
  Runze Liu, Junqi Gao, Jian Zhao, Kaiyan Zhang, Xiu Li, Biqing Qi, Wanli Ouyang, Bowen Zhou
  [Project Page] [GitHub 270+ Stars] [HuggingFace Daily Papers Top 1] [QbitAI (量子位)] [AI Era (新智元)]
  Preprint, 2025

• GenPRM: Scaling Test-Time Compute of Process Reward Models via Generative Reasoning
  Jian Zhao*, Runze Liu*†, Kaiyan Zhang, Zhimu Zhou, Junqi Gao, Dong Li, Jiafei Lyu, Zhouyi Qian, Biqing Qi, Xiu Li, Bowen Zhou
  [Project Page] [GitHub] [Awesome Process Reward Models] [Synced (机器之心)]
  Preprint, 2025

Conference Papers

• PEARL: Zero-shot Cross-task Preference Alignment and Robust Reward Learning for Robotic Manipulation
  Runze Liu, Yali Du, Fengshuo Bai, Jiafei Lyu, Xiu Li
  ICML 2024

• Meta-Reward-Net: Implicitly Differentiable Reward Learning for Preference-based Reinforcement Learning
  Runze Liu, Fengshuo Bai, Yali Du, Yaodong Yang
  NeurIPS 2022

• VLP: Vision-Language Preference Learning for Embodied Manipulation
  Runze Liu, Chenjia Bai, Jiafei Lyu, Shengjie Sun, Yali Du, Xiu Li
  EMNLP 2025

• RAT: Adversarial Attacks on Deep Reinforcement Agents for Targeted Behaviors
  Fengshuo Bai, Runze Liu, Yali Du, Ying Wen, Yaodong Yang
  AAAI 2025 Oral (Top 4.6%)

• Bohdi: Heterogeneous LLM Fusion with Automatic Data Exploration
  Junqi Gao, Zhichang Guo, Dazhi Zhang, Dong Li, Runze Liu, Pengfei Li, Kai Tian, Biqing Qi
  NeurIPS 2025

• ReviewRL: Towards Automated Scientific Review with RL
  Sihang Zeng, Kai Tian, Kaiyan Zhang, Yuru Wang, Junqi Gao, Runze Liu, Sa Yang, Jingxuan Li, Xinwei Long, Jiaheng Ma, Biqing Qi, Bowen Zhou
  EMNLP 2025

• Cross-Domain Offline Policy Adaptation with Optimal Transport and Dataset Constraint
  Jiafei Lyu, Mengbei Yan, Zhongjian Qiao, Runze Liu, Xiaoteng Ma, Deheng Ye, Jing-Wen Yang, Zongqing Lu, Xiu Li
  ICLR 2025

• SEABO: A Simple Search-Based Method for Offline Imitation Learning
  Jiafei Lyu, Xiaoteng Ma, Le Wan, Runze Liu, Xiu Li, Zongqing Lu
  ICLR 2024

• Bridging Confidence and Competence: Evaluating Self-Assessment Alignment in LLM Mathematical Reasoning
  Mingze Zhong, Zijing Shi, Ziyan Wang, Runze Liu, Meng Fang, Ling Chen
  PRICAI 2025

Journal Papers

• A Large Language Model-Driven Reward Design Framework via Dynamic Feedback for Reinforcement Learning
  Shengjie Sun*, Runze Liu*, Jiafei Lyu, Jing-Wen Yang, Liangpeng Zhang, Xiu Li
  Knowledge-Based Systems, 2025

Workshop Papers

• Can 1B LLM Surpass 405B LLM? Rethinking Compute-Optimal Test-Time Scaling
  Runze Liu, Junqi Gao, Jian Zhao, Kaiyan Zhang, Xiu Li, Biqing Qi, Wanli Ouyang, Bowen Zhou
  Reasoning and Planning for LLMs @ ICLR 2025

• Zero-shot Cross-task Preference Alignment for Offline RL via Optimal Transport
  Runze Liu, Yali Du, Fengshuo Bai, Jiafei Lyu, Xiu Li
  Optimal Transport and Machine Learning @ NeurIPS 2023

🎓 Education

• Tsinghua University, 2023.09 - 2026.06
  M.Eng. in Electronic and Information Engineering (AI)
• Shandong University, 2019.09 - 2023.06
  B.S. in Statistics (Data Science & AI) with honors

🎖 Honors and Awards

• National Scholarship (Top 1%), 2022.12
• National Scholarship (Top 1%), 2021.12
• First Prize in China Undergraduate Mathematical Contest in Modeling (CUMCM) (Top 0.65%), 2021.11
• Outstanding Student of Shandong Province (Top 0.6%), 2022.05
• Outstanding Graduate of Shandong Province (Top 6%), 2023.04
• Dishang Scholarship, 2022.10

💻 Internships

• Research Intern, Kuaishou (Kwai Star Plan), 2025.06 - 2025.10.
• Research Intern, Large Model Center, Shanghai AI Laboratory, 2024.10 - 2025.03.
• Research Intern, Intelligent Photonics and Electronics Center (IPEC), Shanghai AI Laboratory, 2024.03 - 2024.09.
• Research Intern, Institute for AI, Peking University, 2022.01 - 2022.09.

🎙 Invited Talks

• Scaling Test-Time Compute of LLMs and PRMs for Mathematical Reasoning. ASAP Seminar. 2025.06.
• Can 1B LLM Surpass 405B LLM? Rethinking Compute-Optimal Test-Time Scaling. Huawei Noah’s Ark Lab. 2025.03.
• Can 1B LLM Surpass 405B LLM? Rethinking Compute-Optimal Test-Time Scaling. Xiaohongshu. 2025.02.

🛠️ Services

• Conference Reviewer: NeurIPS (2024 - 2025), ICLR (2025 - 2026), ICML (2025), AAAI (2026), AAMAS (2024), AISTATS (2025), ECAI (2024)
• Journal Reviewer: IEEE Transactions on Artificial Intelligence (TAI)
• Workshop Reviewer: NeurIPS OTML (2023)