PPaperPicks

Shiwei Liu

University of Oxford, Mathematical Institute, UK

28 papers at tracked venues · 23 at CORE A* · active 2024–2026

DBLP profile ↗ORCID 0009-0001-1255-4436 ↗Google Scholar ↗Homepage ↗

Venues

ICML ×8ICLR ×6NeurIPS ×6ACL ×2EMNLP ×2InterSpeech ×2AAAI ×1MLSys ×1

Frequent coauthors

Lu Yin
×16
Ajay Kumar Jaiswal
×8
Zhangyang Wang
×7
Zhenyu Zhang
×6
Li Shen
×5
Tianlong Chen
×5
Tianjin Huang
×4
Pengxiang Li
×4
Mykola Pechenizkiy
×4
Runjin Chen
×3
Gen Li
×3
Qiao Xiao
×3

Papers

  1. O1-Pruner: Length-Harmonizing Fine-Tuning for O1-Like Reasoning Pruning
    ACL 2026 ·
    Haotian Luo
    ,
    Haiying He
    ,
    Yibo Wang
    , Shiwei Liu,
    Wei Li
    ,
    Xiaochun Cao
    ,
    Dacheng Tao
    ,
    Naiqiang Tan
    ,
    Li Shen
  2. AlphaDecay: Module-wise Weight Decay for Heavy-Tailed Balancing in LLMs
    NeurIPS 2025 ·
    Di He
    ,
    Songjun Tu
    ,
    Ajay Jaiswal
    ,
    Li Shen
    ,
    Ganzhao Yuan
    , Shiwei Liu,
    Lu Yin
  3. Composable Interventions for Language Models
    ICLR 2025 ·
    Arinbjörn Kolbeinsson
    ,
    Kyle O'Brien
    ,
    Tianjin Huang
    ,
    Shanghua Gao
    , Shiwei Liu,
    Jonathan Richard Schwarz
    ,
    Anurag Jayant Vaidya
    ,
    Faisal Mahmood
    ,
    Marinka Zitnik
    ,
    Tianlong Chen
    ,
    Thomas Hartvigsen
  4. From Low Rank Gradient Subspace Stabilization to Low-Rank Weights: Observations, Theories, and Applications
    ICML 2025 ·
    Ajay Kumar Jaiswal
    ,
    Yifan Wang
    ,
    Lu Yin
    , Shiwei Liu,
    Runjin Chen
    ,
    Jiawei Zhao
    ,
    Ananth Grama
    ,
    Yuandong Tian
    ,
    Zhangyang Wang
  5. GPAS: Accelerating Convergence of LLM Pretraining via Gradient-Preserving Activation Scaling
    NeurIPS 2025 ·
    Tianhao Chen
    ,
    Xin Xu
    ,
    Zijing Liu
    ,
    Pengxiang Li
    ,
    Xinyuan Song
    ,
    Ajay Jaiswal
    ,
    Fan Zhang
    ,
    Jishan Hu
    ,
    Yang Wang
    ,
    Hao Chen
    ,
    Shizhe Diao
    , Shiwei Liu,
    Yu Li
    ,
    Lu Yin
    ,
    Can Yang
  6. LIFT the Veil for the Truth: Principal Weights Emerge after Rank Reduction for Reasoning-Focused Supervised Fine-Tuning
    ICML 2025 ·
    Zihang Liu
    ,
    Tianyu Pang
    ,
    Oleg Balabanov
    ,
    Chaoqun Yang
    ,
    Tianjin Huang
    ,
    Lu Yin
    ,
    Yaoqing Yang
    , Shiwei Liu
  7. Mask-Enhanced Autoregressive Prediction: Pay Less Attention to Learn More
    ICML 2025 ·
    Xialie Zhuang
    ,
    Zhikai Jia
    ,
    Jianjin Li
    ,
    Zhenyu Zhang
    ,
    Li Shen
    ,
    Zheng Cao
    , Shiwei Liu
  8. Mix-LN: Unleashing the Power of Deeper Layers by Combining Pre-LN and Post-LN
    ICLR 2025 ·
    Pengxiang Li
    ,
    Lu Yin
    , Shiwei Liu
  9. Outlier-weighed Layerwise Sampling for LLM Fine-tuning
    ACL 2025 ·
    Pengxiang Li
    ,
    Lu Yin
    ,
    Xiaowei Gao
    , Shiwei Liu
  10. SPAM: Spike-Aware Adam with Momentum Reset for Stable LLM Training
    ICLR 2025 ·
    Tianjin Huang
    ,
    Ziquan Zhu
    ,
    Gaojie Jin
    ,
    Lu Liu
    ,
    Zhangyang Wang
    , Shiwei Liu
  11. The Curse of Depth in Large Language Models
    NeurIPS 2025 ·
    Wenfang Sun
    ,
    Xinyuan Song
    ,
    Pengxiang Li
    ,
    Lu Yin
    ,
    Yefeng Zheng
    , Shiwei Liu
  12. Visual Prompting Upgrades Neural Network Sparsification: A Data-Model Perspective
    AAAI 2025 ·
    Can Jin
    ,
    Tianjin Huang
    ,
    Yihua Zhang
    ,
    Mykola Pechenizkiy
    ,
    Sijia Liu
    , Shiwei Liu,
    Tianlong Chen
  13. AdaMerging: Adaptive Model Merging for Multi-Task Learning
    ICLR 2024 ·
    Enneng Yang
    ,
    Zhenyi Wang
    ,
    Li Shen
    , Shiwei Liu,
    Guibing Guo
    ,
    Xingwei Wang
    ,
    Dacheng Tao
  14. Advancing Dynamic Sparse Training by Exploring Optimization Opportunities
    ICML 2024 ·
    Jie Ji
    ,
    Gen Li
    ,
    Lu Yin
    ,
    Minghai Qin
    ,
    Geng Yuan
    ,
    Linke Guo
    , Shiwei Liu,
    Xiaolong Ma
  15. AlphaPruning: Using Heavy-Tailed Self Regularization Theory for Improved Layer-wise Pruning of Large Language Models
    NeurIPS 2024 ·
    Haiquan Lu
    ,
    Yefan Zhou
    , Shiwei Liu,
    Zhangyang Wang
    ,
    Michael W. Mahoney
    ,
    Yaoqing Yang
  16. CaM: Cache Merging for Memory-efficient LLMs Inference
    ICML 2024 ·
    Yuxin Zhang
    ,
    Yuxuan Du
    ,
    Gen Luo
    ,
    Yunshan Zhong
    ,
    Zhenyu Zhang
    , Shiwei Liu,
    Rongrong Ji
  17. Dynamic Data Pruning for Automatic Speech Recognition
    InterSpeech 2024 ·
    Qiao Xiao
    ,
    Pingchuan Ma
    ,
    Adriana Fernandez-Lopez
    ,
    Boqian Wu
    ,
    Lu Yin
    ,
    Stavros Petridis
    ,
    Mykola Pechenizkiy
    ,
    Maja Pantic
    ,
    Decebal Constantin Mocanu
    , Shiwei Liu
  18. Dynamic Sparse No Training: Training-Free Fine-tuning for Sparse LLMs
    ICLR 2024 ·
    Yuxin Zhang
    ,
    Lirui Zhao
    ,
    Mingbao Lin
    ,
    Yunyun Sun
    ,
    Yiwu Yao
    ,
    Xingjia Han
    ,
    Jared Tanner
    , Shiwei Liu,
    Rongrong Ji
  19. E2ENet: Dynamic Sparse Feature Fusion for Accurate and Efficient 3D Medical Image Segmentation
    NeurIPS 2024 ·
    Boqian Wu
    ,
    Qiao Xiao
    , Shiwei Liu,
    Lu Yin
    ,
    Mykola Pechenizkiy
    ,
    Decebal Constantin Mocanu
    ,
    Maurice van Keulen
    ,
    Elena Mocanu
  20. FFN-SkipLLM: A Hidden Gem for Autoregressive Decoding with Adaptive Feed Forward Skipping
    EMNLP 2024 ·
    Ajay Jaiswal
    ,
    Bodun Hu
    ,
    Lu Yin
    ,
    Yeonju Ro
    ,
    Tianlong Chen
    , Shiwei Liu,
    Aditya Akella
  21. Found in the Middle: How Language Models Use Long Contexts Better via Plug-and-Play Positional Encoding
    NeurIPS 2024 ·
    Zhenyu Zhang
    ,
    Runjin Chen
    , Shiwei Liu,
    Zhewei Yao
    ,
    Olatunji Ruwase
    ,
    Beidi Chen
    ,
    Xiaoxia Wu
    ,
    Zhangyang Wang
  22. Is C4 Dataset Optimal for Pruning? An Investigation of Calibration Data for LLM Pruning
    EMNLP 2024 ·
    Abhinav Bandari
    ,
    Lu Yin
    ,
    Cheng-Yu Hsieh
    ,
    Ajay Jaiswal
    ,
    Tianlong Chen
    ,
    Li Shen
    ,
    Ranjay Krishna
    , Shiwei Liu
  23. Junk DNA Hypothesis: Pruning Small Pre-Trained Weights Irreversibly and Monotonically Impairs "Difficult" Downstream Tasks in LLMs
    ICML 2024 ·
    Lu Yin
    ,
    Ajay Kumar Jaiswal
    , Shiwei Liu,
    Souvik Kundu
    ,
    Zhangyang Wang
  24. MSRS: Training Multimodal Speech Recognition Models from Scratch with Sparse Mask Optimization
    InterSpeech 2024 ·
    Adriana Fernandez-Lopez
    ,
    Honglie Chen
    ,
    Pingchuan Ma
    ,
    Lu Yin
    ,
    Qiao Xiao
    ,
    Stavros Petridis
    , Shiwei Liu,
    Maja Pantic
  25. NeurRev: Train Better Sparse Neural Network Practically via Neuron Revitalization
    ICLR 2024 ·
    Gen Li
    ,
    Lu Yin
    ,
    Jie Ji
    ,
    Wei Niu
    ,
    Minghai Qin
    ,
    Bin Ren
    ,
    Linke Guo
    , Shiwei Liu,
    Xiaolong Ma
  26. Outlier Weighed Layerwise Sparsity (OWL): A Missing Secret Sauce for Pruning LLMs to High Sparsity
    ICML 2024 ·
    Lu Yin
    ,
    You Wu
    ,
    Zhenyu Zhang
    ,
    Cheng-Yu Hsieh
    ,
    Yaqing Wang
    ,
    Yiling Jia
    ,
    Gen Li
    ,
    Ajay Kumar Jaiswal
    ,
    Mykola Pechenizkiy
    ,
    Yi Liang
    ,
    Michael Bendersky
    ,
    Zhangyang Wang
    , Shiwei Liu
  27. Q-Hitter: A Better Token Oracle for Efficient LLM Inference via Sparse-Quantized KV Cache
    MLSys 2024 ·
    Zhenyu Zhang
    , Shiwei Liu,
    Runjin Chen
    ,
    Bhavya Kailkhura
    ,
    Beidi Chen
    ,
    Atlas Wang
  28. Sparse Cocktail: Every Sparse Pattern Every Sparse Ratio All At Once
    ICML 2024 ·
    Zhangheng Li
    , Shiwei Liu,
    Tianlong Chen
    ,
    Ajay Kumar Jaiswal
    ,
    Zhenyu Zhang
    ,
    Dilin Wang
    ,
    Raghuraman Krishnamoorthi
    ,
    Shiyu Chang
    ,
    Zhangyang Wang