PPaperPicks

Arman Cohan

82 papers at tracked venues · 42 at CORE A* · active 2024–2026

DBLP profile ↗ORCID search ↗

Venues

ACL ×31EMNLP ×25NAACL ×11NeurIPS ×4EACL ×2ECIR ×2ICLR ×2ICML ×2AAAI ×1CVPR ×1SIGIR ×1

Frequent coauthors

Yilun Zhao
×50
Yixin Liu
×17
Chen Zhao
×15
Manasi Patwardhan
×9
Xiangru Tang
×9
Tiansheng Hu
×6
Simeng Han
×6
Chengye Wang
×6
Ziyao Shangguan
×5
Mark Gerstein
×5
Yitao Long
×5
Luca Soldaini
×5

Papers

  1. A Survey of Multimodal Mathematical Reasoning: From Perception, Alignment to Reasoning
    ACL 2026 ·
    Tianyu Yang
    ,
    Sihong Wu
    ,
    Yilun Zhao
    ,
    Zhenwen Liang
    ,
    Lisen Dai
    ,
    Chen Zhao
    ,
    Minhao Cheng
    , Arman Cohan,
    Xiangliang Zhang
  2. A Survey on Evaluation of LLM-based Agents
    ACL 2026 ·
    Asaf Yehudai
    ,
    Lilach Eden
    ,
    Alan Li
    ,
    Guy Uziel
    ,
    Yilun Zhao
    ,
    Roy Bar-Haim
    , Arman Cohan,
    Michal Shmueli-Scheuer
  3. Anchor: Branch-Point Data Generation for GUI Agents
    ACL 2026 ·
    Jinbiao Wei
    ,
    Yilun Zhao
    ,
    Kangqi Ni
    , Arman Cohan
  4. CPTCoder: A Reliable LLM System for Medical Procedure Code Prediction
    ACL 2026 ·
    Benlu Wang
    ,
    Ziyao Shangguan
    ,
    Kyle Tegtmeyer
    ,
    Zhenyu Zhang
    ,
    Sophie Chheang
    , Arman Cohan
  5. Can AI Be a Good Peer Reviewer? A Survey of Peer Review Process, Evaluation, and the Future
    ACL 2026 ·
    Sihong Wu
    ,
    Owen Jiang
    ,
    Yilun Zhao
    ,
    Tiansheng Hu
    ,
    Yiling Ma
    ,
    Kaiyan Zhang
    ,
    Manasi Patwardhan
    , Arman Cohan
  6. Evaluating Legal Reasoning Traces with Legal Issue Tree Rubrics
    ACL 2026 ·
    Jinu Lee
    ,
    Kyoung-Woon On
    ,
    Sophia Simeng Han
    , Arman Cohan,
    Julia Hockenmaier
  7. Investigating Retrieval-Augmented Generation Systems on Unanswerable, Uncheatable, Realistic, Multi-hop Queries
    ECIR 2026 ·
    Gabrielle Kaili-May Liu
    ,
    Bryan Li
    , Arman Cohan,
    William Gantt Walden
    ,
    Eugene Yang
  8. MMSciCode: Real-world Evaluation of Multilingual Multi-Discipline Scientific Research Coding
    ACL 2026 ·
    Xue Xia
    ,
    Zheyuan Yang
    , Arman Cohan,
    Yilun Zhao
  9. MultiFinBen: Benchmarking Large Language Models for Multilingual and Multimodal Financial Application
    ACL 2026 ·
    Xueqing Peng
    ,
    Lingfei Qian
    ,
    Yan Wang
    ,
    Ruoyu Xiang
    ,
    Yueru He
    ,
    Yang Ren
    ,
    Mingyang Jiang
    ,
    Vincent Jim Zhang
    ,
    Yuqing Guo
    ,
    Jeff Zhao
    ,
    Huan He
    ,
    Yi Han
    ,
    Yun Feng
    ,
    Yuechen Jiang
    ,
    Yupeng Cao
    ,
    Haohang Li
    ,
    Yangyang Yu
    ,
    Xiaoyu Wang
    ,
    Penglei Gao
    ,
    Shengyuan Lin
    ,
    Keyi Wang
    ,
    Shanshan Yang
    ,
    Yilun Zhao
    ,
    Zhiwei Liu
    ,
    Peng Lu
    ,
    Jerry Huang
    ,
    Suyuchen Wang
    ,
    Triantafillos Papadopoulos
    ,
    Polydoros Giannouris
    ,
    Efstathia Soufleri
    ,
    Nuo Chen
    ,
    Zhiyang Deng
    ,
    Heming Fu
    ,
    Yijia Zhao
    ,
    Mingquan Lin
    ,
    Meikang Qiu
    ,
    Kaleb E. Smith
    , Arman Cohan,
    Xiao-Yang Liu
    ,
    Jimin Huang
    ,
    Guojun Xiong
    ,
    Alejandro Lopez-Lira
    ,
    Xi Chen
    ,
    Junichi Tsujii
    ,
    Jian-Yun Nie
    ,
    Sophia Ananiadou
    ,
    Qianqian Xie
  10. Patient-Similarity Cohort Reasoning in Clinical Text-to-SQL
    EACL 2026 ·
    Yifei Shen
    ,
    Yilun Zhao
    ,
    Justice Ou
    ,
    Tinglin Huang
    , Arman Cohan
  11. RbtAct: Rebuttal as Supervision for Actionable Review Feedback Generation
    ACL 2026 ·
    Sihong Wu
    ,
    Yiling Ma
    ,
    Yilun Zhao
    ,
    Tiansheng Hu
    ,
    Owen Jiang
    ,
    Manasi Patwardhan
    , Arman Cohan
  12. Rethinking Reasoning-Intensive Retrieval: Evaluating and Advancing Retrievers in Agentic Search Systems
    ACL 2026 ·
    Yilun Zhao
    ,
    Jinbiao Wei
    ,
    Tingyu Song
    ,
    Siyue Zhang
    ,
    Chen Zhao
    , Arman Cohan
  13. SciMDR: Advancing Scientific Multimodal Document Reasoning
    ACL 2026 ·
    Ziyu Chen
    ,
    Yilun Zhao
    ,
    Chengye Wang
    ,
    Rilyn Han
    ,
    Manasi Patwardhan
    , Arman Cohan
  14. SciRAG: Adaptive, Citation-Aware, and Outline-Guided Retrieval and Synthesis for Scientific Literature
    EACL 2026 ·
    Hang Ding
    ,
    Yilun Zhao
    ,
    Tiansheng Hu
    ,
    Manasi Patwardhan
    , Arman Cohan
  15. A Large-Scale Study of Reranker Relevance Feedback at Inference
    SIGIR 2025 ·
    Revanth Gangi Reddy
    ,
    Pradeep Dasigi
    ,
    Md. Arafat Sultan
    , Arman Cohan,
    Avirup Sil
    ,
    Heng Ji
    ,
    Hannaneh Hajishirzi
  16. AbGen: Evaluating Large Language Models in Ablation Study Design and Evaluation for Scientific Research
    ACL 2025 ·
    Yilun Zhao
    ,
    Weiyuan Chen
    ,
    Zhijian Xu
    ,
    Manasi Patwardhan
    ,
    Chengye Wang
    ,
    Yixin Liu
    ,
    Lovekesh Vig
    , Arman Cohan
  17. Are Multimodal LLMs Robust Against Adversarial Perturbations? RoMMath: A Systematic Evaluation on Multimodal Math Reasoning
    NAACL 2025 ·
    Yilun Zhao
    ,
    Guo Gan
    ,
    Chen Zhao
    , Arman Cohan
  18. Can LLMs Identify Critical Limitations within Scientific Research? A Systematic Evaluation on AI Research Papers
    ACL 2025 ·
    Zhijian Xu
    ,
    Yilun Zhao
    ,
    Manasi Patwardhan
    ,
    Lovekesh Vig
    , Arman Cohan
  19. Can Multimodal Foundation Models Understand Schematic Diagrams? An Empirical Study on Information-Seeking QA over Scientific Papers
    ACL 2025 ·
    Yilun Zhao
    ,
    Chengye Wang
    ,
    Chuhan Li
    , Arman Cohan
  20. ChemAgent: Self-updating Memories in Large Language Models Improves Chemical Reasoning
    ICLR 2025 ·
    Xiangru Tang
    ,
    Tianyu Hu
    ,
    Muyang Ye
    ,
    Yanjun Shao
    ,
    Xunjian Yin
    ,
    Siru Ouyang
    ,
    Wangchunshu Zhou
    ,
    Pan Lu
    ,
    Zhuosheng Zhang
    ,
    Yilun Zhao
    , Arman Cohan,
    Mark Gerstein
  21. CourtReasoner: Can LLM Agents Reason Like Judges?
    EMNLP 2025 ·
    Sophia Simeng Han
    ,
    Yoshiki Takashima
    ,
    Shannon Zejiang Shen
    ,
    Chen Liu
    ,
    Yixin Liu
    ,
    Roque K. Thuo
    ,
    Sonia Knowlton
    ,
    Ruzica Piskac
    ,
    Scott J. Shapiro
    , Arman Cohan
  22. Diffusion vs. Autoregressive Language Models: A Text Embedding Perspective
    EMNLP 2025 ·
    Siyue Zhang
    ,
    Yilun Zhao
    ,
    Liyuan Geng
    , Arman Cohan,
    Anh Tuan Luu
    ,
    Chen Zhao
  23. DyFlow: Dynamic Workflow Framework for Agentic Reasoning
    NeurIPS 2025 ·
    Yanbo Wang
    ,
    Zixiang Xu
    ,
    Yue Huang
    ,
    Xiangqi Wang
    ,
    Zirui Song
    ,
    Lang Gao
    ,
    Chenxi Wang
    ,
    Robert Tang
    ,
    Yue Zhao
    , Arman Cohan,
    Xiangliang Zhang
    ,
    Xiuying Chen
  24. FinLFQA: Evaluating Attributed Text Generation of LLMs in Financial Long-Form Question Answering
    EMNLP 2025 ·
    Yitao Long
    ,
    Tiansheng Hu
    ,
    Yilun Zhao
    , Arman Cohan,
    Chen Zhao
  25. FinTrust: A Comprehensive Benchmark of Trustworthiness Evaluation in Finance Domain
    EMNLP 2025 ·
    Tiansheng Hu
    ,
    Tongyan Hu
    ,
    Liuyang Bai
    ,
    Yilun Zhao
    , Arman Cohan,
    Chen Zhao
  26. FollowIR: Evaluating and Teaching Information Retrieval Models to Follow Instructions
    NAACL 2025 ·
    Orion Weller
    ,
    Benjamin Chang
    ,
    Sean MacAvaney
    ,
    Kyle Lo
    , Arman Cohan,
    Benjamin Van Durme
    ,
    Dawn J. Lawrie
    ,
    Luca Soldaini
  27. From Scores to Steps: Diagnosing and Improving LLM Performance in Evidence-Based Medical Calculations
    EMNLP 2025 ·
    Benlu Wang
    ,
    Iris Xia
    ,
    Yifan Zhang
    ,
    Junda Wang
    ,
    Feiyun Ouyang
    ,
    Shuo Han
    , Arman Cohan,
    Hong Yu
    ,
    Zonghai Yao
  28. HumanEval Pro and MBPP Pro: Evaluating Large Language Models on Self-invoking Code Generation Task
    ACL 2025 ·
    Zhaojian Yu
    ,
    Yilun Zhao
    , Arman Cohan,
    Xiaoping Zhang
  29. IRIS: Interactive Research Ideation System for Accelerating Scientific Discovery
    ACL 2025 ·
    Aniketh Garikaparthi
    ,
    Manasi Patwardhan
    ,
    Lovekesh Vig
    , Arman Cohan
  30. Judging with Many Minds: Do More Perspectives Mean Less Prejudice? On Bias Amplification and Resistance in Multi-Agent Based LLM-as-Judge
    EMNLP 2025 ·
    Chiyu Ma
    ,
    Enpei Zhang
    ,
    Yilun Zhao
    ,
    Wenjun Liu
    ,
    Yaning Jia
    ,
    Peijun Qing
    ,
    Lin Shi
    , Arman Cohan,
    Yujun Yan
    ,
    Soroush Vosoughi
  31. LimRank: Less is More for Reasoning-Intensive Information Reranking
    EMNLP 2025 ·
    Tingyu Song
    ,
    Yilun Zhao
    ,
    Siyue Zhang
    ,
    Chen Zhao
    , Arman Cohan
  32. LocAgent: Graph-Guided LLM Agents for Code Localization
    ACL 2025 ·
    Zhaoling Chen
    ,
    Robert Tang
    ,
    Gangda Deng
    ,
    Fang Wu
    ,
    Jialong Wu
    ,
    Zhiwei Jiang
    ,
    Viktor K. Prasanna
    , Arman Cohan,
    Xingyao Wang
  33. MCTS-RAG: Enhancing Retrieval-Augmented Generation with Monte Carlo Tree Search
    EMNLP 2025 ·
    Yunhai Hu
    ,
    Yilun Zhao
    ,
    Chen Zhao
    , Arman Cohan
  34. MDCure: A Scalable Pipeline for Multi-Document Instruction-Following
    ACL 2025 ·
    Gabrielle Kaili-May Liu
    ,
    Bowen Shi
    ,
    Avi Caciularu
    ,
    Idan Szpektor
    , Arman Cohan
  35. MIR: Methodology Inspiration Retrieval for Scientific Research Problems
    ACL 2025 ·
    Aniketh Garikaparthi
    ,
    Manasi Patwardhan
    ,
    Aditya Sanjiv Kanade
    ,
    Aman Hassan
    ,
    Lovekesh Vig
    , Arman Cohan
  36. MMVU: Measuring Expert-Level Multi-Discipline Video Understanding
    CVPR 2025 ·
    Yilun Zhao
    ,
    Haowei Zhang
    ,
    Lujing Xie
    ,
    Tongyan Hu
    ,
    Guo Gan
    ,
    Yitao Long
    ,
    Zhiyuan Hu
    ,
    Weiyuan Chen
    ,
    Chuhan Li
    ,
    Zhijian Xu
    ,
    Chengye Wang
    ,
    Ziyao Shangguan
    ,
    Zhenwen Liang
    ,
    Yixin Liu
    ,
    Chen Zhao
    , Arman Cohan
  37. Measuring what Matters: Construct Validity in Large Language Model Benchmarks
    NeurIPS 2025 ·
    Andrew M. Bean
    ,
    Ryan Othniel Kearns
    ,
    Angelika Romanou
    ,
    Franziska Sofia Hafner
    ,
    Harry Mayne
    ,
    Jan Batzner
    ,
    Negar Foroutan Eghlidi
    ,
    Chris Schmitz
    ,
    Karolina Korgul
    ,
    Hunar Batra
    ,
    Oishi Deb
    ,
    Emma Beharry
    ,
    Cornelius Emde
    ,
    Thomas Foster
    ,
    Anna Gausen
    ,
    María Grandury
    ,
    Sophia Simeng Han
    ,
    Valentin Hofmann
    ,
    Lujain Ibrahim
    ,
    Hazel Kim
    ,
    Hannah Rose Kirk
    ,
    Fangru Lin
    ,
    Gabrielle K. Liu
    ,
    Lennart Luettgau
    ,
    Jabez Magomere
    ,
    Jonathan Rystrøm
    ,
    Anna Sotnikova
    ,
    Yushi Yang
    ,
    Yilun Zhao
    ,
    Adel Bibi
    ,
    Antoine Bosselut
    ,
    Ronald Clark
    , Arman Cohan,
    Jakob N. Foerster
    ,
    Yarin Gal
    ,
    Scott A. Hale
    ,
    Inioluwa Deborah Raji
    ,
    Christopher Summerfield
    ,
    Philip H. S. Torr
    ,
    Cozmin Ududec
    ,
    Luc Rocher
    ,
    Adam Mahdi
  38. MedTutor: A Retrieval-Augmented LLM System for Case-Based Medical Education
    EMNLP 2025 ·
    Dongsuk Jang
    ,
    Ziyao Shangguan
    ,
    Kyle Tegtmeyer
    ,
    Anurag Gupta
    ,
    Jan Czerminski
    ,
    Sophie Chheang
    , Arman Cohan
  39. MetaFaith: Faithful Natural Language Uncertainty Expression in LLMs
    EMNLP 2025 ·
    Gabrielle Kaili-May Liu
    ,
    Gal Yona
    ,
    Avi Caciularu
    ,
    Idan Szpektor
    ,
    Tim G. J. Rudner
    , Arman Cohan
  40. On Evaluating LLM Alignment by Evaluating LLMs as Judges
    NeurIPS 2025 ·
    Yixin Liu
    ,
    Pengfei Liu
    , Arman Cohan
  41. Physics: Benchmarking Foundation Models on University-Level Physics Problem Solving
    ACL 2025 ·
    Kaiyue Feng
    ,
    Yilun Zhao
    ,
    Yixin Liu
    ,
    Tianyu Yang
    ,
    Chen Zhao
    ,
    John Sous
    , Arman Cohan
  42. Re-evaluating Automatic LLM System Ranking for Alignment with Human Preference
    NAACL 2025 ·
    Mingqi Gao
    ,
    Yixin Liu
    ,
    Xinyu Hu
    ,
    Xiaojun Wan
    ,
    Jonathan Bragg
    , Arman Cohan
  43. ReIFE: Re-evaluating Instruction-Following Evaluation
    NAACL 2025 ·
    Yixin Liu
    ,
    Kejian Shi
    ,
    Alexander R. Fabbri
    ,
    Yilun Zhao
    ,
    Peifeng Wang
    ,
    Chien-Sheng Wu
    ,
    Shafiq Joty
    , Arman Cohan
  44. Ref-Long: Benchmarking the Long-context Referencing Capability of Long-context Language Models
    ACL 2025 ·
    Junjie Wu
    ,
    Gefei Gu
    ,
    Yanan Zheng
    ,
    Dit-Yan Yeung
    , Arman Cohan
  45. RouterRetriever: Routing over a Mixture of Expert Embedding Models
    AAAI 2025 ·
    Hyunji Lee
    ,
    Luca Soldaini
    , Arman Cohan,
    Minjoon Seo
    ,
    Kyle Lo
  46. SCIURus: Shared Circuits for Interpretable Uncertainty Representations in Language Models
    NAACL 2025 ·
    Carter Teplica
    ,
    Yixin Liu
    , Arman Cohan,
    Tim G. J. Rudner
  47. SciArena: An Open Evaluation Platform for Non-Verifiable Scientific Literature-Grounded Tasks
    NeurIPS 2025 ·
    Yilun Zhao
    ,
    Kaiyan Zhang
    ,
    Tiansheng Hu
    ,
    Sihong Wu
    ,
    Ronan Le Bras
    ,
    Yixin Liu
    ,
    Robert Tang
    ,
    Joseph Chee Chang
    ,
    Jesse Dodge
    ,
    Jonathan Bragg
    ,
    Chen Zhao
    ,
    Hanna Hajishirzi
    ,
    Doug Downey
    , Arman Cohan
  48. SciRIFF: A Resource to Enhance Language Model Instruction-Following over Scientific Literature
    EMNLP 2025 ·
    David Wadden
    ,
    Kejian Shi
    ,
    Jacob Morrison
    ,
    Alan Li
    ,
    Aakanksha Naik
    ,
    Shruti Singh
    ,
    Nitzan Barzilay
    ,
    Kyle Lo
    ,
    Tom Hope
    ,
    Luca Soldaini
    ,
    Shannon Zejiang Shen
    ,
    Doug Downey
    ,
    Hannaneh Hajishirzi
    , Arman Cohan
  49. SciSketch: An Open-source Framework for Automated Schematic Diagram Generation in Scientific Papers
    EMNLP 2025 ·
    Zihang Wang
    ,
    Yilun Zhao
    ,
    Kaiyan Zhang
    ,
    Chen Zhao
    ,
    Manasi Patwardhan
    , Arman Cohan
  50. SciVer: Evaluating Foundation Models for Multimodal Scientific Claim Verification
    ACL 2025 ·
    Chengye Wang
    ,
    Yifei Shen
    ,
    Zexi Kuang
    , Arman Cohan,
    Yilun Zhao
  51. TESS 2: A Large-Scale Generalist Diffusion Language Model
    ACL 2025 ·
    Jaesung Tae
    ,
    Hamish Ivison
    ,
    Sachin Kumar
    , Arman Cohan
  52. TOMATO: Assessing Visual Temporal Reasoning Capabilities in Multimodal Foundation Models
    ICLR 2025 ·
    Ziyao Shangguan
    ,
    Chuhan Li
    ,
    Yuxuan Ding
    ,
    Yanan Zheng
    ,
    Yilun Zhao
    ,
    Tesca Fitzgerald
    , Arman Cohan
  53. Table-R1: Inference-Time Scaling for Table Reasoning Tasks
    EMNLP 2025 ·
    Zheyuan Yang
    ,
    Lyuhao Chen
    , Arman Cohan,
    Yilun Zhao
  54. Understanding Reference Policies in Direct Preference Optimization
    NAACL 2025 ·
    Yixin Liu
    ,
    Pengfei Liu
    , Arman Cohan
  55. Z1: Efficient Test-time Scaling with Code
    EMNLP 2025 ·
    Zhaojian Yu
    ,
    Yinghao Wu
    ,
    Yilun Zhao
    , Arman Cohan,
    Xiao-Ping Zhang
  56. mFollowIR: A Multilingual Benchmark for Instruction Following in Retrieval
    ECIR 2025 ·
    Orion Weller
    ,
    Benjamin Chang
    ,
    Eugene Yang
    ,
    Mahsa Yarmohammadi
    ,
    Samuel Barham
    ,
    Sean MacAvaney
    , Arman Cohan,
    Luca Soldaini
    ,
    Benjamin Van Durme
    ,
    Dawn J. Lawrie
  57. Bayesian Calibration of Win Rate Estimation with LLM Evaluators
    EMNLP 2024 ·
    Yicheng Gao
    ,
    Gonghan Xu
    ,
    Zhe Wang
    , Arman Cohan
  58. Benchmarking Generation and Evaluation Capabilities of Large Language Models for Instruction Controllable Summarization
    NAACL 2024 ·
    Yixin Liu
    ,
    Alexander R. Fabbri
    ,
    Jiawen Chen
    ,
    Yilun Zhao
    ,
    Simeng Han
    ,
    Shafiq Joty
    ,
    Pengfei Liu
    ,
    Dragomir Radev
    ,
    Chien-Sheng Wu
    , Arman Cohan
  59. Calibrating Long-form Generations From Large Language Models
    EMNLP 2024 ·
    Yukun Huang
    ,
    Yixin Liu
    ,
    Raghuveer Thirukovalluru
    , Arman Cohan,
    Bhuwan Dhingra
  60. DocMath-Eval: Evaluating Math Reasoning Capabilities of LLMs in Understanding Financial Documents
    ACL 2024 ·
    Yilun Zhao
    ,
    Yitao Long
    ,
    Hongjun Liu
    ,
    Ryo Kamoi
    ,
    Linyong Nan
    ,
    Lyuhao Chen
    ,
    Yixin Liu
    ,
    Xiangru Tang
    ,
    Rui Zhang
    , Arman Cohan
  61. FOLIO: Natural Language Reasoning with First-Order Logic
    EMNLP 2024 ·
    Simeng Han
    ,
    Hailey Schoelkopf
    ,
    Yilun Zhao
    ,
    Zhenting Qi
    ,
    Martin Riddell
    ,
    Wenfei Zhou
    ,
    James Coady
    ,
    David Peng
    ,
    Yujie Qiao
    ,
    Luke Benson
    ,
    Lucy Sun
    ,
    Alexander Wardle-Solano
    ,
    Hannah Szabó
    ,
    Ekaterina Zubova
    ,
    Matthew Burtell
    ,
    Jonathan Fan
    ,
    Yixin Liu
    ,
    Brian Wong
    ,
    Malcolm Sailor
    ,
    Ansong Ni
    ,
    Linyong Nan
    ,
    Jungo Kasai
    ,
    Tao Yu
    ,
    Rui Zhang
    ,
    Alexander R. Fabbri
    ,
    Wojciech Kryscinski
    ,
    Semih Yavuz
    ,
    Ye Liu
    ,
    Xi Victoria Lin
    ,
    Shafiq Joty
    ,
    Yingbo Zhou
    ,
    Caiming Xiong
    ,
    Rex Ying
    , Arman Cohan,
    Dragomir Radev
  62. FinDVer: Explainable Claim Verification over Long and Hybrid-content Financial Documents
    EMNLP 2024 ·
    Yilun Zhao
    ,
    Yitao Long
    ,
    Tintin Jiang
    ,
    Chengye Wang
    ,
    Weiyuan Chen
    ,
    Hongjun Liu
    ,
    Xiangru Tang
    ,
    Yiming Zhang
    ,
    Chen Zhao
    , Arman Cohan
  63. Investigating Data Contamination in Modern Benchmarks for Large Language Models
    NAACL 2024 ·
    Chunyuan Deng
    ,
    Yilun Zhao
    ,
    Xiangru Tang
    ,
    Mark Gerstein
    , Arman Cohan
  64. KnowledgeFMath: A Knowledge-Intensive Math Reasoning Dataset in Finance Domains
    ACL 2024 ·
    Yilun Zhao
    ,
    Hongjun Liu
    ,
    Yitao Long
    ,
    Rui Zhang
    ,
    Chen Zhao
    , Arman Cohan
  65. M3SciQA: A Multi-Modal Multi-Document Scientific QA Benchmark for Evaluating Foundation Models
    EMNLP 2024 ·
    Chuhan Li
    ,
    Ziyao Shangguan
    ,
    Yilun Zhao
    ,
    Deyuan Li
    ,
    Yixin Liu
    , Arman Cohan
  66. MIMIR: A Customizable Agent Tuning Platform for Enhanced Scientific Applications
    EMNLP 2024 ·
    Xiangru Tang
    ,
    Chunyuan Deng
    ,
    Hanming Wang
    ,
    Haoran Wang
    ,
    Yilun Zhao
    ,
    Wenqi Shi
    ,
    May Fung
    ,
    Wangchunshu Zhou
    ,
    Jiannan Cao
    ,
    Heng Ji
    , Arman Cohan,
    Mark Gerstein
  67. MedAgents: Large Language Models as Collaborators for Zero-shot Medical Reasoning
    ACL 2024 ·
    Xiangru Tang
    ,
    Anni Zou
    ,
    Zhuosheng Zhang
    ,
    Ziming Li
    ,
    Yilun Zhao
    ,
    Xingyao Zhang
    , Arman Cohan,
    Mark Gerstein
  68. NExT: Teaching Large Language Models to Reason about Code Execution
    ICML 2024 ·
    Ansong Ni
    ,
    Miltiadis Allamanis
    , Arman Cohan,
    Yinlin Deng
    ,
    Kensen Shi
    ,
    Charles Sutton
    ,
    Pengcheng Yin
  69. OLMo: Accelerating the Science of Language Models
    ACL 2024 ·
    Dirk Groeneveld
    ,
    Iz Beltagy
    ,
    Evan Pete Walsh
    ,
    Akshita Bhagia
    ,
    Rodney Kinney
    ,
    Oyvind Tafjord
    ,
    Ananya Harsh Jha
    ,
    Hamish Ivison
    ,
    Ian Magnusson
    ,
    Yizhong Wang
    ,
    Shane Arora
    ,
    David Atkinson
    ,
    Russell Authur
    ,
    Khyathi Raghavi Chandu
    , Arman Cohan,
    Jennifer Dumas
    ,
    Yanai Elazar
    ,
    Yuling Gu
    ,
    Jack Hessel
    ,
    Tushar Khot
    ,
    William Merrill
    ,
    Jacob Morrison
    ,
    Niklas Muennighoff
    ,
    Aakanksha Naik
    ,
    Crystal Nam
    ,
    Matthew E. Peters
    ,
    Valentina Pyatkin
    ,
    Abhilasha Ravichander
    ,
    Dustin Schwenk
    ,
    Saurabh Shah
    ,
    Will Smith
    ,
    Emma Strubell
    ,
    Nishant Subramani
    ,
    Mitchell Wortsman
    ,
    Pradeep Dasigi
    ,
    Nathan Lambert
    ,
    Kyle Richardson
    ,
    Luke Zettlemoyer
    ,
    Jesse Dodge
    ,
    Kyle Lo
    ,
    Luca Soldaini
    ,
    Noah A. Smith
    ,
    Hannaneh Hajishirzi
  70. OMG-QA: Building Open-Domain Multi-Modal Generative Question Answering Systems
    EMNLP 2024 ·
    Linyong Nan
    ,
    Weining Fang
    ,
    Aylin Rasteh
    ,
    Pouya Lahabi
    ,
    Weijin Zou
    ,
    Yilun Zhao
    , Arman Cohan
  71. Observable Propagation: Uncovering Feature Vectors in Transformers
    ICML 2024 ·
    Jacob Dunefsky
    , Arman Cohan
  72. On Evaluating the Integration of Reasoning and Action in LLM Agents with Database Question Answering
    NAACL 2024 ·
    Linyong Nan
    ,
    Ellen Zhang
    ,
    Weijin Zou
    ,
    Yilun Zhao
    ,
    Wenfei Zhou
    , Arman Cohan
  73. On Learning to Summarize with Large Language Models as References
    NAACL 2024 ·
    Yixin Liu
    ,
    Kejian Shi
    ,
    Katherine He
    ,
    Longtian Ye
    ,
    Alexander R. Fabbri
    ,
    Pengfei Liu
    ,
    Dragomir Radev
    , Arman Cohan
  74. OpenT2T: An Open-Source Toolkit for Table-to-Text Generation
    EMNLP 2024 ·
    Haowei Zhang
    ,
    Shengyun Si
    ,
    Yilun Zhao
    ,
    Lujing Xie
    ,
    Zhijian Xu
    ,
    Lyuhao Chen
    ,
    Linyong Nan
    ,
    Pengcheng Wang
    ,
    Xiangru Tang
    , Arman Cohan
  75. P-FOLIO: Evaluating and Improving Logical Reasoning with Abundant Human-Written Reasoning Chains
    EMNLP 2024 ·
    Simeng Han
    ,
    Aaron Yu
    ,
    Rui Shen
    ,
    Zhenting Qi
    ,
    Martin Riddell
    ,
    Wenfei Zhou
    ,
    Yujie Qiao
    ,
    Yilun Zhao
    ,
    Semih Yavuz
    ,
    Ye Liu
    ,
    Shafiq Joty
    ,
    Yingbo Zhou
    ,
    Caiming Xiong
    ,
    Dragomir Radev
    ,
    Rex Ying
    , Arman Cohan
  76. Quantifying Contamination in Evaluating Code Generation Capabilities of Language Models
    ACL 2024 ·
    Martin Riddell
    ,
    Ansong Ni
    , Arman Cohan
  77. Rethinking Efficient Multilingual Text Summarization Meta-Evaluation
    ACL 2024 ·
    Rilyn Han
    ,
    Jiawen Chen
    ,
    Yixin Liu
    , Arman Cohan
  78. SciDQA: A Deep Reading Comprehension Dataset over Scientific Papers
    EMNLP 2024 ·
    Shruti Singh
    ,
    Nandan Sarkar
    , Arman Cohan
  79. Struc-Bench: Are Large Language Models Good at Generating Complex Structured Tabular Data?
    NAACL 2024 ·
    Xiangru Tang
    ,
    Yiming Zong
    ,
    Jason Phang
    ,
    Yilun Zhao
    ,
    Wangchunshu Zhou
    , Arman Cohan,
    Mark Gerstein
  80. TAIL: A Toolkit for Automatic and Realistic Long-Context Large Language Model Evaluation
    EMNLP 2024 ·
    Gefei Gu
    ,
    Yilun Zhao
    ,
    Ruoxi Ning
    ,
    Yanan Zheng
    , Arman Cohan
  81. TaPERA: Enhancing Faithfulness and Interpretability in Long-Form Table QA by Content Planning and Execution-based Reasoning
    ACL 2024 ·
    Yilun Zhao
    ,
    Lyuhao Chen
    , Arman Cohan,
    Chen Zhao
  82. Unveiling the Spectrum of Data Contamination in Language Model: A Survey from Detection to Remediation
    ACL 2024 ·
    Chunyuan Deng
    ,
    Yilun Zhao
    ,
    Yuzhao Heng
    ,
    Yitong Li
    ,
    Jiannan Cao
    ,
    Xiangru Tang
    , Arman Cohan