SciNumQA: A Benchmark for Complex Numerical Reasoning over Scientific Tables
- 문서 유형
- 연구출판물 · 학술지 논문
- 등재
- SCIEScopus
- 저널 순위
- JCR Q2Scopus Q1
- 저널
- IEEE ACCESS14 · pp.82812 · 2026
- 피인용
- Web of Science 0Scopus 0
- FWCI (Scopus)
- 0
- JCR IF
- JCR 2024 3.6
- ISSN
- 2169-3536
저자
- Son, Sohyun제1저자Kyung Hee Univ, Dept Artificial Intelligence, Yongin 17104, South Korea
- Kim, YubinHanyang Univ, Dept Artificial Intelligence, Seoul 04763, South Korea
- Kim, GangwooAmazon Web Serv AWS, Seattle, WA 98109 USA
- 최동희교신저자Pusan Natl Univ, Sch Comp Sci & Engn, Busan 46241, South Korea
- Sung, Mujeen교신저자Kyung Hee Univ, Sch Comp, Yongin 17104, South Korea
초록
Numerical reasoning is crucial for extracting meaningful information from scientific tables, enabling deeper analysis beyond simple lookup. As automated analysis of structured data becomes increasingly important not only in scientific research but also in industrial applications such as technical decisionmaking and compliance, the demand for robust numerical reasoning has grown. While recent advances in large language models (LLMs) have improved tabular reasoning, the ability to perform advanced numerical reasoning over scientific tables remains underexplored. Most prior work has focused on simple lookup or basic numerical tasks, often using small or trivial tables. In this work, we introduce SciNumQA, a benchmark dataset comprising 1,607 question-answer pairs based on 572 scientific tables, specifically designed to evaluate complex, multi-step numerical reasoning. Our benchmark systematically filters out trivial cases, ensuring that every question requires compositional reasoning and explicit numerical operations. Experiments show that even recent LLMs (e.g., GPT-4.1 mini) struggle to perform accurate reasoning on these challenging tasks, particularly when table structure becomes more complex. This highlights the need for more robust and generalizable models for scientific table understanding and advanced numerical reasoning. The SciNumQA dataset is available at this link.
키워드
Benchmark testingLarge language modelsMachine learningNatural language processingNumerical analysis
식별자
- Scopus EID
- 2-s2.0-105040643598
- WoS UID
- 001786037400028



로그인
연구물 목록으로