본문 바로가기

SciNumQA: A Benchmark for Complex Numerical Reasoning over Scientific Tables

문서 유형
연구출판물 · 학술지 논문
등재
SCIEScopus
저널 순위
JCR Q2Scopus Q1
저널
IEEE ACCESS14 · pp.82812 · 2026
피인용
Web of Science 0Scopus 0
FWCI (Scopus)
0
JCR IF
JCR 2024 3.6
ISSN
2169-3536

저자

  1. Son, Sohyun제1저자Kyung Hee Univ, Dept Artificial Intelligence, Yongin 17104, South Korea
  2. Kim, YubinHanyang Univ, Dept Artificial Intelligence, Seoul 04763, South Korea
  3. Kim, GangwooAmazon Web Serv AWS, Seattle, WA 98109 USA
  4. 최동희교신저자Pusan Natl Univ, Sch Comp Sci & Engn, Busan 46241, South Korea
  5. Sung, Mujeen교신저자Kyung Hee Univ, Sch Comp, Yongin 17104, South Korea

초록

Numerical reasoning is crucial for extracting meaningful information from scientific tables, enabling deeper analysis beyond simple lookup. As automated analysis of structured data becomes increasingly important not only in scientific research but also in industrial applications such as technical decisionmaking and compliance, the demand for robust numerical reasoning has grown. While recent advances in large language models (LLMs) have improved tabular reasoning, the ability to perform advanced numerical reasoning over scientific tables remains underexplored. Most prior work has focused on simple lookup or basic numerical tasks, often using small or trivial tables. In this work, we introduce SciNumQA, a benchmark dataset comprising 1,607 question-answer pairs based on 572 scientific tables, specifically designed to evaluate complex, multi-step numerical reasoning. Our benchmark systematically filters out trivial cases, ensuring that every question requires compositional reasoning and explicit numerical operations. Experiments show that even recent LLMs (e.g., GPT-4.1 mini) struggle to perform accurate reasoning on these challenging tasks, particularly when table structure becomes more complex. This highlights the need for more robust and generalizable models for scientific table understanding and advanced numerical reasoning. The SciNumQA dataset is available at this link.

키워드

Benchmark testingLarge language modelsMachine learningNatural language processingNumerical analysis

식별자

Scopus EID
2-s2.0-105040643598
WoS UID
001786037400028