본문 바로가기

Noise Resilience of Successor and Predecessor Feature Algorithms in One- and Two-Dimensional Environments

문서 유형
연구출판물 · 학술지 논문
등재
SCIEScopus
저널 순위
JCR Q2Scopus Q1
저널
SENSORS25(3) · 2025
피인용
Web of Science 1Scopus 2
FWCI (Scopus)
0.44
JCR IF
JCR 2024 3.5
ISSN
1424-3210

저자

  1. 이현수제1저자교신저자Pusan Natl Univ, Sch Med, Dept Physiol, Busandaehak Ro, Yangsan 50612, South Korea

초록

Noisy inputs pose significant challenges for reinforcement learning (RL) agents navigating real-world environments. While animals demonstrate robust spatial learning under dynamic conditions, the mechanisms underlying this resilience remain understudied in RL frameworks. This paper introduces a novel comparative analysis of predecessor feature (PF) and successor feature (SF) algorithms under controlled noise conditions, revealing several insights. Our key innovation lies in demonstrating that SF algorithms achieve superior noise resilience compared to traditional approaches, with cumulative rewards of (Formula presented.) (mean ± SEM), even under high noise conditions ( (Formula presented.) ) in one-dimensional environments, while Q learning achieves only (Formula presented.). In two-dimensional environments, we discover an unprecedented nonlinear relationship between noise level and algorithm performance, with SF showing optimal performance at moderate noise levels ( (Formula presented.) ), achieving cumulative rewards of (Formula presented.) compared to (Formula presented.) for Q learning. The (Formula presented.) parameter in PF learning is a significant factor, with (Formula presented.) consistently achieving higher (Formula presented.) values under most noise conditions. These findings bridge computational neuroscience and RL, offering practical insights for developing noise-resistant learning systems. Our results have direct applications in robotics, autonomous navigation, and sensor-based AI systems, particularly in environments with inherent observational uncertainty.

키워드

autonomous navigationnoisy environmentpredecessor feature learningreinforcement learningsuccessor features learningSPATIAL MEMORYHIPPOCAMPALREPTILESMAPGO

식별자

Scopus EID
2-s2.0-85217620098
WoS UID
001433178500001