본문 바로가기

Motion-to-Attention: Enhancing Attention Maps to Improve Performance of Text-Guided Video Editing

문서 유형
연구출판물 · 학술지 논문
등재
SCIEScopus
저널 순위
JCR Q1Scopus Q1
피인용
Web of Science 0Scopus 0
FWCI (Scopus)
0
JCR IF
JCR 2024 6
ISSN
1551-6857

저자

  1. 정성훈제1저자Pusan Natl Univ, Dept Elect Engn, Busan, South Korea
  2. 진인환Pusan Natl Univ, Dept Elect Engn, Busan, South Korea
  3. Choo, HaesooPukyong Natl Univ, Busan, South Korea
  4. Na, HyeonjunPukyong Natl Univ, Busan, South Korea
  5. 공경보교신저자Pusan Natl Univ, JYS AI Nexus Inst, Busan, South Korea

초록

Recent research in text-guided video editing aims to extend image-based editing models to video domains. A significant challenge in this transition is ensuring temporal consistency across frames. However, existing methods often exhibit limited editing accuracy when processing prompts associated with motion, such as “floating” or “moving.” Our analysis indicates that this limitation arises from inaccurate attention maps corresponding to motion-related prompts. To address this, we introduce the Motion-to-Attention (M2A) module, explicitly integrating motion information for enhanced video editing precision. Specifically, we first convert optical flow extracted from the video into a comprehensive motion map. Optionally, users can specify directional information to refine motion map extraction further. The proposed M2A module incorporates two complementary techniques: “Attention–Motion Swap,” which directly substitutes the imprecise attention map of motion prompts with the extracted motion map, and “Attention–Motion Fusion,” which adaptively enhances attention maps based on the correlation with the motion map using a carefully selected Fusion metric. Experimental validation demonstrates that incorporating our M2A module into existing text-to-video editing frameworks significantly improves both quantitative performance metrics (CLIP-Acc, Masked PSNR, BRISQUE) and qualitative visual quality. Extensive experiments and comparative studies confirm the superior editability and robustness of our method over current state-of-the-art approaches. Comprehensive results are publicly available at https://currycurry915.github.io/Motion-to-Attention/.

키워드

attentionoptical flowVideo editing

식별자

Scopus EID
2-s2.0-105044000726
WoS UID
001829277100001