Motion-to-Attention: Enhancing Attention Maps to Improve Performance of Text-Guided Video Editing
- 문서 유형
- 연구출판물 · 학술지 논문
- 등재
- SCIEScopus
- 저널 순위
- JCR Q1Scopus Q1
- 피인용
- Web of Science 0Scopus 0
- FWCI (Scopus)
- 0
- JCR IF
- JCR 2024 6
- ISSN
- 1551-6857
저자
초록
Recent research in text-guided video editing aims to extend image-based editing models to video domains. A significant challenge in this transition is ensuring temporal consistency across frames. However, existing methods often exhibit limited editing accuracy when processing prompts associated with motion, such as “floating” or “moving.” Our analysis indicates that this limitation arises from inaccurate attention maps corresponding to motion-related prompts. To address this, we introduce the Motion-to-Attention (M2A) module, explicitly integrating motion information for enhanced video editing precision. Specifically, we first convert optical flow extracted from the video into a comprehensive motion map. Optionally, users can specify directional information to refine motion map extraction further. The proposed M2A module incorporates two complementary techniques: “Attention–Motion Swap,” which directly substitutes the imprecise attention map of motion prompts with the extracted motion map, and “Attention–Motion Fusion,” which adaptively enhances attention maps based on the correlation with the motion map using a carefully selected Fusion metric. Experimental validation demonstrates that incorporating our M2A module into existing text-to-video editing frameworks significantly improves both quantitative performance metrics (CLIP-Acc, Masked PSNR, BRISQUE) and qualitative visual quality. Extensive experiments and comparative studies confirm the superior editability and robustness of our method over current state-of-the-art approaches. Comprehensive results are publicly available at https://currycurry915.github.io/Motion-to-Attention/.
키워드
식별자
- Scopus EID
- 2-s2.0-105044000726
- WoS UID
- 001829277100001



로그인
연구물 목록으로