All Predictions Matter: an Online Video Prediction Approach - Equipe Multimédia Accéder directement au contenu
Communication Dans Un Congrès Année : 2023

All Predictions Matter: an Online Video Prediction Approach

Résumé

To effectively manage and utilize the massive amount of visual data generated by the surging number of videos, decision-making systems must predict and reason about future outcomes. This paper proposes a novel online approach for video prediction that enables continual learning in the presence of new data, as periodic training of neural networks may not be practical. We utilize all predictions, including intermediate computations obtained during the inference process, to improve the performance of video prediction. To achieve this, we incorporate a weighting scheme in the loss that accounts for all the predictions during the learning process. Additionally, we leverage semantic segmentation to assess the performance of extrapolated frames by focusing on the position of the objects in the scene. Our approach stands out from state-of-the-art methods as it uses intermediate predictions, which are available due to the iterative nature of forecasting future frames. Our method improves the offline counterpart for the same network by 1.45 dB for predicting five steps in the future.
Fichier principal
Vignette du fichier
EUVIP_2023_Melan_final_camera_ready.pdf (431.03 Ko) Télécharger le fichier
Origine : Fichiers produits par l'(les) auteur(s)

Dates et versions

hal-04204130 , version 1 (11-09-2023)

Identifiants

  • HAL Id : hal-04204130 , version 1

Citer

Melan Vijayaratnam, Marco Cagnazzo, Giuseppe Valenzise, Enzo Tartaglione. All Predictions Matter: an Online Video Prediction Approach. 11th European Workshop on Visual Information Processing, Sep 2023, Gjøvik, Norway. ⟨hal-04204130⟩
92 Consultations
33 Téléchargements

Partager

Gmail Facebook X LinkedIn More