Affiliation:
1. Computer Science Dept., Universidade de Vigo, Escuela Superior de Ingeniería Informática, Ourense, Spain and CINBIO - Biomedical Research Centre, Universidade de Vigo, Vigo, Spain and SING Research Group, Galicia Sur Health Research Institute (IIS Galicia Sur), Vigo, Spain
Abstract
Image generative models have advanced in many areas to produce synthetic images of high resolution and detail. This success has enabled its use in the biomedical field, paving the way for the generation of videos showing the biological evolution of its content. Despite the power of generative video models, their use has not yet extended to time-based development, focusing almost exclusively on generating motion in space. This situation is largely due to the lack of specific data sets and metrics to measure the individual quality of videos, particularly when there is no ground truth available for comparison. We propose a new dataset, called GoldenDOT, which tracks the evolution of apples cut in parallel over 10 days, allowing to observe their progress over time while remaining static. In addition, four new metrics are proposed that provide different analyses of the generated videos as a whole and individually. In this paper, the proposed dataset and measures are used to study three state of the art video generative models and their feasibility for video generation with biological development: TemporalGAN (TGANv2), Low Dimensional Video Discriminator GAN (LDVDGAN), and Video Diffusion Model (VDM). Among them, the TGANv2 model has managed to obtain the best results in the vast majority of metrics, including those already known in the state of the art, demonstrating the viability of the new proposed metrics and their congruence with these standard measures.
Publisher
Association for Computing Machinery (ACM)
Reference50 articles.
1. Video Generative Adversarial Networks: A Review
2. Gedas Bertasius, Heng Wang, and Lorenzo Torresani. 2021. Is Space-Time Attention All You Need for Video Understanding?. In Proceedings of the 38th International Conference on Machine Learning(Proceedings of Machine Learning Research, Vol. 139), Marina Meila and Tong Zhang (Eds.). PMLR, Virtual conference, 813–824.
3. Global, voxel, and cluster tests, by theory and permutation, for a difference between two groups of structural MR images of the brain
4. A survey on deep learning applied to medical images: from simple artificial neural networks to generative models
5. P. Celard A Seara Vieira E. L. Iglesias and L. Borrajo. 2023. GoldenDOT: Biological Development Time-Lapse Video Dataset. Universidade de Vigo. https://doi.org/10.5281/zenodo.7778821