Social Video Advertisement Replacement and its Evaluation in Convolutional Neural Networks
Yang, Cheng (Auckland University of Technology (Nova Zelanda). Department of Electrical and Electronic Engineering)
Yu, Xiang (Auckland University of Technology (Nova Zelanda))
Kumar, Arun (National Institute of Technology (Odisha, Índia). Department of Computer Science & Engineering)
Ali, G. G. Md. Nawaz (University of Charleston (Estats Units d'Amèrica). Department of Applied Computer Science)
Chong, Peter Han Joo (Auckland University of Technology (Nova Zelanda). Department of Electrical and Electronic Engineering)
Lam, Patrick (Auckland University of Technology (Nova Zelanda))

Date:	2021
Abstract:	This paper introduces a method to use deep convolutional neural networks (CNNs) to automatically replace advertisement (AD) photo on social (or self-media) videos and provides the suitable evaluation method to compare different CNNs. An AD photo can replace a picture inside a video. However, if a human being occludes the replaced picture in the original video, the newly pasted AD photo will block the human occluded part. The deep learning algorithm is implemented to segment the human being from the video. The segmented human pixels are then pasted back to the occluded area, so that the AD photo replacement becomes natural and perfect appearance in the video. This process requires the predicted occlusion edge to be closed to the ground truth occlusion edge, so that the AD photo can be occluded naturally. Therefore, this research introduces a curve fitting method to measure the predicted occlusion edge's error. By using this method, three CNN methods are applied and compared for the AD replacement. They are mask of regions convolutional neural network (Mask RCNN), recurrent network for video object segmentation (ROVS) and DeeplabV3. The experimental results show the comparative segmentation accuracy of the different models and DeeplabV3 shows the best performance.
Rights:	Aquest document està subjecte a una llicència d'ús Creative Commons. Es permet la reproducció total o parcial, la distribució, i la comunicació pública de l'obra, sempre que no sigui amb finalitats comercials, i sempre que es reconegui l'autoria de l'obra original. No es permet la creació d'obres derivades.
Language:	Anglès
Document:	Article ; recerca ; Versió publicada
Subject:	Deep Learning ; Image Processing ; Image Segmentation ; Video Advertisement Replacement
Published in:	ELCVIA : Electronic Letters on Computer Vision and Image Analysis, Vol. 20 Núm. 1 (2021) , p. 117-136 (Regular Issue) , ISSN 1577-5097

Adreça original: https://elcvia.cvc.uab.es/article/view/v20-n1-yang
DOI: 10.5565/rev/elcvia.1347

20 p, 756.0 KB

The record appears in these collections:
Articles > Published articles > ELCVIA
Articles > Research articles