“Virtual bench: a hybrid research and computation platform for digital surrogates of motion picture films”

FAIN: PR-284350-22

University of South Carolina (Columbia, SC 29208-0001)
Greg Wilsbacher (Project Director: May 2021 to present)
Song Wang (Co Project Director: March 2022 to August 2022)
Jun Zhou (Co Project Director: March 2022 to present)

A Tier II project to develop two specialized open-source software tools, Virtual Bench Research Platform and Virtual Bench Compute, for improving the preservation and material study of digitized film.

Motion picture film is more than an image. While the history of cinema provides ample evidence that the film industry from its beginnings strove to promote the illusion of an immaterial presence illuminating a screen in a darkened theater, the reality of film’s physical presence rolled through projectors in booths, weighed down shipping containers as it was shuttled from one theater to another, and fell to the cutting room floor during editing. The residue of a century of filmmaking (theatrical and non-theatrical, professional and amateur, documentary and fiction, news and nonsense) now resides in film archives once or twice removed from the industries and communities that produced the content. How this large collective archive will live on to be studied by scholars of the future remains a question without a satisfactory answer. The University of South Carolina seeks a $349,106 award to fund a two-year project that will push the boundaries of possibility for scholarly access to motion picture film elements surviving in film archives. We propose a two-pronged project that will demonstrate the inherent value of digitizing the entirety of a film element, known as a full overscan, to create a digital surrogate of the material motion picture film object.

Associated Products

Background-Insensitive Scene Text Recognition with Text Semantic Segmentation (Article)
Title: Background-Insensitive Scene Text Recognition with Text Semantic Segmentation
Author: Zhao, Liang
Author: Wilsbacher, Greg
Author: Wu, Zhenyao
Author: Wu, Xinyi
Author: Wang, Song
Abstract: Scene Text Recognition (STR) has many important applications in computer vision. Complex backgrounds continue to be a big challenge for STR because they interfere with text feature extraction. Many existing methods use attentional regions, bounding boxes or polygons to reduce such interference. However, the text regions located by these methods still contain much undesirable background interference. In this paper, we propose a Background-Insensitive approach BINet by explicitly leveraging the text Semantic Segmentation (SSN) to extract texts more accurately. SSN is trained on a set of existing segmentation data, whose volume is only 0.03% of STR training data. This prevents the large-scale pixel-level annotations of the STR training data. To effectively utilize the segmentation cues, we design new segmentation refinement and embedding blocks for refining text-masks and reinforcing visual features. Additionally, we propose an efficient pipeline that utilizes Synthetic Initialization (SI) for STR models trained only on real data (1.7% of STR training data), instead of on both synthetic and real data from scratch. Experiments show that the proposed method can recognize text from complex backgrounds more effectively, achieving state-of-the-art performance on several public datasets.
Year: 2022
Primary URL: https://link.springer.com/chapter/10.1007/978-3-031-19806-9_10
Primary URL Description: Computer Vision--ECCV 2022 (Conference Proceedings). Springer
Format: Journal
Periodical Title: European Conference on Computer Vision
Publisher: Springer

Virtual Bench: An Experimental Application of Artificial Intelligence for Archivists (Conference Paper/Presentation)
Title: Virtual Bench: An Experimental Application of Artificial Intelligence for Archivists
Author: Image Archivists of Moving Association
Author: Wilsbacher, Greg
Author: Aschenbach, Tommy
Abstract: Aschenbach and Wilsbacher delivered a talk to a large audience at the Association of Moving Image Archivists in which they described the scope and goals of the grant and demonstrated an alpha version of the Virtual Bench Research Platform as well as sample output of the splice detection algorithm.
Date: 12/9/2022
Primary URL: https://amiaconference.net/amia-2022-program/
Primary URL Description: Conference webpage.
Conference Name: Image Archivists of Moving Association

---