SnowVision: A Machine Learning-Based Image Processing Tool for the Study of Archaeological Collections

FAIN: HAA-266472-19

South Carolina Department of Natural Resources (Columbia, SC 29202-0167)
Karen Yvonne Smith (Project Director: January 2019 to present)
Song Wang (Co Project Director: May 2019 to present)
Jun Zhou (Co Project Director: May 2019 to present)
Colin Wilder (Co Project Director: May 2019 to present)

The expansion and extension of a set of machine learning-based tools designed to assist scholars with identifying and classifying artifacts from archaeological sites based on design motifs.

Two years of NEH Digital Humanities Advancement Grant Level III funding is sought to increase availability and strengthen usability of SnowVision. The grant will support 1) the integration of SnowVision with an interactive, online user interface, 2) the acquisition and integration feedback from scholars working in laboratories and curation facilities across the Southeast, 3) the enhancement of the technological infrastructure of SnowVision so that the newly integrated system meets the needs of the user community and has a framework built for long-term success, and 4) providing select institutions with start-up funds to begin digitizing collections, providing the USC team with rigorous, off-site testing of the system. Collaboration between the USC development team and an Advisory Committee will increase the utility of SnowVision, secure buy-in from stakeholders, and ensure extensibility of the software. NEH funding will support software enhancement of accuracy, reliability, and speed.

Associated Products

Building Science Gateways for Humanities (Conference Paper/Presentation)
Title: Building Science Gateways for Humanities
Author: Zhou, Jun
Author: Karen Smith
Author: Greg Wilsbacher
Author: Paul Sagona
Author: David Reddy
Author: Ben Torkian
Abstract: Building science gateways for humanities content poses new challenges to the science gateway community. Compared with science gateways devoted to scientific content, humanities-related projects usually require 1) processing data in various formats, such as text, image, video, etc., 2) constant public access from a broad audience, and therefore 3) reliable security, ideally with low maintenance. Many traditional science gateways are monolithic in design, which makes them easier to write, but they can be computationally inefficient when integrated with numerous scientific packages for data capture and pipeline processing. Since these packages tend to be single-threaded or nonmodular, they can create traffic bottlenecks when processing large numbers of requests. Moreover, these science gateways are usually challenging to resume development on due to long gaps between funding periods and the aging of the integrated scientific packages. In this paper, we study the problem of building science gateways for humanities projects by developing a service-based architecture, and present two such science gateways: the Moving Image Research Collections (MIRC) – a science gateway focusing on image analysis for digital surrogates of historical motion picture film, and SnowVision - a science gateway for studying pottery fragments in southeastern North America. For each science gateway, we present an overview of the background of the projects, and some unique challenges in their design and implementation. These two science gateways are deployed on XSEDE’s Jetstream academic clouding computing resource and are accessed through web interfaces. Apache Airavata middleware is used to manage the interactions between the web interface and the deep-learning-based (DL) backend service running on the Bridges graphics processing unit (GPU) cluster.
Date: 07/28/2020
Primary URL: https://dl.acm.org/doi/10.1145/3311790.3396628
Primary URL Description: AMC Digital Library
Conference Name: Practice and Experience in Advanced Research Computing: PEARC '20

Snowvision: The Promise of Algorithmic Methods in Southeastern Archaeological Research (Article)
Title: Snowvision: The Promise of Algorithmic Methods in Southeastern Archaeological Research
Author: Colin Wilder
Author: Sam T. McDorman
Author: Jun Zhou
Author: Adam King
Author: Yuhang Lu
Author: Karen Y. Smith
Author: Song Wang
Author: W. Matthew J. Simmons
Abstract: This article presents the contexts, methods, contributions, and preliminary findings of Snowvision, a digital archaeology project developed by faculty and students at the University of South Carolina and the South Carolina Department of Natural Resources. Snowvision uses computer vision to reconstruct southeastern Native American paddle designs from the Swift Creek period, ca. 100-850 CE. In this essay, we first present the context of the Swift Creek culture of the southeastern United States, along with broader related issues in prehistoric archaeology. Then, the relevant methods from archaeology and computer vision are introduced and discussed. We also introduce World Engraved, our public-facing digital archive of sherd designs and distributions, and explain its role in our overall project. We then explore, in some level of technical detail, the ways in which our work refines existing pattern-matching algorithms used in the field of computer vision. Finally, we discuss our accomplishments and findings to date and the possibilities for future research that Snowvision provides.
Year: 2020
Primary URL: https://paas.org.pl/wp-content/uploads/2020/12/PJAS_14_autumn_2020.pdf
Access Model: open access
Format: Journal
Periodical Title: Polish Journal for American Studies
Publisher: Polish Association for American Studies

Snowvision: Segmenting, Identifying, and Discovering Stamped Curve Patterns from Fragments of Pottery (Article)
Title: Snowvision: Segmenting, Identifying, and Discovering Stamped Curve Patterns from Fragments of Pottery
Author: Yuhang Lu
Author: Jun Zhou
Author: Sam T. McDorman
Author: Canyu Zhang
Author: Deja Scott
Author: Jake Bukuts
Author: Colin Wilder
Author: Karen Y. Smith
Author: Song Wang
Abstract: In southeastern North America, Indigenous potters and woodworkers carved complex, primarily abstract, designs into wooden pottery paddles, which were subsequently used to thin the walls of hand-built, clay vessels. Original paddle designs carry rich historical and cultural information, but pottery paddles from ancient times have not survived. Archaeologists have studied design fragments stamped on sherds to reconstruct complete or nearly complete designs, which is extremely laborious and time-consuming. In Snowvision, we aim to develop computer vision methods to assist archaeologists to accomplish this goal more efficiently and effectively. For this purpose, we identify and study three computer vision tasks: (1) extracting curve structures stamped on pottery sherds; (2) matching sherds to known designs; (3) clustering sherds with unknown designs. Due to the noisy, highly fragmented, composite-curve patterns, each task poses unique challenges to existing methods. To solve them, we propose (1) a weakly-supervised CNN-based curve structure segmentation method that takes only curve skeleton labels to predict full curve masks; (2) a patch-based curve pattern matching method to address the problem of partial matching in terms of noisy binary images; (3) a curve pattern clustering method consisting of pairwise curve matching, graph partitioning and sherd stitching. We evaluate the proposed methods on a set of collected sherds and extensive experimental results show the effectiveness of the proposed algorithms.
Year: 2022
Primary URL: https://trebuchet.public.springernature.app/get_content/dfc427fe-5f36-4d45-b2e9-e673e8f36eb5
Secondary URL: https://link.springer.com/article/10.1007/s11263-022-01669-7
Access Model: Public for 30 days; Subscription only after 30 days
Format: Journal
Periodical Title: International Journal of Computer Vision
Publisher: Springer

Introduction to Snowvision and World Engraved (Conference Paper/Presentation)
Title: Introduction to Snowvision and World Engraved
Author: McDorman, Sam T.
Author: Smith, Karen Y.
Abstract: The Snowvision Project is an interdisciplinary, interagency effort aimed at advancing the study of Southeastern complicated stamped ceramics. Developed at the University of South Carolina, computer vision algorithms match 3D depth patterns on sherds to reconstructed paddle designs and to RGB (photo) images. Project depth and RGB images, along with robust metadata, are shared through the World Engraved website. Although Snowvision has been discussed in computer science and humanities publications, this poster introduces the project to the archaeological community for the first time. Project funding, publications, algorithms, website, student involvement, and research partnerships will be reviewed.
Date: 10/28/2023
Primary URL: https://core.tdar.org/document/490855/introduction-to-snowvision-and-world-engraved
Conference Name: 79th Annual Meeting of the Southeastern Archaeological Conference

World Engraved (Web Resource)
Title: World Engraved
Author: Snowvision Team
Abstract: World Engraved is a website for sharing design, sherd, and Snowvision algorthim information. Significantly improved for the project, the website provides customizable search queries for designs and sherds. The ability to filter by site, design, or design attribute provides users with the opportunity to browse designs by location or style.
Year: 2020
Primary URL: https://www.worldengraved.org/

Contour-based Shape Matching and Segmentation of Cultural Heritage Objects (Conference Paper/Presentation)
Title: Contour-based Shape Matching and Segmentation of Cultural Heritage Objects
Author: Scott, Deja
Abstract: Within the Snowvision Project, there was a critical need to find a solution for splitting images of individual sherds from an image containing several North American Southeastern paddle stamped sherds. Utilizing this science gateway enables researchers to share information about cultural heritage objects through a web interface and gain additional insight from further analysis. The Contour-Shape matching algorithm is a data preprocessing step that will segment groups of sherds into individual images and map them to their respective depth (.xyz) image. In this context, an RGB image is a colour image taken with a traditional camera and a depth image is a 3D scan of the object. To test the Contour-Based Shape Matching Algorithm’s matching accuracy, the algorithm was tested on a data set containing 101 RGB images, each containing 1 to 6 sherds, and 136 depth images, which contained a singular scan of a sherd. Once the algorithm has been run, it identifies the shape of the individual sherds from contours produced by analyzing binary versions of the original images. These images were then compared to the contours found in the depth images with each sherd contour being compared to an individual depth contour with the lowest score of dissimilarity in the set confirming the match. For future processing, the new segmented sherd, respective depth, and the resulting image are stored in a subdirectory denominated with the sherd’s identifying name. The method was improved upon by blocking out non-sherd objects with a binary mask. The results of the test indicate that the sherds were accurately matched with corresponding depth images regardless of rotation, orientation or scale of both images. Additionally, the algorithm was successfully able to separate multiple sherds from an RGB image into its own RGB image and correlate to a depth image match. We concluded that through the Contour-Based Shape Matching Algorithm, sherds were able to be separated from their original RGB image.
Date: 04/23/2021
Primary URL: https://sc.edu/about/signature_events/discover_uofsc/documents/2021_abstract_book_final.pdf
Primary URL Description: Collected Abstracts
Conference Name: DISCOVER UofSC

Digital Collection and Dissemination of Southeastern Complicated Stamped Pottery: A User Needs Study for Snowvision/World Engraved (Conference Paper/Presentation)
Title: Digital Collection and Dissemination of Southeastern Complicated Stamped Pottery: A User Needs Study for Snowvision/World Engraved
Author: McDorman, Sam T.
Abstract: Complicated stamped pottery is ubiquitous across what is now called the southeastern United States. Vessels were stamped with carved wooden paddles that do not survive in the archaeological record, but the paddle designs can be reconstructed from careful study of the pottery. The most complex type, called Swift Creek, was produced from approximately 100-850 AD and is found primarily in Georgia, Florida, and Alabama. The process of sorting thousands of pottery sherds across hundreds of sites to identify matches is a daunting task for archaeologists but is necessary to reconstruct paddle designs and to understand the movements of and connections between the people who created these artifacts. To aid archaeologists in this task, a multi-disciplinary team at UofSC has built Snowvision, a machine learning computer vision algorithm to match sherds and designs, and World Engraved, a free and public online digital archive of reconstructed designs and sherds to accumulate data from across the southeast. My thesis builds on the fields of information science and archaeology by using a two-part user needs study to gain an understanding of the expected user population. A survey was distributed in late 2019, followed by user testing and interviews conducted virtually in late 2020. Respondents were professional archaeologists interested in sharing large volumes of data held at their institutions. Respondent feedback was incorporated into the interface of the World Engraved website, created changes to proposed metadata collection, and assisted in the development of data sharing policies for the digital archive.
Date: 04/23/2021
Primary URL: https://sc.edu/about/signature_events/discover_uofsc/documents/2021_abstract_book_final.pdf
Primary URL Description: Collected Abstracts
Conference Name: DISCOVER UofSC

Swift Creek Design Organization at the Woodland Period Arcuate Community of Hartford (Book Section)
Title: Swift Creek Design Organization at the Woodland Period Arcuate Community of Hartford
Author: Smith, Karen Y.
Author: Stephenson, Keith
Author: Snow, Frankie
Editor: Hollingshead, Analise
Editor: Messer, Haley
Editor: Menz, Martin
Abstract: This volume is an important contribution to our understanding of the social dynamics and evolution of ring-shaped villages throughout Eastern North America, particularly from the Woodland through Mississippian periods, a time when villages first became widespread throughout this half of the continent. A great deal of attention has been paid to Archaic shell rings along the Lower Atlantic and Gulf Coasts of North America, as these may represent some of the earliest nucleated communities in the region. Less attention has been given to later, Woodland period ring-shaped settlements, which proliferated throughout the Eastern Woodlands in a much wider variety of environmental contexts. Our volume addresses this gap in knowledge by assembling case studies analyzing the social and spatial organization of Woodland and Early Mississippian ring-shaped settlements and how these changed through time across a broad swathe of the region: from Iowa to Florida and Ohio to Louisiana.
Year: 2022
Publisher: University of Alabama Press
Book Title: Arcuate Communities of the Eastern Woodlands: Spatial Patterning and Settlement in the Eastern Woodlands
ISBN: In Press

Swift Creek Pottery from Pinson Mounds and the Development of a Complicated Stamped Pottery Design Matching Application (Blog Post)
Title: Swift Creek Pottery from Pinson Mounds and the Development of a Complicated Stamped Pottery Design Matching Application
Author: Keith, Scot
Author: Smith, Karen
Author: Blackmon, Josh
Abstract: This blog post introduces Snowvision and presents our efforts to extend the scanning project to new collections, including the small but significant Swift Creek pottery assemblage from Pinson Mounds.
Date: 09/18/2019
Primary URL: https://tennesseearchaeologycouncil.wordpress.com/2019/09/18/swift-creek-pottery-from-pinson-mounds-and-the-development-of-a-complicated-stamped-pottery-design-matching-application/
Blog Title: 30 Days of Tennessee Archaeology – 2019
Website: TENNESSEE COUNCIL FOR PROFESSIONAL ARCHAEOLOGY

---