Scientific Workflows, Image Analysis, and Visual Stylometry in the Digital Analysis of Art

FAIN: HD-248360-16

Fitchburg State University (Fitchburg, MA 01420-2697)
Catherine Buell (Project Director: September 2015 to June 2019)
Ricky Sethi (Co Project Director: March 2016 to June 2019)
William P. Seeley (Co Project Director: March 2016 to June 2019)

The development of an alpha-level prototype for a tool that would help statistically identify artistic style, and a workshop to discuss the tool’s use and implications in the study of art history.

The goal of the project is to develop a tool for digital image analysis of paintings that is powerful enough to support advanced research in computer science, cognitive science, art history, and the philosophy of art while providing an accessible interface that can be used by researchers or students with little or no computer science background. The tool we envision will implement a broad range of digital image analysis algorithms as scientific workflows using the WINGS semantic workflow system. Scientific workflows allow users to build programs like one would draw a flowchart, dragging shapes representing data sets and image analysis procedures onto the workspace and drawing links between them. The tool can be used to promote computational literacy and data analytic skills among humanities students, introduce science students to research in art and the humanities, and help us understand how viewers perceptually categorize/recognize paintings and otherwise engage with artworks.

Media Coverage

Using computers to better understand art (Media Coverage)
Author(s): Ricky Sethi, William Seeley, Catherine Buell
Publication: The Conversation
Date: 7/7/2016
Abstract: When we talk about paintings, artistic style can refer to image features like the brushstrokes, contour and distribution of colors that painters employ, often implicitly, to construct their works. An artist’s style helps convey meaning and intent, and affects the aesthetic experience a user has when interacting with that artwork. Style also helps us identify and sometimes categorize their work, often placing it in the context of a specific period or place.
URL: http://theconversation.com/using-computers-to-better-understand-art-56887

Fitchburg State faculty receive grant to further develop art-study tool (Media Coverage)
Author(s): Sentinel & Enterprise
Publication: Sentinel & Enterprise
Date: 3/24/2016
Abstract: Local newspaper covering the announcement of the NEH grant recipients and the project at Fitchburg State University.
URL: http://www.sentinelandenterprise.com/breakingnews/ci_29680360/fitchburg-state-faculty-receive-grant-further-develop-art

Faculty members awarded national grant for art exploration (Media Coverage)
Author(s): Matthew Bruun
Publication: Fitchburg State University Today
Date: 4/4/2016
Abstract: Announcement to the Fitchburg State University committee about the NEH grant and the project.

Where the canvas meets the code (Media Coverage)
Author(s): Elizabeth Dobbins
Publication: Sentinel & Enterprise
Date: 3/16/2017
Abstract: FSU computer project offers new tool to explore artistry at the local art museum.
URL: http://www.sentinelandenterprise.com/news/ci_30860217/where-canvas-meets-code

Using Intelligent Workflows to Analyze Artistic Style (Review)
Author(s): Yolanda Gil, Ricky Sethi, Catherine Buell, William Seeley
Publication: ISI I-Knowledge Capture News
Date: 6/1/2017
Abstract: Review of the Making WAIVS! workshop
URL: https://www.isi.edu/ikcap/news/WAIVS-2017/

Associated Products

WAIVS website (Web Resource)
Title: WAIVS website
Author: Catherine Buell
Author: Ricky Sethi
Author: William Seeley
Abstract: The WAIVS website serves as an introduction to the future WAIVS interface, a timeline of events, and eventually a portal for usage.
Year: 2016
Primary URL: http://waivs.org/
Primary URL Description: Homepage for WAIVS

Making WAIVS! (Computer Program)
Title: Making WAIVS!
Author: Catherine Buell
Author: Ricky Sethi
Author: Allen Perry
Author: William Ianacci
Author: Peter Cole
Abstract: Welcome to the WAIVS The goal of our project is to develop a tool called WAIVS (Workflows for Analysis of Images and Visual Stylometry) for digital image analysis of paintings that is powerful enough to support advanced research in computer science, cognitive science, art history, and the philosophy of art while providing an accessible interface that can be used by researchers or students with little or no computer science background. A scientific workflow made in WAIVS is essentially a computer program. So once the user designs the workflow, they can simply click a button to conduct the analysis. Scientific workflows are different from ordinary flowcharts in that they can be directly executed. Once the user designs the workflow, they can simply click the run button to execute the program and conduct the analysis. WAIVS, built upon the Wings workflow system, includes the latest computer vision and artistic style algorithms.
Year: 2017
Primary URL: http://wings.fitchburgcsclub.net/wings-portal/
Primary URL Description: This URL brings the user to the tutorials and log-in screen.
Access Model: Subsciption only (right now)
Programming Language/Platform: WINGS supported.
Source Available?: No

Reproducibility in computer vision: Towards open publication of image analysis experiments as semantic workflows (Conference Paper/Presentation)
Title: Reproducibility in computer vision: Towards open publication of image analysis experiments as semantic workflows
Author: Ricky J. Sethi
Author: Yolanda Gil
Abstract: Reproducibility of research is an area of growing concern in computer vision. Scientific workflows provide a structured methodology for standardized replication and testing of state-of-the-art models, open publication of datasets and software together, and ease of analysis by re-using pre-existing components. In this paper, we present initial work in developing a framework that will allow reuse and extension of many computer vision methods, as well as allowing easy reproducibility of analytical results, by publishing dadasets and workflows packaged together as linked data. Our approach uses the WINGS semantic workflow system which validates semantic constraints of the computer vision algorithms, making it easy for non-experts to correctly apply state-of-the-art image processing methods to their data. We show the ease of use of semantic workflows for reproducibility in computer vision by both utilizing pre-developed workflow fragments and developing novel computer vision workflow fragments for a video activity recognition task, analysis of multimedia web content, and the analysis of artistic style in paintings using convolutional neural networks.
Date: 2016-10-30
Primary URL: http://ieeexplore.ieee.org/abstract/document/7870918/
Primary URL Description: IEEE DOI
Conference Name: 2016 IEEE 12th International Conference on e-Science (e-Science)

What Can Cognitive Science Tell Us about Art? (Conference Paper/Presentation)
Title: What Can Cognitive Science Tell Us about Art?
Author: William P. Seeley
Abstract: There has been an explosion of interest in the marriage of cognitive science and aesthetics in recent years. However, some prominent philosophers (Alva Noë, Strange Tools, Hill and Wang, 2016) and art critics (Blake Gopnik, “Aesthetic Science and Artistic Knowledge,” in eds. Arthur Shimamura & Stephen Palmer, Aesthetic Science: Connecting Minds, Brains, and Experience, Oxford, 2013) have questioned whether this research truly adds anything to our understanding of art. The question is whether cognitive science and aesthetics is looking for art in the right place. Can an investigation of our ordinary psychological engagement with artworks reveal what makes this category of artifacts unique and so deeply meaningful to us? Bill Seeley will discuss how recent research in visual stylometry and machine learning has been used to help answer this skeptical concern.
Date: 03/06/2017
Conference Name: Franklin W. Olin College of Engineering, Needham, Massachusetts.

A Question of Artistic Style: Digital Image Analysis and the Classification of Paintings (Conference Paper/Presentation)
Title: A Question of Artistic Style: Digital Image Analysis and the Classification of Paintings
Author: William P. Seeley and Catherine A. Buell
Abstract: Recent interdisciplinary research in visual stylometry employs digital image analysis algorithms to study the image features and statistics that underwrite our experience and understanding of artworks. This research brings philosophers, psychologists, computer scientists, and art historians together to explore the formal image qualities that define artistic style. We introduce the field of visual stylometry, discuss it's implications for our understanding of both the nature of categories of art and the role artistic style plays in our engagement with artworks. We then discuss the results of our research employing entropy analyses and discrete tonal analyses to classify paintings by school (Impressionism/Hudson River School) and medium/technique (Wyeth/egg-tempura/watercolor).
Date: 04/07/2017
Conference Name: Pacific Division Meeting of the American Society for Aesthetics, Asilomar Conference Grounds, Pacific Grove, California.

Making WAIVS! Workshop (Public Lecture or Presentation)
Title: Making WAIVS! Workshop
Abstract: Making WAIVS is a one-day workshop designed to introduce philosophers, art historians, art educators, and museum administrators to the potential that digital image analysis tools might hold for their practices. WAIVS or Workflows for the Analysis of Images in Visual Stylometry is designed to support the use of digital image analysis techniques by researchers in the humanities with little to no computer science background. The workshop will introduce participants to the field of visual stylometry through a series of hands-on exercises using WAIVS. Visual stylometry is a new field that lies at the intersection of art, cognitive science, computer science, and the digital humanities. Researchers in this field use digital image analysis tools to study the image features and image statistics of artistic style. This information can be used to classify paintings by era, school, individual artist within a school, technique, or period within the body of work of an individual artist, and to provide insight into the unique ways that different painters communicate through the subject matter or medium. Speakers at the workshop include Fitchburg Art Museum Director or Curator, Daniel J. Graham (Psychology of Art, Hobart & William Smith College), Yolanda Gil (Computer Science, University of Southern California), and Charlene Villasenor Black (Art History, UCLA). More information can be found at waivs.org. * Making WAIVS is funded by an NEH Digital Humanities Startup Grant (Award HD-248360-16), an American Society for Aesthetics Major Projects Initiative Grant, and the generous support of the Fitchburg Art Museum.
Author: Catherine Buell
Author: Ricky Sethi
Author: William Seeley
Author: Yolanda Gil
Author: Daniel Graham
Author: Charlene Villasenor-Black
Date: 05/19/17
Location: Fitchburg Art Museum
Primary URL: http://fam.waivs.org
Primary URL Description: Workshop Website

---