Automatic Music Performance Analysis and Comparison Toolkit (AMPACT)

FAIN: HD-228966-15

Ohio State University (Columbus, OH 43210-1349)
Johanna Catriona Devaney (Project Director: September 2014 to present)

The further development of a suite of analytical tools for music scholarship, with a particular focus on the development of a tool for analyzing polyphonic performances from musical scores.

This project proposes to develop a core technology for a suite of automatic software tools for quantitatively analyzing musical performances for which a corresponding musical score is available and an encoding format for storing the analyses, entitled the Automatic Music Performance and Comparison Toolkit (AMPACT). A musical performance can convey both the musicians' interpretation of the written score as well as emphasize, or even manipulate, the emotional content of the music through small variations in timing, dynamics, and tuning. The target audience for AMPACT is music scholars are who are interested in performing empirical analyses of recorded performances but who lack the technical skills or the time necessary to develop their own tools or implement existing algorithms. The proposed project will allow the researchers to develop an algorithm for analyzing polyphonic performances for which musical scores are available.

Associated Products

Automated Music Performance Analysis and Comparison Toolkit (Web Resource)
Title: Automated Music Performance Analysis and Comparison Toolkit
Author: Johanna Devaney
Author: Michael Mandel
Abstract: The Automated Music Performance Analysis and Comparison Toolkit for the MATLAB programming environment automatically analzyes performance data from recordings where a score is available By using the information available in the score about what notes are expected in the performance and the order in which they occur. AMPACT provides estimates of note onsets and offsets for tones with non-percussive onsets (e.g., vocalists) that are more robust than blind onset detection algorithms The alignment algorithm and the analysis portion of toolkit are be available in AMPACT Version 0.3. The analysis portion for monophonic audio includes the extraction of various performance parameters: inter-onset intervals between notes and tempo information; relative dynamic level between notes; mean frequency for each note and interval sizes in cents; vibrato rate and depth. For polyphonic audio, AMPACT produces estimates of onsets and offsets for all of the voices in notated simultaneities.
Year: 2015
Primary URL: https://ampact.tumblr.com/

Inter- versus intra-singer similarity and variation in vocal performances. (Article)
Title: Inter- versus intra-singer similarity and variation in vocal performances.
Author: Johanna Devaney
Abstract: Inter-performer deviations in pitch, timing, dynamics, and timbre can be used to model individuals’ music performance style provided that there is sufficient intra-performer similarity. If a performer’s variability in performance parameters exceeds the variability across performers it is hard to create a discriminative model between performers that can predict performer identity. This paper investigates the intra- and inter-performer consistency in singers trained in the western art music tradition using a range of pitch, timing, dynamics, and timbre descriptors. Specifically, it examines the utility of these descriptors in a singer-identity task completed both computationally by a support vector machine classifier and by human listeners and considers the broader implications for intra- and inter-performance similarity and variability in music performance modelling.
Year: 2016
Primary URL: https://doi.org/10.1080/09298215.2016.1205631
Access Model: Subscription
Format: Journal
Publisher: Taylor and Francis

Score-informed estimation of performance parameters (Conference Paper/Presentation)
Title: Score-informed estimation of performance parameters
Author: Johanna Devaney
Author: Michael Mandel
Abstract: A musical performance can convey both the musicians’ interpretation of the written score as well as emphasize, or even manipulate, the emotional content of the music through small variations in timing, dynamics, and tuning. This paper describes the latest developments in a suite of automatic software tools for quantitatively analyzing musical performances for which a corresponding musical score is available, entitled the Automatic Music Performance Analysis and Comparison Toolkit (AMPACT). AMPACT uses a score-informed approach to estimate timing, pitch, and loudness parameters from both monophonic and, now, polyphonic audio. Robust extraction of higher-level timing, pitch, and loudness performance descriptors requires precise frame-level estimation of note onsets and offsets, fundamental frequency, and power. This paper describes the score-informed approaches implemented in AMPACT for this frame-level estimation in polyphonic audio.
Date: 8/11/2016
Primary URL: https://s18798.pcdn.co/ismir2016/wp-content/uploads/sites/2294/2016/08/devaney-score.pdf
Primary URL Description: Proceedings of the Late-Breaking Demo Session, ISMIR 2016
Conference Name: International Society for Music Information Retrieval Conference

Representing and Linking Music Performance Data with Score Information (Conference Paper/Presentation)
Title: Representing and Linking Music Performance Data with Score Information
Author: Johanna Devaney
Author: H. Léveillé Gauvin
Abstract: This paper argues for the need to develop a representation for music performance data that is linked with corresponding score information at the note, beat, and measure levels. Building on the results of a survey of music scholars about their music performance data encoding needs, we propose best-practices for encoding perceptually relevant descriptors of the timing, pitch, loudness, and timbral aspects of performance. We are specifically interested in using descriptors that are sufficiently generalized that multiple performances of the same piece can be directly compared with one another. This paper also proposes a specific representation for encoding performance data and presents prototypes of this representation in both Humdrum and Music Encoding Initiative (MEI) formats.
Date: 8/12/2016
Primary URL: https://dl.acm.org/citation.cfm?doid=2970044.2970052
Primary URL Description: Proceedings of the 3rd International workshop on Digital Libraries for Musicology
Conference Name: 3rd International Workshop on Digital Libraries for Musicology

An evaluation of score-informed methods for estimating fundamental frequency and power from polyphonic audio (Conference Paper/Presentation)
Title: An evaluation of score-informed methods for estimating fundamental frequency and power from polyphonic audio
Author: Johanna Devaney
Author: Michael Mandel
Abstract: Robust extraction of performance data from polyphonic musical performances requires precise frame-level estimation of fundamental frequency (f0) and power. This paper evaluates a new score-guided approach to f0 and power estimation in polyphonic audio and compares the use of four different input features: the central bin frequencies of the spectrogram, the instantaneous frequency, and two variants of a high resolution spectral analysis. These four features were evaluated on four-part multi-track ensemble recordings, consisting of either four vocalists or bassoon, clarinet, saxophone, and violin (the Bach10 data set) created from polyphonic mixes of the monophonic tracks both with and without artificial reverberation. Score information was used to identify time-frequency regions of interest in the polyphonic mixes for each note in a corresponding aligned score, from which f0 and power estimates were made. The approach was able to recover ground truth f0 within 20 cents on average in reverberation and power within 5 dB for anechoic mixtures, but only within 10 dB for reverberant.
Date: 3/9/2017
Primary URL: https://ieeexplore.ieee.org/document/7952142
Conference Name: International Conference on Acoustics, Speech, and Signal Processing (ICASSP) 2017

Encoding music performance data in Humdrum and MEI (Article)
Title: Encoding music performance data in Humdrum and MEI
Author: Johanna Devaney
Author: Hubert Léveillé Gauvin
Abstract: This paper proposes extensions to two existing music encoding formats, Humdrum and Music Encoding Initiative (MEI), in order to facilitate linking music performance data with corresponding score information. We began by surveying music scholars about their needs for encoding timing, loudness, pitch, and timbral performance data. We used the results of this survey to design and implement new spines in Humdrum syntax to encode summary descriptors at note, beat, and measure levels and new attributes in the MEI format to encode both note-wise summaries and continuous data. These extensions allow for multiple performances of the same piece to be directly compared with one another, facilitating both humanistic and computational study of recorded musical performances.
Year: 2017
Primary URL: https://link.springer.com/article/10.1007%2Fs00799-017-0229-3
Access Model: Subscription
Format: Journal
Periodical Title: International Journal on Digital Libraries
Publisher: Springer

---