"Specmurt Anasylis" Visualization and MIDI-Conversion Samples

by Shigeki Sagayama and Hirokazu Kameoka, The University of Tokyo.

Specmurt Anasylis

"Specmurt Anasylis" is an unique method invented by Prof. S. Sagayama in 2003 that enables "pitch likelihood distribution visualization" unlike the standard "pitch estimation" from a single channel multi-pitch audio signal. It is a frame-wise signal processing method like short-time spectrum analysis. Opposed to "Cepstrum Alanysis" which is the (inverse) Fourier transform of short-time power spectrum with log-scaled magnitude, proposed by Tukey et al. in 1964, we defined "Specmurt Anasylis" as the (inverse) Fourier transform of short-time power spectrum with log-scaled frequency. Division of the power spectrum by an assumed "common harmonic structure" in the specmurt domain gives the fundamental frequency distribution. In the following sample data, we used an iterative algorithm, proposed by H. Kameoka and S. Saito in 2004, that chooses proper characteristics of the "common harmonic structure" [Kameoka2004MUS08] followed by an HMM to trace multiple pitch contours for converting from the output distribution to MIDI data, though this improvement is not mentioned in the SAPA2004 paper.


Original short-time spectrum of mixture of violin sounds C4 and E4	Specmurt Anasylis with a 1/f charactristics as the common harmonic structure	Specmurt Anasylis with iterative estimation of the common harmonic structure

Keywords: multipitch, overtones, harmonics, automatic transcription, multitone analysis, fundamental frequencies, spectrum analysis

Bibliography

This idea and preliminary results were first published in Japanese [Takahashi2003MUS12] and in English [Sagayama2004SAPA10]. Iterative estimation of the optimal common harmonic structure was first included in [Kameoka2004MUS08] written in Japanese. Its English version is under preparation.

[Sagayama2004SAPA]
Shigeki Sagayama, Keigo Takahashi, Hirokazu Kameoka and Takuya Nishimoto, ``Specmurt Anasylis: A Piano-Roll-Visualization of Polyphonic Music Signal by Deconvolution of Log-Frequency Spectrum,'' Proc. 2004 ISCA Tutorial and Research Workshop on Statistical and Perceptual Audio Processing (SAPA2004), (3 October 2004, Jeju, Korea), Oct. 2004. [PDF file (570kB)] [HTML document]
[Kameoka2004MUS08]
Hirokazu Kameoka, Shoichiro Saito, Takuya Nishimoto and Shigeki Sagayama, ``Recursive Estimation of Quasi-Optimal Common Harmonic Structure Pattern for Specmurt Anasylis: Piano-Roll-Display Visiualization and MIDI Conversion of Polyphonic Music Signal,'' Technical Report of IPSJ, 2004-MUS-56, pp.41-48, Aug. 2004 (in Japanese) [PDF file]
Awarded "The Best Presentation Award".
[Takahashi2003MUS12]
Keigo Takahashi, Takuya Nishimoto and Shigeki Sagayama, ``F0 Multi-Pitch Analysis Using Deconvolution of Log-Frequency Spectrum,'' Technical Report of IPSJ, 2003-MUS-53, pp. 61-66, Dec. 2003. (in Japanese) [PDF file]

A Sample of pitch visualization of audio data

The 5-th voice (flute) entering J. S. Bach's 6-voice Ricercare from "Musical Offering" (BWV1079) excerpted from the RWC Music Database.

Click here to enlarge.

score original spectrum "Specmurt Anasylis"
result Handicrafted MIDI
data for reference

Preliminary results of MIDI conversion from audio data

Music material excerpted from the RWC music database.
They are still preliminary as no parameter adjustment have been done yet.
The academic use of these sample data is granted if it is associated with a notification of "These data were prepared by H. Kameoka on June 7, 2004 for academic use".
Click the music title to listen to the original audio data; click the rightmost MIDI sound numbers to listen to the MIDI-converted results.

Title Composer/arranger Genre Instrument MIDI conversion
pitch accuracy(%) MIDI sound#
Nocturne No.2 in E flat, op.9-2 F. Chopin Classic Piano 80.4 #24, #46
For two (Guitar solo) H. Chubachi Jazz Guitar 76.9 #1, #4
Jive (Piano solo) M. Nakamura Jazz Piano 77.8 #10
Jive (Guitar solo) H. Chubachi Jazz Guitar 77.6 #10
Crescent Serenade (Guitar solo) S. Yamamoto Jazz Guitar 74.5 #10, #24
Lounge Away (Piano solo) T. Nagai Jazz Piano 78.4 -
Abyss (Guitar solo) H. Chubachi Jazz Guitar 72.0 -

[ Back to Lab Home ]