.. DO NOT EDIT.
.. THIS FILE WAS AUTOMATICALLY GENERATED BY SPHINX-GALLERY.
.. TO MAKE CHANGES, EDIT THE SOURCE PYTHON FILE:
.. "auto_examples/plot_chroma.py"
.. LINE NUMBERS ARE GIVEN BELOW.
.. only:: html
.. note::
:class: sphx-glr-download-link-note
:ref:`Go to the end <sphx_glr_download_auto_examples_plot_chroma.py>`
to download the full example code.
.. rst-class:: sphx-glr-example-title
.. _sphx_glr_auto_examples_plot_chroma.py:
===================================
Enhanced chroma and chroma variants
===================================
This notebook demonstrates a variety of techniques for enhancing chroma features and
also, introduces chroma variants implemented in librosa.
.. GENERATED FROM PYTHON SOURCE LINES 13-23
Enhanced chroma
^^^^^^^^^^^^^^^
Beyond the default parameter settings of librosa's chroma functions, we apply the following
enhancements:
1. Harmonic-percussive-residual source separation to eliminate transients.
2. Nearest-neighbor smoothing to eliminate passing tones and sparse noise. This is inspired by the
recurrence-based smoothing technique of
`Cho and Bello, 2011 <https://zenodo.org/record/1417557>`_.
3. Local median filtering to suppress remaining discontinuities.
.. GENERATED FROM PYTHON SOURCE LINES 24-36
.. code-block:: Python
# Code source: Brian McFee
# License: ISC
# sphinx_gallery_thumbnail_number = 5
import numpy as np
import scipy
import matplotlib.pyplot as plt
import librosa
.. GENERATED FROM PYTHON SOURCE LINES 37-39
We'll use a track that has harmonic, melodic, and percussive elements
Karissa Hobbs - Let's Go Fishin'
.. GENERATED FROM PYTHON SOURCE LINES 39-42
.. code-block:: Python
y, sr = librosa.load(librosa.ex('fishin'))
.. GENERATED FROM PYTHON SOURCE LINES 43-44
First, let's plot the original chroma
.. GENERATED FROM PYTHON SOURCE LINES 44-65
.. code-block:: Python
chroma_orig = librosa.feature.chroma_cqt(y=y, sr=sr)
# For display purposes, let's zoom in on a 15-second chunk from the middle of the song
idx = tuple([slice(None), slice(*list(librosa.time_to_frames([45, 60])))])
# And for comparison, we'll show the CQT matrix as well.
C = np.abs(librosa.cqt(y=y, sr=sr, bins_per_octave=12*3, n_bins=7*12*3))
fig, ax = plt.subplots(nrows=2, sharex=True)
img1 = librosa.display.specshow(librosa.amplitude_to_db(C, ref=np.max)[idx],
y_axis='cqt_note', x_axis='time', bins_per_octave=12*3,
ax=ax[0])
fig.colorbar(img1, ax=[ax[0]], format="%+2.f dB")
ax[0].label_outer()
img2 = librosa.display.specshow(chroma_orig[idx], y_axis='chroma', x_axis='time', ax=ax[1])
fig.colorbar(img2, ax=[ax[1]])
ax[1].set(ylabel='Default chroma')
.. image-sg:: /auto_examples/images/sphx_glr_plot_chroma_001.png
:alt: plot chroma
:srcset: /auto_examples/images/sphx_glr_plot_chroma_001.png
:class: sphx-glr-single-img
.. GENERATED FROM PYTHON SOURCE LINES 66-68
We can do better by isolating the harmonic component of the audio signal.
We'll use a large margin for separating harmonics from percussives:
.. GENERATED FROM PYTHON SOURCE LINES 68-81
.. code-block:: Python
y_harm = librosa.effects.harmonic(y=y, margin=8)
chroma_harm = librosa.feature.chroma_cqt(y=y_harm, sr=sr)
fig, ax = plt.subplots(nrows=2, sharex=True, sharey=True)
librosa.display.specshow(chroma_orig[idx], y_axis='chroma', x_axis='time', ax=ax[0])
ax[0].set(ylabel='Default chroma')
ax[0].label_outer()
librosa.display.specshow(chroma_harm[idx], y_axis='chroma', x_axis='time', ax=ax[1])
ax[1].set(ylabel='Harmonic')
.. image-sg:: /auto_examples/images/sphx_glr_plot_chroma_002.png
:alt: plot chroma
:srcset: /auto_examples/images/sphx_glr_plot_chroma_002.png
:class: sphx-glr-single-img
.. GENERATED FROM PYTHON SOURCE LINES 82-85
There's still some noise in there though.
We can clean it up using non-local filtering.
This effectively removes any sparse additive noise from the features.
.. GENERATED FROM PYTHON SOURCE LINES 85-100
.. code-block:: Python
chroma_filter = np.minimum(chroma_harm,
librosa.decompose.nn_filter(chroma_harm,
aggregate=np.median,
metric='cosine'))
fig, ax = plt.subplots(nrows=2, sharex=True, sharey=True)
librosa.display.specshow(chroma_harm[idx], y_axis='chroma', x_axis='time', ax=ax[0])
ax[0].set(ylabel='Harmonic')
ax[0].label_outer()
librosa.display.specshow(chroma_filter[idx], y_axis='chroma', x_axis='time', ax=ax[1])
ax[1].set(ylabel='Non-local')
.. image-sg:: /auto_examples/images/sphx_glr_plot_chroma_003.png
:alt: plot chroma
:srcset: /auto_examples/images/sphx_glr_plot_chroma_003.png
:class: sphx-glr-single-img
.. GENERATED FROM PYTHON SOURCE LINES 101-103
Local discontinuities and transients can be suppressed by
using a horizontal median filter.
.. GENERATED FROM PYTHON SOURCE LINES 103-115
.. code-block:: Python
chroma_smooth = scipy.ndimage.median_filter(chroma_filter, size=(1, 9))
fig, ax = plt.subplots(nrows=2, sharex=True, sharey=True)
librosa.display.specshow(chroma_filter[idx], y_axis='chroma', x_axis='time', ax=ax[0])
ax[0].set(ylabel='Non-local')
ax[0].label_outer()
librosa.display.specshow(chroma_smooth[idx], y_axis='chroma', x_axis='time', ax=ax[1])
ax[1].set(ylabel='Median-filtered')
.. image-sg:: /auto_examples/images/sphx_glr_plot_chroma_004.png
:alt: plot chroma
:srcset: /auto_examples/images/sphx_glr_plot_chroma_004.png
:class: sphx-glr-single-img
.. GENERATED FROM PYTHON SOURCE LINES 116-118
A final comparison between the CQT, original chromagram
and the result of our filtering.
.. GENERATED FROM PYTHON SOURCE LINES 118-131
.. code-block:: Python
fig, ax = plt.subplots(nrows=3, sharex=True)
librosa.display.specshow(librosa.amplitude_to_db(C, ref=np.max)[idx],
y_axis='cqt_note', x_axis='time',
bins_per_octave=12*3, ax=ax[0])
ax[0].set(ylabel='CQT')
ax[0].label_outer()
librosa.display.specshow(chroma_orig[idx], y_axis='chroma', x_axis='time', ax=ax[1])
ax[1].set(ylabel='Default chroma')
ax[1].label_outer()
librosa.display.specshow(chroma_smooth[idx], y_axis='chroma', x_axis='time', ax=ax[2])
ax[2].set(ylabel='Processed')
.. image-sg:: /auto_examples/images/sphx_glr_plot_chroma_005.png
:alt: plot chroma
:srcset: /auto_examples/images/sphx_glr_plot_chroma_005.png
:class: sphx-glr-single-img
.. GENERATED FROM PYTHON SOURCE LINES 132-139
Chroma variants
^^^^^^^^^^^^^^^
There are three chroma variants implemented in librosa: `chroma_stft`, `chroma_cqt`, and `chroma_cens`.
`chroma_stft` and `chroma_cqt` are two alternative ways of plotting chroma.
`chroma_stft` performs short-time fourier transform of an audio input and maps each STFT bin to chroma, while `chroma_cqt` uses constant-Q transform and maps each cq-bin to chroma.
A comparison between the STFT and the CQT methods for chromagram.
.. GENERATED FROM PYTHON SOURCE LINES 139-152
.. code-block:: Python
chromagram_stft = librosa.feature.chroma_stft(y=y, sr=sr)
chromagram_cqt = librosa.feature.chroma_cqt(y=y, sr=sr)
fig, ax = plt.subplots(nrows=2, sharex=True, sharey=True)
librosa.display.specshow(chromagram_stft[idx], y_axis='chroma', x_axis='time', ax=ax[0])
ax[0].set(ylabel='STFT')
ax[0].label_outer()
librosa.display.specshow(chromagram_cqt[idx], y_axis='chroma', x_axis='time', ax=ax[1])
ax[1].set(ylabel='CQT')
.. image-sg:: /auto_examples/images/sphx_glr_plot_chroma_006.png
:alt: plot chroma
:srcset: /auto_examples/images/sphx_glr_plot_chroma_006.png
:class: sphx-glr-single-img
.. GENERATED FROM PYTHON SOURCE LINES 153-167
CENS features (`chroma_cens`) are variants of chroma features introduced in
`Müller and Ewart, 2011 <https://zenodo.org/record/1416032>`_, in which
additional post processing steps are performed on the constant-Q chromagram to obtain features
that are invariant to dynamics and timbre.
Thus, the CENS features are useful for applications, such as audio matching and retrieval.
Following steps are additional processing done on the chromagram, and are implemented in `chroma_cens`:
1. L1-Normalization across each chroma vector
2. Quantization of the amplitudes based on "log-like" amplitude thresholds
3. Smoothing with sliding window (optional parameter)
4. Downsampling (not implemented)
A comparison between the original constant-Q chromagram and the CENS features.
.. GENERATED FROM PYTHON SOURCE LINES 167-176
.. code-block:: Python
chromagram_cens = librosa.feature.chroma_cens(y=y, sr=sr)
fig, ax = plt.subplots(nrows=2, sharex=True, sharey=True)
librosa.display.specshow(chromagram_cqt[idx], y_axis='chroma', x_axis='time', ax=ax[0])
ax[0].set(ylabel='Orig')
librosa.display.specshow(chromagram_cens[idx], y_axis='chroma', x_axis='time', ax=ax[1])
ax[1].set(ylabel='CENS')
.. image-sg:: /auto_examples/images/sphx_glr_plot_chroma_007.png
:alt: plot chroma
:srcset: /auto_examples/images/sphx_glr_plot_chroma_007.png
:class: sphx-glr-single-img
.. rst-class:: sphx-glr-timing
**Total running time of the script:** (0 minutes 14.725 seconds)
.. _sphx_glr_download_auto_examples_plot_chroma.py:
.. only:: html
.. container:: sphx-glr-footer sphx-glr-footer-example
.. container:: sphx-glr-download sphx-glr-download-jupyter
:download:`Download Jupyter notebook: plot_chroma.ipynb <plot_chroma.ipynb>`
.. container:: sphx-glr-download sphx-glr-download-python
:download:`Download Python source code: plot_chroma.py <plot_chroma.py>`
.. container:: sphx-glr-download sphx-glr-download-zip
:download:`Download zipped: plot_chroma.zip <plot_chroma.zip>`
.. only:: html
.. rst-class:: sphx-glr-signature
`Gallery generated by Sphinx-Gallery <https://sphinx-gallery.github.io>`_