librosa.decompose.nn_filter

librosa.decompose.nn_filter(S, *, rec=None, aggregate=None, axis=- 1, **kwargs)

Filtering by nearest-neighbors.

Each data point (e.g, spectrogram column) is replaced by aggregating its nearest neighbors in feature space.

This can be useful for de-noising a spectrogram or feature matrix.

The non-local means method 1 can be recovered by providing a weighted recurrence matrix as input and specifying aggregate=np.average.

Similarly, setting aggregate=np.median produces sparse de-noising as in REPET-SIM 2.

1

Buades, A., Coll, B., & Morel, J. M. (2005, June). A non-local algorithm for image denoising. In Computer Vision and Pattern Recognition, 2005. CVPR 2005. IEEE Computer Society Conference on (Vol. 2, pp. 60-65). IEEE.

2

Rafii, Z., & Pardo, B. (2012, October). “Music/Voice Separation Using the Similarity Matrix.” International Society for Music Information Retrieval Conference, 2012.

Parameters

Snp.ndarray

The input data (spectrogram) to filter. Multi-channel is supported.

rec(optional) scipy.sparse.spmatrix or np.ndarray

Optionally, a pre-computed nearest-neighbor matrix as provided by librosa.segment.recurrence_matrix

aggregatefunction

aggregation function (default: np.mean)

If aggregate=np.average, then a weighted average is computed according to the (per-row) weights in rec.

For all other aggregation functions, all neighbors are treated equally.

axisint

The axis along which to filter (by default, columns)

**kwargs

Additional keyword arguments provided to librosa.segment.recurrence_matrix if rec is not provided

Returns

S_filterednp.ndarray

The filtered data, with shape equivalent to the input S.

Raises

ParameterError

if rec is provided and its shape is incompatible with S.

See also

decompose

hpss

librosa.segment.recurrence_matrix

Notes

This function caches at level 30.

Examples

De-noise a chromagram by non-local median filtering. By default this would use euclidean distance to select neighbors, but this can be overridden directly by setting the metric parameter.

>>> y, sr = librosa.load(librosa.ex('brahms'),
...                      offset=30, duration=10)
>>> chroma = librosa.feature.chroma_cqt(y=y, sr=sr)
>>> chroma_med = librosa.decompose.nn_filter(chroma,
...                                          aggregate=np.median,
...                                          metric='cosine')

To use non-local means, provide an affinity matrix and aggregate=np.average.

>>> rec = librosa.segment.recurrence_matrix(chroma, mode='affinity',
...                                         metric='cosine', sparse=True)
>>> chroma_nlm = librosa.decompose.nn_filter(chroma, rec=rec,
...                                          aggregate=np.average)

>>> import matplotlib.pyplot as plt
>>> fig, ax = plt.subplots(nrows=5, sharex=True, sharey=True, figsize=(10, 10))
>>> librosa.display.specshow(chroma, y_axis='chroma', x_axis='time', ax=ax[0])
>>> ax[0].set(title='Unfiltered')
>>> ax[0].label_outer()
>>> librosa.display.specshow(chroma_med, y_axis='chroma', x_axis='time', ax=ax[1])
>>> ax[1].set(title='Median-filtered')
>>> ax[1].label_outer()
>>> imgc = librosa.display.specshow(chroma_nlm, y_axis='chroma', x_axis='time', ax=ax[2])
>>> ax[2].set(title='Non-local means')
>>> ax[2].label_outer()
>>> imgr1 = librosa.display.specshow(chroma - chroma_med,
...                          y_axis='chroma', x_axis='time', ax=ax[3])
>>> ax[3].set(title='Original - median')
>>> ax[3].label_outer()
>>> imgr2 = librosa.display.specshow(chroma - chroma_nlm,
...                          y_axis='chroma', x_axis='time', ax=ax[4])
>>> ax[4].label_outer()
>>> ax[4].set(title='Original - NLM')
>>> fig.colorbar(imgc, ax=ax[:3])
>>> fig.colorbar(imgr1, ax=[ax[3]])
>>> fig.colorbar(imgr2, ax=[ax[4]])