utils/audio

Helper module for audio processing.

These functions and classes are only used internally, meaning an end-user shouldn’t need to access anything here.

• utils/audio
  • static
    • .read_audio(url, sampling_rate) ⇒ Promise.<Float32Array>
      • ~audio : Float32Array
    • .hanning(M) ⇒ Float64Array
    • .hamming(M) ⇒ Float64Array
    • .mel_filter_bank(num_frequency_bins, num_mel_filters, min_frequency, max_frequency, sampling_rate, [norm], [mel_scale], [triangularize_in_mel_space]) ⇒ Array.<Array<number>>
    • .spectrogram(waveform, window, frame_length, hop_length, options) ⇒ Promise.<Tensor>
    • .window_function(window_length, name, options) ⇒ Float64Array
  • inner
    • ~generalized_cosine_window(M, a_0) ⇒ Float64Array
    • ~hertz_to_mel(freq, [mel_scale]) ⇒ T
    • ~mel_to_hertz(mels, [mel_scale]) ⇒ T
    • ~_create_triangular_filter_bank(fft_freqs, filter_freqs) ⇒ Array.<Array<number>>
    • ~linspace(start, end, num) ⇒
    • ~padReflect(array, left, right) ⇒ T
    • ~_db_conversion_helper(spectrogram, factor, reference, min_value, db_range) ⇒ T
    • ~amplitude_to_db(spectrogram, [reference], [min_value], [db_range]) ⇒ T
    • ~power_to_db(spectrogram, [reference], [min_value], [db_range]) ⇒ T

---

utils/audio.read_audio(url, sampling_rate) ⇒ <code> Promise. < Float32Array > </code>

Helper function to read audio from a path/URL.

Kind: static method of utils/audio
Returns: Promise.<Float32Array> - The decoded audio as a Float32Array.

Param	Type	Description
url	string | URL	The path/URL to load the audio from.
sampling_rate	number	The sampling rate to use when decoding the audio.

---

read_audio~audio : <code> Float32Array </code>

Kind: inner property of read_audio

---

utils/audio.hanning(M) ⇒ <code> Float64Array </code>

Generates a Hanning window of length M. See https://numpy.org/doc/stable/reference/generated/numpy.hanning.html for more information.

Kind: static method of utils/audio
Returns: Float64Array - The generated Hanning window.

Param	Type	Description
M	number	The length of the Hanning window to generate.

---

utils/audio.hamming(M) ⇒ <code> Float64Array </code>

Generates a Hamming window of length M. See https://numpy.org/doc/stable/reference/generated/numpy.hamming.html for more information.

Kind: static method of utils/audio
Returns: Float64Array - The generated Hamming window.

Param	Type	Description
M	number	The length of the Hamming window to generate.

---

utils/audio.mel_filter_bank(num_frequency_bins, num_mel_filters, min_frequency, max_frequency, sampling_rate, [norm], [mel_scale], [triangularize_in_mel_space]) ⇒ <code> Array. < Array < number > > </code>

Creates a frequency bin conversion matrix used to obtain a mel spectrogram. This is called a mel filter bank, and various implementation exist, which differ in the number of filters, the shape of the filters, the way the filters are spaced, the bandwidth of the filters, and the manner in which the spectrum is warped. The goal of these features is to approximate the non-linear human perception of the variation in pitch with respect to the frequency.

Kind: static method of utils/audio
Returns: Array.<Array<number>> - Triangular filter bank matrix, which is a 2D array of shape (num_frequency_bins, num_mel_filters). This is a projection matrix to go from a spectrogram to a mel spectrogram.

Param	Type	Description
num_frequency_bins	number	Number of frequencies used to compute the spectrogram (should be the same as in stft).
num_mel_filters	number	Number of mel filters to generate.
min_frequency	number	Lowest frequency of interest in Hz.
max_frequency	number	Highest frequency of interest in Hz. This should not exceed sampling_rate / 2.
sampling_rate	number	Sample rate of the audio waveform.
[norm]	string	If "slaney", divide the triangular mel weights by the width of the mel band (area normalization).
[mel_scale]	string	The mel frequency scale to use, "htk" or "slaney".
[triangularize_in_mel_space]	boolean	If this option is enabled, the triangular filter is applied in mel space rather than frequency space. This should be set to true in order to get the same results as torchaudio when computing mel filters.

---

utils/audio.spectrogram(waveform, window, frame_length, hop_length, options) ⇒ <code> Promise. < Tensor > </code>

Calculates a spectrogram over one waveform using the Short-Time Fourier Transform.

This function can create the following kinds of spectrograms:

• amplitude spectrogram (power = 1.0)
• power spectrogram (power = 2.0)
• complex-valued spectrogram (power = None)
• log spectrogram (use log_mel argument)
• mel spectrogram (provide mel_filters)
• log-mel spectrogram (provide mel_filters and log_mel)

In this implementation, the window is assumed to be zero-padded to have the same size as the analysis frame. A padded window can be obtained from window_function(). The FFT input buffer may be larger than the analysis frame, typically the next power of two.

Kind: static method of utils/audio
Returns: Promise.<Tensor> - Spectrogram of shape (num_frequency_bins, length) (regular spectrogram) or shape (num_mel_filters, length) (mel spectrogram).

Param	Type	Default	Description
waveform	Float32Array | Float64Array		The input waveform of shape (length,). This must be a single real-valued, mono waveform.
window	Float32Array | Float64Array		The windowing function to apply of shape (frame_length,), including zero-padding if necessary. The actual window length may be shorter than frame_length, but we're assuming the array has already been zero-padded.
frame_length	number		The length of the analysis frames in samples (a.k.a., fft_length).
hop_length	number		The stride between successive analysis frames in samples.
options	Object
[options.fft_length]	number		The size of the FFT buffer in samples. This determines how many frequency bins the spectrogram will have. For optimal speed, this should be a power of two. If null, uses frame_length.
[options.power]	number	1.0	If 1.0, returns the amplitude spectrogram. If 2.0, returns the power spectrogram. If null, returns complex numbers.
[options.center]	boolean	true	Whether to pad the waveform so that frame t is centered around time t * hop_length. If false, frame t will start at time t * hop_length.
[options.pad_mode]	string	""reflect""	Padding mode used when center is true. Possible values are: "constant" (pad with zeros), "edge" (pad with edge values), "reflect" (pads with mirrored values).
[options.onesided]	boolean	true	If true, only computes the positive frequencies and returns a spectrogram containing fft_length // 2 + 1 frequency bins. If false, also computes the negative frequencies and returns fft_length frequency bins.
[options.preemphasis]	number		Coefficient for a low-pass filter that applies pre-emphasis before the DFT.
[options.mel_filters]	Array.<Array<number>>		The mel filter bank of shape (num_freq_bins, num_mel_filters). If supplied, applies this filter bank to create a mel spectrogram.
[options.mel_floor]	number	1e-10	Minimum value of mel frequency banks.
[options.log_mel]	string	null	How to convert the spectrogram to log scale. Possible options are: null (don't convert), "log" (take the natural logarithm) "log10" (take the base-10 logarithm), "dB" (convert to decibels). Can only be used when power is not null.
[options.reference]	number	1.0	Sets the input spectrogram value that corresponds to 0 dB. For example, use max(spectrogram)[0] to set the loudest part to 0 dB. Must be greater than zero.
[options.min_value]	number	1e-10	The spectrogram will be clipped to this minimum value before conversion to decibels, to avoid taking log(0). For a power spectrogram, the default of 1e-10 corresponds to a minimum of -100 dB. For an amplitude spectrogram, the value 1e-5 corresponds to -100 dB. Must be greater than zero.
[options.db_range]	number		Sets the maximum dynamic range in decibels. For example, if db_range = 80, the difference between the peak value and the smallest value will never be more than 80 dB. Must be greater than zero.
[options.remove_dc_offset]	boolean		Subtract mean from waveform on each frame, applied before pre-emphasis. This should be set to true in order to get the same results as torchaudio.compliance.kaldi.fbank when computing mel filters.
[options.max_num_frames]	number		If provided, limits the number of frames to compute to this value.
[options.min_num_frames]	number		If provided, ensures the number of frames to compute is at least this value.
[options.do_pad]	boolean	true	If true, pads the output spectrogram to have max_num_frames frames.
[options.transpose]	boolean	false	If true, the returned spectrogram will have shape (num_frames, num_frequency_bins/num_mel_filters). If false, the returned spectrogram will have shape (num_frequency_bins/num_mel_filters, num_frames).

---

utils/audio.window_function(window_length, name, options) ⇒ <code> Float64Array </code>

Returns an array containing the specified window.

Kind: static method of utils/audio
Returns: Float64Array - The window of shape (window_length,) or (frame_length,).

Param	Type	Default	Description
window_length	number		The length of the window in samples.
name	string		The name of the window function.
options	Object		Additional options.
[options.periodic]	boolean	true	Whether the window is periodic or symmetric.
[options.frame_length]	number		The length of the analysis frames in samples. Provide a value for frame_length if the window is smaller than the frame length, so that it will be zero-padded.
[options.center]	boolean	true	Whether to center the window inside the FFT buffer. Only used when frame_length is provided.

---

utils/audio~generalized_cosine_window(M, a_0) ⇒ <code> Float64Array </code>

Helper function to generate windows that are special cases of the generalized cosine window. See https://www.mathworks.com/help/signal/ug/generalized-cosine-windows.html for more information.

Kind: inner method of utils/audio
Returns: Float64Array - The generated window.

Param	Type	Description
M	number	Number of points in the output window. If zero or less, an empty array is returned.
a_0	number	Offset for the generalized cosine window.

---

utils/audio~hertz_to_mel(freq, [mel_scale]) ⇒ <code> T </code>

Kind: inner method of utils/audio

Param	Type	Default
freq	T
[mel_scale]	string	"htk"

---

utils/audio~mel_to_hertz(mels, [mel_scale]) ⇒ <code> T </code>

Kind: inner method of utils/audio

Param	Type	Default
mels	T
[mel_scale]	string	"htk"

---

utils/audio~_create_triangular_filter_bank(fft_freqs, filter_freqs) ⇒ <code> Array. < Array < number > > </code>

Creates a triangular filter bank.

Adapted from torchaudio and librosa.

Kind: inner method of utils/audio
Returns: Array.<Array<number>> - of shape (num_frequency_bins, num_mel_filters).

Param	Type	Description
fft_freqs	Float64Array	Discrete frequencies of the FFT bins in Hz, of shape (num_frequency_bins,).
filter_freqs	Float64Array	Center frequencies of the triangular filters to create, in Hz, of shape (num_mel_filters,).

---

utils/audio~linspace(start, end, num) ⇒

Return evenly spaced numbers over a specified interval.

Kind: inner method of utils/audio
Returns: num evenly spaced samples, calculated over the interval [start, stop].

Param	Type	Description
start	number	The starting value of the sequence.
end	number	The end value of the sequence.
num	number	Number of samples to generate.

---

utils/audio~padReflect(array, left, right) ⇒ <code> T </code>

Kind: inner method of utils/audio
Returns: T - The padded array.

Param	Type	Description
array	T	The array to pad.
left	number	The amount of padding to add to the left.
right	number	The amount of padding to add to the right.

---

utils/audio~_db_conversion_helper(spectrogram, factor, reference, min_value, db_range) ⇒ <code> T </code>

Helper function to compute amplitude_to_db and power_to_db.

Kind: inner method of utils/audio

Param	Type
spectrogram	T
factor	number
reference	number
min_value	number
db_range	number

---

utils/audio~amplitude_to_db(spectrogram, [reference], [min_value], [db_range]) ⇒ <code> T </code>

Converts an amplitude spectrogram to the decibel scale. This computes 20 * log10(spectrogram / reference), using basic logarithm properties for numerical stability. NOTE: Operates in-place.

The motivation behind applying the log function on the (mel) spectrogram is that humans do not hear loudness on a linear scale. Generally to double the perceived volume of a sound we need to put 8 times as much energy into it. This means that large variations in energy may not sound all that different if the sound is loud to begin with. This compression operation makes the (mel) spectrogram features match more closely what humans actually hear.

Kind: inner method of utils/audio
Returns: T - The modified spectrogram in decibels.

Param	Type	Default	Description
spectrogram	T		The input amplitude (mel) spectrogram.
[reference]	number	1.0	Sets the input spectrogram value that corresponds to 0 dB. For example, use np.max(spectrogram) to set the loudest part to 0 dB. Must be greater than zero.
[min_value]	number	1e-5	The spectrogram will be clipped to this minimum value before conversion to decibels, to avoid taking log(0). The default of 1e-5 corresponds to a minimum of -100 dB. Must be greater than zero.
[db_range]	number		Sets the maximum dynamic range in decibels. For example, if db_range = 80, the difference between the peak value and the smallest value will never be more than 80 dB. Must be greater than zero.

---

utils/audio~power_to_db(spectrogram, [reference], [min_value], [db_range]) ⇒ <code> T </code>

Converts a power spectrogram to the decibel scale. This computes 10 * log10(spectrogram / reference), using basic logarithm properties for numerical stability. NOTE: Operates in-place.

The motivation behind applying the log function on the (mel) spectrogram is that humans do not hear loudness on a linear scale. Generally to double the perceived volume of a sound we need to put 8 times as much energy into it. This means that large variations in energy may not sound all that different if the sound is loud to begin with. This compression operation makes the (mel) spectrogram features match more closely what humans actually hear.

Based on the implementation of librosa.power_to_db.

Kind: inner method of utils/audio
Returns: T - The modified spectrogram in decibels.

Param	Type	Default	Description
spectrogram	T		The input power (mel) spectrogram. Note that a power spectrogram has the amplitudes squared!
[reference]	number	1.0	Sets the input spectrogram value that corresponds to 0 dB. For example, use np.max(spectrogram) to set the loudest part to 0 dB. Must be greater than zero.
[min_value]	number	1e-10	The spectrogram will be clipped to this minimum value before conversion to decibels, to avoid taking log(0). The default of 1e-10 corresponds to a minimum of -100 dB. Must be greater than zero.
[db_range]	number		Sets the maximum dynamic range in decibels. For example, if db_range = 80, the difference between the peak value and the smallest value will never be more than 80 dB. Must be greater than zero.

---

< > Update on GitHub