3
$\begingroup$

The documentation for AudioSpectralTransformation says:

AudioSpectralTransformation computes the short-time Fourier transform of audio, maps every value at position f[{time,freq}] to {time,freq}, and computes the inverse using the overlap-add method.

But looking at the 1st example of this function under Scope, I found it difficult to understand. Its behavior seems opposite to my thinking. It seems like it does the transform from new short-time Fourier transform to old short-time Fourier transform.

I used to think the frequency would get smaller, but just the opposite happens.

a = ExampleData[{"Audio", "Apollo11SmallStep"}, "Audio"]
f[{time_, frequency_}] := {time, frequency*.5}
AudioSpectralTransformation[f, a]

Column[Spectrogram[#, ImageSize -> Medium] & /@ {a, %}]

enter image description here

When looking at a more complicated example, such as the following, I can not figure out how to do the transform.

a = ExampleData[{"Audio", "Apollo11SmallStep"}, "Audio"]
f[{time_, frequency_}] := {time*1.2, frequency + time*frequency^.5};
AudioSpectralTransformation[f, a]
Spectrogram[%, ImageSize -> Medium]

enter image description here

I find there is a ImageTransformation. Is this like AudioSpectralTransformation?

Would it be correct to think that AudioSpectralTransformation is to AudioSpectralMap what ImageTransformation is to ImageApply?

enter image description here

$\endgroup$
3
$\begingroup$

The important part here is: maps every value at position f[{time, freq}] to {time, freq}.

In your first example, frequency is doubled because the value from {t, f/2} gets mapped to {t, f}.

If you want to consider it from the perspective of: "where does the function send the value from the spectrogram of the original audio", you have to consider the inverse function.

I'll use the more complicated example to show what I mean.

For the sake of not confusing myself with f and freq, I have changed the function name to g.

aud = ExampleData[{"Audio", "Apollo11SmallStep"}, "Audio"]
dur = QuantityMagnitude[Duration[aud]]

We will define the function and it's inverse:

g[{t_, f_}] := {1.2 t, f + t Sqrt[f]}
gi[{t_, f_}] := {t/1.2, (-(t/2.4) + Sqrt[f + t^2/5.76])^2}

The result is the image of the original spectrogram under the inverse of the mapping function, so the vector for each value is:

vM[v_] := gi[v] - v

We can use that to predict what time and frequency in the result will have the value at a given time and frequency in the original:

mapping = VectorPlot[vM[{t, f}], {t, 0, dur}, {f, 0, 11000}, 
                    VectorScale -> {Automatic, 0.02}, PlotRange -> {{0, dur}, {0, 11000}}]

Then show the Spectrogram with our original aud:

spectrogram = Spectrogram[aud, ImageSize -> Medium]

Now, to visualize what we've done:

Show[spectrogram, mapping]

the original spectrogram with the mapping

This mapping should produce the result you see when you do:

Spectrogram[AudioSpectralTransformation[g, aud]]

Transformed spectrogram

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.