# Confused about the usage of AudioSpectralTransformation

The documentation for AudioSpectralTransformation says:

AudioSpectralTransformation computes the short-time Fourier transform of audio, maps every value at position f[{time,freq}] to {time,freq}, and computes the inverse using the overlap-add method.

But looking at the 1st example of this function under Scope, I found it difficult to understand. Its behavior seems opposite to my thinking. It seems like it does the transform from new short-time Fourier transform to old short-time Fourier transform.

I used to think the frequency would get smaller, but just the opposite happens.

a = ExampleData[{"Audio", "Apollo11SmallStep"}, "Audio"]
f[{time_, frequency_}] := {time, frequency*.5}
AudioSpectralTransformation[f, a]

Column[Spectrogram[#, ImageSize -> Medium] & /@ {a, %}]


When looking at a more complicated example, such as the following, I can not figure out how to do the transform.

a = ExampleData[{"Audio", "Apollo11SmallStep"}, "Audio"]
f[{time_, frequency_}] := {time*1.2, frequency + time*frequency^.5};
AudioSpectralTransformation[f, a]
Spectrogram[%, ImageSize -> Medium]


I find there is a ImageTransformation. Is this like AudioSpectralTransformation?

Would it be correct to think that AudioSpectralTransformation is to AudioSpectralMap what ImageTransformation is to ImageApply?

The important part here is: maps every value at position f[{time, freq}] to {time, freq}.

In your first example, frequency is doubled because the value from {t, f/2} gets mapped to {t, f}.

If you want to consider it from the perspective of: "where does the function send the value from the spectrogram of the original audio", you have to consider the inverse function.

I'll use the more complicated example to show what I mean.

For the sake of not confusing myself with f and freq, I have changed the function name to g.

aud = ExampleData[{"Audio", "Apollo11SmallStep"}, "Audio"]
dur = QuantityMagnitude[Duration[aud]]


We will define the function and it's inverse:

g[{t_, f_}] := {1.2 t, f + t Sqrt[f]}
gi[{t_, f_}] := {t/1.2, (-(t/2.4) + Sqrt[f + t^2/5.76])^2}


The result is the image of the original spectrogram under the inverse of the mapping function, so the vector for each value is:

vM[v_] := gi[v] - v


We can use that to predict what time and frequency in the result will have the value at a given time and frequency in the original:

mapping = VectorPlot[vM[{t, f}], {t, 0, dur}, {f, 0, 11000},
VectorScale -> {Automatic, 0.02}, PlotRange -> {{0, dur}, {0, 11000}}]


Then show the Spectrogram with our original aud:

spectrogram = Spectrogram[aud, ImageSize -> Medium]


Now, to visualize what we've done:

Show[spectrogram, mapping]


This mapping should produce the result you see when you do:

Spectrogram[AudioSpectralTransformation[g, aud]]