# Music animations

I've been tempted to use Mathematica to create some artistic musical animations. That is, if I have a music file, I want to be able to edit, for example, its waveform animation. For example, get something like the following styles

which are some cool examples found in this website. I for instance really like this one (which I wonder how it was programmed)

However, I'm not too sure how to do this and where to start. I know Audio-type objects are manageable within Mathematica notebooks, but don't even know how to extract the waveform animation in the audio file, as seen here

Any ideas or suggestions?

• You may want to have a look at how the Audio player output is constructed. Have a look at ExampleData[{"Audio", "Water"}] // ToBoxes . It doesn't seem that configurable - so you might end up needing to build your own dynamic player with Graphics Commented Oct 24, 2021 at 0:03

The basic idea is the following:

1. Split the audio file in non-overlapping segments, you can use AudioPartition for that
2. Extract the power spectrum of the audio segment, you can use Periodogram, PeriodogramArray, Fourier, etc to do that. For simplicity I use Periodogram below with ScalingFunctions->"Absolute"
3. Style the resulting power spectrum values to taste
4. Repeat for each audio segment and animate

Here's a simple implementation of that:

stylePeriodogram[periodogram_] := Block[{values, cf},
cf = ResourceFunction["ColorBrewerData"]["Purples", "ColorFunction"];
values = Cases[periodogram, Line[a__] :> a, Infinity][[1, All, 2]];
PairedBarChart[values, values, AspectRatio -> 1/4,
BarOrigin -> "XAxis", BarSpacing -> {0, 0, 0}, Axes -> False,
ColorFunction -> cf, ChartStyle -> EdgeForm[None]]
]

waveForm[audioPartition_, window_ : 100] :=
stylePeriodogram[
Periodogram[audioPartition, window, window, HammingWindow, window,
ScalingFunctions -> "Absolute", PlotRange -> All]]

waveFormAnimationFrames[audio_, partition_ : 50/1000(*50 ms*), window_ : 100] :=
With[{audioPartitions = AudioPartition[audio, partition]},
waveForm[#, window] & /@ audioPartitions]


Let's see how this looks for the Apollo11 clip in ExampleData:

apolloAudio = ExampleData[{"Audio", "Apollo11SmallStep"}];
frames = waveFormAnimationFrames[apolloAudio];
Export["apollo11-small_step.gif", frames]


Notes:

• For single track audio files you should really only need to change the stylePeriodogram function, e.g. something like this could work for the circular waveform you showed
• To achieve the last effect you showed, you'd need an audio file with multiple tracks, and then simply superimpose graphics with some transparency for each track
• Wonderful. I think this gave me the basis to start with. Commented Oct 30, 2021 at 21:53

Here is a rough sketch. Not very beautiful - but you can make it with some color and flairs.

file = Import[ "https://file-examples-com.github.io/uploads/2017/11/file_example_\MP3_700KB.mp3"];

peaks = 4; (*No. of peaks you want in your visual output*)
sr = AudioSampleRate[file][[1]];
nt = Round[N@AudioLength[file][[1]]/sr];
nc = AudioChannels[file];
cols = {Blue, Red};
ListAnimate[Table[
dat = Table[pts = adat[[c, (i - 1) sr + 1 ;; i*sr]];
ms = Normalize[Mean[#^2] & /@ Partition[pts,Length[pts]/peaks]];
pts = Table[{x, Sum[ms[[i]] Exp[-(i - x)^2*8], {i, peaks}]},
{x, 0.0, peaks + 1.0, 0.01}], {c, nc}];
ListLinePlot[dat, Filling -> Axis, PlotRange -> {{0, peaks + 1}, {0, 1}},
Frame -> True, FrameTicks -> None, PlotStyle -> None, Background -> LightGray,
FillingStyle ->  MapIndexed[#2 -> Directive[Opacity[0.5], #] &, cols]]
, {i, nt - 1}]]