Music animations

Question

I've been tempted to use Mathematica to create some artistic musical animations. That is, if I have a music file, I want to be able to edit, for example, its waveform animation. For example, get something like the following styles

which are some cool examples found in this website. I for instance really like this one (which I wonder how it was programmed)

However, I'm not too sure how to do this and where to start. I know Audio-type objects are manageable within Mathematica notebooks, but don't even know how to extract the waveform animation in the audio file, as seen here

Any ideas or suggestions?

Answer 1

The basic idea is the following:

1. Split the audio file in non-overlapping segments, you can use AudioPartition for that
2. Extract the power spectrum of the audio segment, you can use Periodogram, PeriodogramArray, Fourier, etc to do that. For simplicity I use Periodogram below with ScalingFunctions->"Absolute"
3. Style the resulting power spectrum values to taste
4. Repeat for each audio segment and animate

Here's a simple implementation of that:

stylePeriodogram[periodogram_] := Block[{values, cf},
  cf = ResourceFunction["ColorBrewerData"]["Purples", "ColorFunction"];
  values = Cases[periodogram, Line[a__] :> a, Infinity][[1, All, 2]];
  PairedBarChart[values, values, AspectRatio -> 1/4, 
   BarOrigin -> "XAxis", BarSpacing -> {0, 0, 0}, Axes -> False, 
   ColorFunction -> cf, ChartStyle -> EdgeForm[None]]
  ]

waveForm[audioPartition_, window_ : 100] := 
 stylePeriodogram[
  Periodogram[audioPartition, window, window, HammingWindow, window, 
   ScalingFunctions -> "Absolute", PlotRange -> All]]

waveFormAnimationFrames[audio_, partition_ : 50/1000(*50 ms*), window_ : 100] := 
 With[{audioPartitions = AudioPartition[audio, partition]},
  waveForm[#, window] & /@ audioPartitions]

Let's see how this looks for the Apollo11 clip in ExampleData:

apolloAudio = ExampleData[{"Audio", "Apollo11SmallStep"}];
frames = waveFormAnimationFrames[apolloAudio];
Export["apollo11-small_step.gif", frames]

Notes:

• For single track audio files you should really only need to change the stylePeriodogram function, e.g. something like this could work for the circular waveform you showed
• To achieve the last effect you showed, you'd need an audio file with multiple tracks, and then simply superimpose graphics with some transparency for each track

Answer 2

Here is a rough sketch. Not very beautiful - but you can make it with some color and flairs.

file = Import[ "https://file-examples-com.github.io/uploads/2017/11/file_example_\MP3_700KB.mp3"];

peaks = 4; (*No. of peaks you want in your visual output*)
adat = AudioData[file];
sr = AudioSampleRate[file][[1]];
nt = Round[N@AudioLength[file][[1]]/sr];
nc = AudioChannels[file];
cols = {Blue, Red};
ListAnimate[Table[
dat = Table[pts = adat[[c, (i - 1) sr + 1 ;; i*sr]];
ms = Normalize[Mean[#^2] & /@ Partition[pts,Length[pts]/peaks]];
pts = Table[{x, Sum[ms[[i]] Exp[-(i - x)^2*8], {i, peaks}]}, 
      {x, 0.0, peaks + 1.0, 0.01}], {c, nc}];
ListLinePlot[dat, Filling -> Axis, PlotRange -> {{0, peaks + 1}, {0, 1}}, 
Frame -> True, FrameTicks -> None, PlotStyle -> None, Background -> LightGray, 
FillingStyle ->  MapIndexed[#2 -> Directive[Opacity[0.5], #] &, cols]]
, {i, nt - 1}]]