6
$\begingroup$

I've been tempted to use Mathematica to create some artistic musical animations. That is, if I have a music file, I want to be able to edit, for example, its waveform animation. For example, get something like the following styles

enter image description here

which are some cool examples found in this website. I for instance really like this one (which I wonder how it was programmed)

enter image description here

However, I'm not too sure how to do this and where to start. I know Audio-type objects are manageable within Mathematica notebooks, but don't even know how to extract the waveform animation in the audio file, as seen here

enter image description here

Any ideas or suggestions?

$\endgroup$
1
  • 2
    $\begingroup$ You may want to have a look at how the Audio player output is constructed. Have a look at ExampleData[{"Audio", "Water"}] // ToBoxes . It doesn't seem that configurable - so you might end up needing to build your own dynamic player with Graphics $\endgroup$
    – flinty
    Oct 24, 2021 at 0:03

2 Answers 2

6
+200
$\begingroup$

The basic idea is the following:

  1. Split the audio file in non-overlapping segments, you can use AudioPartition for that
  2. Extract the power spectrum of the audio segment, you can use Periodogram, PeriodogramArray, Fourier, etc to do that. For simplicity I use Periodogram below with ScalingFunctions->"Absolute"
  3. Style the resulting power spectrum values to taste
  4. Repeat for each audio segment and animate

Here's a simple implementation of that:

stylePeriodogram[periodogram_] := Block[{values, cf},
  cf = ResourceFunction["ColorBrewerData"]["Purples", "ColorFunction"];
  values = Cases[periodogram, Line[a__] :> a, Infinity][[1, All, 2]];
  PairedBarChart[values, values, AspectRatio -> 1/4, 
   BarOrigin -> "XAxis", BarSpacing -> {0, 0, 0}, Axes -> False, 
   ColorFunction -> cf, ChartStyle -> EdgeForm[None]]
  ]

waveForm[audioPartition_, window_ : 100] := 
 stylePeriodogram[
  Periodogram[audioPartition, window, window, HammingWindow, window, 
   ScalingFunctions -> "Absolute", PlotRange -> All]]

waveFormAnimationFrames[audio_, partition_ : 50/1000(*50 ms*), window_ : 100] := 
 With[{audioPartitions = AudioPartition[audio, partition]},
  waveForm[#, window] & /@ audioPartitions]

Let's see how this looks for the Apollo11 clip in ExampleData:

apolloAudio = ExampleData[{"Audio", "Apollo11SmallStep"}];
frames = waveFormAnimationFrames[apolloAudio];
Export["apollo11-small_step.gif", frames]

enter image description here

Notes:

  • For single track audio files you should really only need to change the stylePeriodogram function, e.g. something like this could work for the circular waveform you showed
  • To achieve the last effect you showed, you'd need an audio file with multiple tracks, and then simply superimpose graphics with some transparency for each track
$\endgroup$
1
  • 1
    $\begingroup$ Wonderful. I think this gave me the basis to start with. $\endgroup$
    – sam wolfe
    Oct 30, 2021 at 21:53
2
$\begingroup$

Here is a rough sketch. Not very beautiful - but you can make it with some color and flairs.

file = Import[ "https://file-examples-com.github.io/uploads/2017/11/file_example_\MP3_700KB.mp3"];

peaks = 4; (*No. of peaks you want in your visual output*)
adat = AudioData[file];
sr = AudioSampleRate[file][[1]];
nt = Round[N@AudioLength[file][[1]]/sr];
nc = AudioChannels[file];
cols = {Blue, Red};
ListAnimate[Table[
dat = Table[pts = adat[[c, (i - 1) sr + 1 ;; i*sr]];
ms = Normalize[Mean[#^2] & /@ Partition[pts,Length[pts]/peaks]];
pts = Table[{x, Sum[ms[[i]] Exp[-(i - x)^2*8], {i, peaks}]}, 
      {x, 0.0, peaks + 1.0, 0.01}], {c, nc}];
ListLinePlot[dat, Filling -> Axis, PlotRange -> {{0, peaks + 1}, {0, 1}}, 
Frame -> True, FrameTicks -> None, PlotStyle -> None, Background -> LightGray, 
FillingStyle ->  MapIndexed[#2 -> Directive[Opacity[0.5], #] &, cols]]
, {i, nt - 1}]]

enter image description here

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.