Audio Digital Signal Processing

Question

Resource and strategy request

I want to identify resources and strategies to implement a Mathematica based solution for audio digital signal processing (DSP) and digital to analog conversion (DAC).

Jump to section DAC & DSP for more specific Mathematica related discussion. Else more extensive background and descriptions of the specific application follows…

Matrix decoding

Matrix decoding in audio, decodes an audio signal with M channels into an audio signal with N channels (where N > M) for play back on a system with N speakers. The original audio signal usually gets generated using a matrix encoder, which transforms N-channel signals to M-channel signals.

Matrix encoding and decoding enables playing the same audio content on different systems. For example, a surround sound multichannel signal gets encoded into a stereo signal. One can play back the stereo signal on a stereo system (2 channels) to accommodate settings where a surround sound receiver does not exist, or decoded and played as surround (many channels) with appropriate surround equipment.

I’ve begun assembling components to create a modest home theater audio playback system. I have a long history in DIY audio. Component descriptions follow…

Distributed Mode Loudspeaker (DML)

Flat panel loudspeaker technology, which produces sound by inducing uniformly distributed vibration modes in panels through a special electro-acoustic exciter. Distributed mode loudspeakers function differently from most others, which typically produce sound by inducing pistonic motion in the diaphragm.

Exciters for distributed mode loudspeakers include, moving coil and piezoelectric devices, placed to correspond to the natural resonant model of a given panel.

I put together amazing DML speakers from parts I had tucked away in a closet. Other DIYers have had great success from inexpensive extruded poly styrene (EPS), canvas art panels, balsa wood, spruce, plexiglass, or aluminum. DMLs more like instruments then conventional speakers, like stuff that has high compressive strength, but also wobbles (think a guitar sound board).

Also of some interest, they generate the same sound output front and back. Unlike conventional speaker drivers (cones, planar, ribbon, or electrostatic in an open baffle, DML front and back output plays in phase. DMLs can truly sound like instruments/performers in the room with you and they’ve brought their recording venue with them.

You can find Lots of YouTube videos on DMLs exploring their physics and acoustical properties.

Left and right of the TV.

I’ll try something a little unusual with the DMLs, using a pair of exciters on each panel - one exciter delivering the left or right stereo channel, then the second delivering center channel information to the same panel.

Speaker manufacture, Magnepan, (which makes a different kind of panel speaker) has explored using a pair of their panels as this kind of dual center channel. It appears to work well especially when one doesn’t have a good place for a typical center speaker above or below the TV.

DMLs introduce interesting new possibilities in audio reproduction.

No surround channels or subwoofer for this installation. I want to explore what the DMLs will do in this application before even thinking about anything else.

Amplifiers

I have a pair of stereo Hypex N-Core Class D amp modules. I’ve used versions of these with other DMLs with great success.

Streaming

I stream from a single source, essentially a smart TV will connect by optical audio to something (e.g. a computer running a Mathematica program) to handle DSP (digital signal processing) & DAC (digital to analog conversion).

DSP & DAC

Here I can use some Mathematica advice.

I don’t want to buy an AV preamp/processor. I don’t see the sense in paying a premium for a lot of functionality I’ll never use.

More importantly, I want to do as much of the DSP and maybe DAC in software and it seems like Mathematica can enable me to do this.

I’ve got an extra computer to dedicate to this effort and a Mathematica license I can use. If I can do this in Mathematica I see nothing standing in the way of doing this in a CDF.

I may need some additional hardware, maybe a sound card. None of that worries me.

While Mathematic has lots of support for signal processing (see: Digital Filter Design), I haven’t found specific solutions/code for decoding the current Dolby Surround stuff critical for a successful implementation.

I can extract the stereo left and right channel information in a straight forward manner. Decoding the center channel information from a Dolby Atmos stream gets trickier.

For deeper background see: Dolby Pro Logic system here

I can pretty simply fabricate/synthesize center channel information from stereo left and right channels rather than decoding/extracting it from the stream. Serviceable, but decoding would certainly work better and also provide the basis to later decode channels beyond left - center - right.

I’ve looked around for surround decoder code in Mathematica, but haven’t found anything that looks like it actually decodes Dolby.

I do not underestimate the task. Doing DSP and DAC within a general purpose computer rather than in dedicated purpose-built hardware & firmware presents significant timing/coordination & buffering issues, let alone aligning things like speech with the video on the TV.

That said, DIYers have done this. If they can home brew this sort of thing on other platforms or by cobbling together bits from here and there, surely Mathematica can achieve an elegant solution.

Thoughts on strategy and resources to inform this project appreciated.

—————————-

MatLab resource

MatLab has a good page on this at: Surround Sound Matrix Encoding and Decoding

A Mathematica equivalent to the MatLab page would go a long way to giving me the direction I need.

Answer 1

I am interested in writing accept-worthy code which is specific to this question.

In the interim, here's some resources.

• This demonstrates generation of C code, and uses basic audio filtering as a toy example. create-standalone-executables

• You might be able to use OpenRead on a /dev object, and Read custom frames (length, delimiter etc.; perhaps Dolby has some custom format). ref/Read.html, ref/OpenRead.html. There are of course OpenWrite and Write.

• This might help with reading data from a stream How to wait for new input from file InputStream

• Keep in mind you can use Streams[] to list all open (I and O) streams while debugging

• It might be worth looking at guide/StreamMethods, though this would probably not be at all useful to begin with.

• Along the same lines, tutorial/StreamMethods shows how to do some basic stream processing, but not necessarily concurrent and linked IO.

I'd be curious to see if it's possible to get MMA to handle data quick enough and consistently enough. I also don't know the best way to handle buffer over/underflows

• You can monitor memory usage with MemoryInUse[], ref/MemoryInUse.html. It may be worth popping up a new front end each time you test stuff so you can monitor the memory of just the streams and processing, see CreateNotebook.

• If you decide to interface MMA with an external program, LinkCreate may be useful. I I've linked MMA with Java, and I've done direct sample generation audio with Java, though both projects were separate. ref/LinkCreate

• I'll just note that I can't find anyone using AudioCapture["Memory"] anywhere, but it may be useful.

• There is an example in ref/AudioStream to

Display the power spectrum of a signal while it is playing:

I modified it slightly and it worked to plot my microphone input just fine:

ClearAll[astream,spectrum]
astream = AudioStream[First@$AudioInputDevices];
spectrum[audio_] := 
  Periodogram[audio, 2000, PlotRange -> {{40, 20000}, {-80, 10}}, 
  ScalingFunctions -> {"Log", "dB"}, Frame -> True, Axes -> False]
Dynamic[spectrum[astream["CurrentAudio"]]]
astream@"Status"="Recording";
(*later*)
AudioStop@astream;

You might try to get input from the proper device, try to output to the proper device, and then try to shunt data between them.

I'm interested to hear what you make of/from this.