3
$\begingroup$

As I see from documentation, Classify function supports Audio and Pictures as input, video is not supported unfortunately :(

Will it work if I try to use classify for mixed data input like

Classify[{Audio1, {Picture1.1, Picture1.2, Picture1.3}}->"Train arrival", {Audio2, {Picture2.1, Picture2.2, Picture2.3, Picture2.4}}->"Plane arrival"]

especially when pictures array is not the same size. here {Audio1, {Picture1.1, Picture1.2, Picture1.3}} is an extraction from video.

Or my expectation from classify is too high?

UPDATE1:

When I run my example to classify this data {Audio1, {Picture1.1, Picture1.2, Picture1.3}} I got something like this. looks like mathematica does not recognize it as audio.

enter image description here

I load sample data like this

{Import[NotebookDirectory[] <> 
   "SampleData/Audio_Pics_separated/01/*.wav"], 
 Import[NotebookDirectory[] <> 
   "SampleData/Audio_Pics_separated/01/*.jpg"]}

UPDATE2

When I run example from Alexey's answer I got many errors like this

enter image description here

The same happens if I load audio from wav file by import command.

Is it OK and I can ignore these errors or these errors means that my classifier function will not work at all?

UPDATE3: There are 2 option how to fix errors mentioned in UPDATE2.

  1. Sign in with Wolfram ID inside Mathematica
  2. Disconnect Mathematica from internet
$\endgroup$
4
$\begingroup$

Yes, it will work.

audio = {AudioGenerator["White", 3], AudioGenerator["Pink", 4]};
images = Table[RandomImage[1, ColorSpace -> "RGB"], {7}];

X = Riffle[audio, TakeList[images, {3, 4}]] // Partition[#, 2] &;
Y = {1, 2};

c = Classify[X -> Y, Method -> "NeuralNetwork"]

enter image description here

ClassifierInformation[c]

enter image description here

Options[c][[1]]["Model"]["Network"]

enter image description here

$\endgroup$
  • $\begingroup$ Thanks a lot. when I run my example, it looks different from yours. I put screenshot to the question. looks like mathematica does not recognize it as audio, but get it as NominalVector. Also in your example, second input is nominalsequence, but I was expecting an image. P.S. Я тоже Алексей :) $\endgroup$ – Zlelik Nov 26 '17 at 9:25
  • 1
    $\begingroup$ @Zlelik Nice to meet you :) 1. Try: Audio /@ Import[NotebookDirectory[] <> "SampleData/Audio_Pics_separated/01/*.wav"]. Additionally you can set feature types Classify[...,FeatureTypes->{"Audio",...}]. 2. I think it's okay that second feature has type NominalSequence because you have variable number of images for each example. $\endgroup$ – Alexey Golyshev Nov 26 '17 at 17:45
  • $\begingroup$ ok, thanks. I will try. $\endgroup$ – Zlelik Nov 27 '17 at 8:24
  • $\begingroup$ It is strange. If I try like this zzz = { {Audio /@ Import[NotebookDirectory[] <> "SampleData/Audio_Pics_separated/1/*.wav"], Import[NotebookDirectory[] <> "SampleData/Audio_Pics_separated/1/*.jpg"]} -> 1, {Audio /@ Import[NotebookDirectory[] <> "SampleData/Audio_Pics_separated/2/*.wav"], Import[NotebookDirectory[] <> "SampleData/Audio_Pics_separated/2/*.jpg"]} -> 2 } ccc = Classify[zzz] it works without errors but still tells me nominalvector, instead of audio. If I add FeatureTypes -> {"Audio", Automatic} It gives me error $\endgroup$ – Zlelik Nov 30 '17 at 22:31
  • $\begingroup$ Error text: Classify::mlincfttp: Incompatible variable type (Audio) and variable value ({Audio[RawArray[Integer16,<2,96096>],SignedInteger16,Appearance->Automatic,AudioOutputDevice->Automatic,SampleRate->48000,SoundVolume->1]}). $\endgroup$ – Zlelik Nov 30 '17 at 22:31

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.