3
$\begingroup$

As I see from documentation, Classify function supports Audio and Pictures as input, video is not supported unfortunately :(

Will it work if I try to use classify for mixed data input like

Classify[{Audio1, {Picture1.1, Picture1.2, Picture1.3}}->"Train arrival", {Audio2, {Picture2.1, Picture2.2, Picture2.3, Picture2.4}}->"Plane arrival"]

especially when pictures array is not the same size. here {Audio1, {Picture1.1, Picture1.2, Picture1.3}} is an extraction from video.

Or my expectation from classify is too high?

UPDATE1:

When I run my example to classify this data {Audio1, {Picture1.1, Picture1.2, Picture1.3}} I got something like this. looks like mathematica does not recognize it as audio.

enter image description here

I load sample data like this

{Import[NotebookDirectory[] <> 
   "SampleData/Audio_Pics_separated/01/*.wav"], 
 Import[NotebookDirectory[] <> 
   "SampleData/Audio_Pics_separated/01/*.jpg"]}

UPDATE2

When I run example from Alexey's answer I got many errors like this

enter image description here

The same happens if I load audio from wav file by import command.

Is it OK and I can ignore these errors or these errors means that my classifier function will not work at all?

UPDATE3: There are 2 option how to fix errors mentioned in UPDATE2.

  1. Sign in with Wolfram ID inside Mathematica
  2. Disconnect Mathematica from internet
Improve this question
$\endgroup$

1 Answer 1

Reset to default
4
$\begingroup$

Yes, it will work.

audio = {AudioGenerator["White", 3], AudioGenerator["Pink", 4]};
images = Table[RandomImage[1, ColorSpace -> "RGB"], {7}];

X = Riffle[audio, TakeList[images, {3, 4}]] // Partition[#, 2] &;
Y = {1, 2};

c = Classify[X -> Y, Method -> "NeuralNetwork"]

enter image description here

ClassifierInformation[c]

enter image description here

Options[c][[1]]["Model"]["Network"]

enter image description here

Improve this answer
$\endgroup$
8
  • $\begingroup$ Thanks a lot. when I run my example, it looks different from yours. I put screenshot to the question. looks like mathematica does not recognize it as audio, but get it as NominalVector. Also in your example, second input is nominalsequence, but I was expecting an image. P.S. Я тоже Алексей :) $\endgroup$
    – Zlelik
    Commented Nov 26, 2017 at 9:25
  • 1
    $\begingroup$ @Zlelik Nice to meet you :) 1. Try: Audio /@ Import[NotebookDirectory[] <> "SampleData/Audio_Pics_separated/01/*.wav"]. Additionally you can set feature types Classify[...,FeatureTypes->{"Audio",...}]. 2. I think it's okay that second feature has type NominalSequence because you have variable number of images for each example. $\endgroup$
    – Alexey Golyshev
    Commented Nov 26, 2017 at 17:45
  • $\begingroup$ ok, thanks. I will try. $\endgroup$
    – Zlelik
    Commented Nov 27, 2017 at 8:24
  • $\begingroup$ It is strange. If I try like this zzz = { {Audio /@ Import[NotebookDirectory[] <> "SampleData/Audio_Pics_separated/1/*.wav"], Import[NotebookDirectory[] <> "SampleData/Audio_Pics_separated/1/*.jpg"]} -> 1, {Audio /@ Import[NotebookDirectory[] <> "SampleData/Audio_Pics_separated/2/*.wav"], Import[NotebookDirectory[] <> "SampleData/Audio_Pics_separated/2/*.jpg"]} -> 2 } ccc = Classify[zzz] it works without errors but still tells me nominalvector, instead of audio. If I add FeatureTypes -> {"Audio", Automatic} It gives me error $\endgroup$
    – Zlelik
    Commented Nov 30, 2017 at 22:31
  • $\begingroup$ Error text: Classify::mlincfttp: Incompatible variable type (Audio) and variable value ({Audio[RawArray[Integer16,<2,96096>],SignedInteger16,Appearance->Automatic,AudioOutputDevice->Automatic,SampleRate->48000,SoundVolume->1]}). $\endgroup$
    – Zlelik
    Commented Nov 30, 2017 at 22:31

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.