2
$\begingroup$

I'm looking to train a image classifier with a large number of examples and categories (30,000 & 100). I would then import a file of image URLs that need to be classified, and export the results. I'm able to get good results with smaller training sets (these are not real URLs):

In[1]=trainingset=Classify[{Import["www.url.com/asdf.jpg"]-> "category 1", 
    Import["www.url.com/qwer.jpg"]-> "category 2", 
    Import["www.url.com/1234.jpg"]-> "category 3",
    ET AL
    },Method->"RandomForest"]
Out[1]=ClassifierFuntion[Feature type: Image
    Number of examples: 70]

I would like to continue to use this Import["url"] method, and I would like to use a CSV because I have so much available training data. Is there a way to use the import function in a CSV for image URLs?

The following simple code is how I want it to work.

trainingset=Import["file1","CSV"]

imageclassifier[Classify[{trainingset},Method->"RandomForest"]]

needscategory=Import["file2","CSV"]

imageclassifier[needscategory]

Export["results","CSV"]

How should I structure my code/CSV in order to import the URL as an image?

UPDATE:

@JasonB excellent explanation, thank you for breaking it down!

I was able to replicate your results perfectly. I was able to import a CSV of images needing classification using this method as well by editing out the directions for a second column in the 'table' line:

In[5]=needscategory=Import["needscategory.csv"];
    needscategory=Table[Import[needscategory[[n,1]]],{n,1,Length[needscategory]}]
Out[6]={(penguin.jpg),(pickle.jpg)}

In[7]=imageclassifier[%6]
Out[7]={Penguin,Penguin}

For those looking trying to replicate this for their own projects, my working code look like this:

imglist=Import["test.csv"]
trainingset=Table[Import[imglist[[n,1]]]->imglist[[n,2]],{n,1,Length[imglist]}]
imageclassifier=Classify[trainingset,Method->"RandomForest"]

    ClassifierFunction[Method: RandomForest, Number of Classes: 2]

needscategory=Import["needscategory.csv"];
needscategory=Table[Import[needscategory[[n,1]]],{n,1,Length[needscategory]}]

The last step I want to take is to export my results as a CSV in the following format:

Column 1, Column 2
testimageURL1, result1
testimageURL2, result2

I am able to export the test image URLs separately:

Export["imageurls.csv", %6]

but I'm having trouble exporting the results:

Export["results.csv",%7]

Assuming it's possible to export results, how would I structure my Export line to combine both the original URLs and the results of the classifier?

$\endgroup$
  • 1
    $\begingroup$ So you already have the URLs in CSV format? Like on each line there is an URL and a catagory name, separated by a comma? It's hard to say how to best do this without an example, even if it only had a small number of URLs $\endgroup$ – Jason B. Apr 4 '16 at 13:30
  • $\begingroup$ Hard to format in this comment, but my training file looks like this: Column 1: Import["partstown.com/is-bin/intershop.static/WFS/Reedy-PartsTown-Site/…, Column 2: "Switches & Dials" $\endgroup$ – JonathanKolar Apr 4 '16 at 15:07
  • $\begingroup$ I just happened to reopen this and saw the new question, but it might not have been seen. Generally here, the questions should be separate and self-contained, so that if one question is answered and you have a followup, you could post it as a new question. Sometimes the new question might be too minor for a post of its own, and then the chatroom linked at the bottom is a good place for it. You might not have enough reputation to go to the chat, but I think you could get more by doing some of the things in the tour. $\endgroup$ – Jason B. Apr 6 '16 at 7:52
  • $\begingroup$ As to your current question, try to get the URLs for the images you need categorized into a list, like "needsCategories = {"http://.....", "http://......",.....}` Then you can Map the classifying function onto this list with just one line, classifiedCategories = imageclassifier /@ needsCategories. So now, needsCategories and classifiedCategories are both lists of strings. You can convert them into a table and export to CSV via Export["results.csv", Transpose[ { needsCategories , classifiedCategories} ]] $\endgroup$ – Jason B. Apr 6 '16 at 7:56
  • $\begingroup$ @JasonB I checked out the tour and will complete it, thank you for that recommendation. I will post my follow up as a new question. I wasn't quite able to map them onto one line. $\endgroup$ – JonathanKolar Apr 7 '16 at 20:29
4
$\begingroup$

Let's say this is your CSV file,

http://i.imgur.com/8giNhRb.jpg, penguins
http://i.imgur.com/ZQXCY7c.jpg, penguins
http://i.imgur.com/bQ6lqNU.jpg, penguins
http://i.imgur.com/lxzTgH9.jpg, penguins
http://i.imgur.com/eHdgXnt.jpg, penguins
http://i.imgur.com/Bmy54vW.jpg, penguins
http://i.imgur.com/MtzebxR.jpg, penguins
http://i.imgur.com/pkDhR1w.jpg, penguins
https://upload.wikimedia.org/wikipedia/commons/b/bb/Pickle.jpg, pickles
https://upload.wikimedia.org/wikipedia/commons/c/c7/Og%C3%B3rki_kiszone.jpg, pickles
https://upload.wikimedia.org/wikipedia/commons/8/81/Pickled_fruits_and_veggies.jpg, pickles

When I use Import to get this into Mathematica, it automatically parses it as an array, and the individual items are recognized as strings,

imglist = Import["test.csv"]

(* {{"http://i.imgur.com/8giNhRb.jpg", " penguins"}, 
    {"http://i.imgur.com/ZQXCY7c.jpg", " penguins"}, 
    {"http://i.imgur.com/bQ6lqNU.jpg", " penguins"}, 
     .....} *)

Now I want to convert this to the form that Classify wants, which I can do using Table,

trainingset = Table[Import[imglist[[n, 1]]] -> imglist[[n, 2]], {n, 1, 
  Length[imglist]}]

or, using a bit of shorthand this can be achieved using pure functions and Apply,

trainingset = Import[#1] -> #2 & @@@ imglist

Both of these give the following output,

enter image description here

Now you create the classifier function as you wish,

imageclassifier = Classify[trainingset, Method -> "RandomForest"]

enter image description here

However, you may need more images, as this training set did not achieve good results,

enter image description here

$\endgroup$
  • $\begingroup$ Thank you so much @JasonB! I would not have guessed that syntax. $\endgroup$ – JonathanKolar Apr 4 '16 at 16:19
  • $\begingroup$ @JonathanKolar Later I can break it down and explain what it all does, so check back tomorrow if you like $\endgroup$ – Jason B. Apr 4 '16 at 16:25
1
$\begingroup$

Note that your use of Table introduces substantial syntactical overhead. You can do better with Map.

Instead of

Table[
  Import[imglist[[n, 1]]] -> imglist[[n, 2]], 
  {n, 1, Length[imglist]}
]

you can

Import[#[[1]]] -> #[[2]] & /@ imglist

or even better

Import@#1 -> #2 & @@@ imglist
$\endgroup$

Your Answer

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Not the answer you're looking for? Browse other questions tagged or ask your own question.