66
$\begingroup$

I see a low use of Mathematica in Kaggle competitions.

Why would one use the Wolfram Language versus R, Python, or Julia for machine learning?

Besides prettier plots and the Manipulate function, do we have something that is useful for ml that other languages are lacking?

$\endgroup$
  • 22
    $\begingroup$ To the closers: while I partly agree that the question is a bit broad, it seems to be an important one, and having some good answers for it is important. I also think that the decision of being too broad or not should depend on how much specialized information on a praticular topic is currently available on the site. For ML, there isn't yet much accumulated common wisdom on this site in the context of Mathematica. When we started the site, we had similarly broad questions on other topics (like, e.g., meta-programming), which seem to have proved highly appreciated despite that. $\endgroup$ – Leonid Shifrin Jun 23 '15 at 18:40
  • 4
    $\begingroup$ Its worth noting that a good proportion of the competitions on kaggle require Open source solutions thus ruling out MMA. $\endgroup$ – Gordon Coale Jun 24 '15 at 6:52
  • 2
    $\begingroup$ I think it is a good idea to give answers to these questions in the original post. 1. Why should we care about Kaggle competitions? 2. What type of machine learning problems are in Kaggle (supervised, unsupervised, etc.) 3. What type of usability is considered? Meaning, out of the box functionalities, and/or the ability to program novel algorithms. $\endgroup$ – Anton Antonov Jul 22 '18 at 13:19
  • 2
    $\begingroup$ @gwr See my answer/rant. $\endgroup$ – Anton Antonov Jan 11 at 15:54
  • 1
    $\begingroup$ @gwr As I understand it, Kaggle kernels can only be written in R or Python. That's why you won't see anything else. This is a choice that managers at Kaggle have made, it doesn't mean that R and Python are the best languages for machine learning. $\endgroup$ – C. E. Jan 11 at 18:25
54
$\begingroup$

I have worked in pattern classification and machine learning for decades, taught the subject in a number of elite academic departments, am writing the third edition of Pattern classification by Duda, Hart and Stork as well as its companion computer manual in Mathematica, and am an expert Mathematica programmer, a solid Matlab programmer, but very weak in R and Python and have no experience with Julia. As such, I feel I'm fairly well positioned to answer this question.

Here, as in most disciplines, the "best" language depends upon what you seek to accomplish. In pattern recognition and machine learning, the early stages are ones of experimentation and exploration--trying different algorithms, feature pre-processing, and such, as well as integrating different functions and visualizing preliminary results. One of the many great benefits of Mathematica is its seamless integration between different functionality, so it is easy to use statistical learning with graph-theoretic methods, and pre-processing of images, sound, financial data, etc., without the need to load libraries of special-purpose functions. I really like Mathematica's symbolics for statistical analyses, Probability, and so on. I'm not aware of any other language that comes close to its power and ease in such tasks.

On the other hand, a late stage of machine learning work can involve implementing algorithms on a massive scale, with large datasets, where careful control of data type, bit resolution, parallel threading and such may become more important. All these steps can be done in Mathematica, of course, but I haven't seen that it is superior to other languages in this realm. So at this moment, if I were to start a project in deep learning where the goal was a neural network with dozens of layers and hundreds of thousands of nodes ("neurons"), I'd likely turn to a language other than Mathematica. The folks at Wolfram Research are often anticipating needs, and that has led to their curated databases, free-form and semantic input, and so on, so perhaps they'll make some optimized deep learning code (running in the cloud?) that will make calling from a standard Mathematica session easy.

$\endgroup$
47
$\begingroup$

Mathematica doesn't have the depth of algorithm support that is present in R or Python. Julia has much more limited algorithm support but does exhibit a good turn of speed.

The few algorithms that Mathematica does support are not particularly well exposed for the type of tweaking needed to win Kaggle competitions.

Mathematica, as of version 10, supports the following classifiers: "LogisticRegression", "Markov", "NaiveBayes", "NearestNeighbors", "NeuralNetwork", "RandomForest", "SupportVectorMachine".

Whilst it does offer one ensemble method, RandomForest, it lacks both Bagging and any flavour of boosting, such as Adaboost. These latter general ensemble methods allow you to leverage the power of a base classifier, either built-in or, of greater utility when exploring performance improvements, of your own design. This limits significantly the potential for creating novel methods within the existing machine learning framework.

Mathematica's strengths lay with it's expressive, compact, language, strong visualisation capabilities and easy parallelization. These make it very efficient to explore new strategies and combinations of ML algorithms you may already have implemented.

A less often mentioned benefit is that use of Mathematica in a functional programming style tends to lead to fewer bugs in the final code.

$\endgroup$
  • 16
    $\begingroup$ Rephrasing this: Mma is rarely the specialized toolbox you are looking for - but if you are the guy trying to find out what that toolbox would contain and it's mathematical, it's a wonderful tool to explore those possibilities. $\endgroup$ – kirma Jun 23 '15 at 18:08
17
$\begingroup$

From my perspective, (I was the original developer for Evolved Analytics' DataModeler Mathematica add-on package, www.evolved-analytics.com), what Mathematica brings to the table is the semi-seamless integration of symbolics and numerics as well as the freeform programming. From the point of view of symbolic regression, things are possible which are very difficult from a procedural orientation.

We have built a number of different symbolic regression engines over the years — some at very low levels of machine code — and the Mathematica-based ones stack up extremely well in terms of modeling efficacy and efficiency.

Addtionally, although I have not tried the new neural net capabilities, Jonas Sjoberg's neural net package was also pretty slick because, after development, you could simply provide symbols to the developed model and get an expression which could be easily exported into production environments.

$\endgroup$
15
$\begingroup$

I actually failed miserably in a Kaggle contest using Mathematica Enterprise. I tested every single variation of Classify and Predict and even combinations of the two.

I also tested Microsoft ML Studio, Google Prediction API, IBM Watson, BigML and others. Amazon ML got me the highest score but they all failed miserably in comparison to the custom implementation of XgBoost in Python or R the Kaggle veterans use.

I am hoping XgBoost and new deep learning variations of the Neural Network algorithms are added when they launch the new Wolfram Data Science platform.

I read that running an evolutionary algorithm like:

NMinimize[f, vars, Method -> "DifferentialEvolution"] 

and

NMaximize[f, vars, Method -> "DifferentialEvolution"]

in combination with:

Classify[data, Method->"NeuralNetwork", PerformanceGoal->"Quality"]

can create similar high quality results. However, I haven't found any tutorials that show exactly how to combine the two.

$\endgroup$
  • $\begingroup$ I really hope Mathematica can progress a bit more in this subject... :-| $\endgroup$ – Rod Nov 16 '16 at 21:34
13
$\begingroup$

I would add its high degree of automation makes it a very easy introduction to ML - although I can't comment if it's more automated than the other languages on offer. The fact you can simply import a mixed set of columnar data and it will automatically choose suitable pre processing such as bag of words and tokenisation on each column and produce a credible result very quickly.

Add in automatic cross validation and dozens of post run analysis options via ClassifierInformation, ClassifierMeasurements and the Equivalent predictor functions and its a great introduction to ML.

As noted the typical list support of all MMA functions makes it easy to throw in multiple parallel runs with different parameters and sample sizes.

Downsides to hard to fine tune the input processing and its ease of use could encourage some people to run to far before they fully understand some of the common ML pitfalls.

This represents a ML beginner using it in MMA for the first time and gives a good display of its flexibilty (in my biased opinion ;)) Using a dataset of PredictorFunctions

A quick google suggests the above creation of a structured array/dataset/table (delete according to taste) of functions is non-trivial is most of the mentioned languages. Yet an MMA novice managed it first time out.

Some useful q&a

How to know the internal algorithms of functions like Predict or Classify?

How to do n-fold cross validation with Classify?

How to split a Dataset into training and testing for machine learning?

$\endgroup$
  • $\begingroup$ How do you "import a mixed set of columnar data" and get MMA to automatically choose preprocessing etc? $\endgroup$ – Ralph Dratman Jun 23 '15 at 23:02
  • $\begingroup$ @RalphDratman - If you look at my dataset of predictor functions question linked above, all you have to do is replace the test and training sets with an Import of a relevant file from csv or similar and you are there. If you want to post this as Question in itself I would try to do a model answer. I would just need some time to find/create a dataset. $\endgroup$ – Gordon Coale Jun 24 '15 at 6:43
12
$\begingroup$

The page "Summary of New Features in 10.1" introduces a few new functions that fall into the category of machine learning. Chief among these is the new ImageIdentify and related function ImageInstanceQ.

As this barely scratches the surface, I'm keeping my fingers crossed that this is a sign of things to come in future versions!

There are a couple of functions marked as [[Experimental]] in the documentation, namely:

  • DimensionReduce: projects vectors onto an approximating manifold in lower-dimensional space. Method options include "Principal ComponentsAnalysis", "LatentSemanticAnalysis" and "LowRankMatrixFactorization".
  • FindDistribution: finds a simple functional form to fit the distribution of data.

Finally, it is of course also worth mentioning the rest of the documentation on machine learning.

$\endgroup$
8
$\begingroup$

This answer is somewhat inspired (or may be triggered) by @gwr's comment:

(A) Somebody correct me, but as of now I only see R or Python Kernels for Kaggle (maybe Julia?). It seems that Python is the quasi standard for any serious ML.

combined with the OP question:

(B) Besides prettier plots and the Manipulate function, do we have something that is useful for ml that other languages are lacking?

(I think the comment (A) or its appropriate generalization should be part of the question.)

Is it Python or the underlying libraries?

(Here I am attempting to answer both (A) and (B).)

I use R a lot in industrial and healthcare related projects and when it comes to data processing and classification the performance speed and precision strength of my solutions come from:

  1. using the R-libraries for interfacing Spark and H2O. (Written in Scala and Java respectively.)

The other "R successful solution" components are:

  1. the interactive abilities in R, RStudio's Shiny, similar to Mathematica's Manipulate and related functions;

  2. RStudio's unit tests harness, testthat, similar to Mathematica's unit test functionalities; and

  3. R as a programming language, which shares a lot of features with Mathematica (and LISP.)

A further, relevant comparison of R and Mathematica over Deep Learning can be found here: "Deep learning examples".

Python as a language

(Mostly a rant triggered by (A).)

Python as a language is too simplistic and institution-centric. That is why it is embraced by managers and people who do not like or want to do programing that much. Python is a fine first language (for say Machine Learning students and junior Data Scientists) but if Python is your last language I simply cannot accept you seriously.

For example, people who are adept in using Spark say that Scala acts like a "force multiplier" to Spark. And Scala is a very functional language. (Pretty close to R and Mathematica, especially compared to Python and Java.)

Notebook kernels for Kaggle

A comment from @C.E. :

As I understand it, Kaggle kernels can only be written in R or Python. That's why you won't see anything else. This is a choice that managers at Kaggle have made, it doesn't mean that R and Python are the best languages for machine learning.

See the Kaggle kernels documentation for more details. The section "Notebooks" says:

Notebooks may be written in either R or Python.

Notebooks

(Rant extended...)

For me Jupyter notebooks -- one of the main ways Python is utilized in Data Science -- are too clunky for everyday use. Others have similar frustrations, see "5 reasons why jupyter notebooks suck" and "I don't like notebooks.".

R notebooks are based in Markdown and much more convenient. As far as I know both R notebooks and Jupyter notebooks are inspired from Mathematica's notebooks and try to emulate them (as much as they can.)

$\endgroup$
  • 1
    $\begingroup$ Great rant! I hope you respect me having Mathematica/WL as my pretty much only language so far? ;-) Looks like really serious people might turn to Scala (a dev-friend of mine has done so from Java) and the more analytics-only-minded people could have R (instead of Python) as a companion to Mathematica? $\endgroup$ – gwr Jan 11 at 16:41
  • 1
    $\begingroup$ @gwr "Looks like really serious people might turn to Scala (a dev-friend of mine has done so from Java)" -- I would like to mention that Scala is not a better Java (Kotlin is.) Scala is much more about having an intrinsic and solid functional programming language support based on Haskell ideas plus object-oriented support as a pragmatic and implementational ingredient. Scala though is too complicated paradigm-wise and in that regard reminds me of C++. It is fair to say that I would not use Scala now with it wasn't for Spark. $\endgroup$ – Anton Antonov Jan 11 at 17:00
  • 1
    $\begingroup$ @gwr "[...] more analytics-only-minded people could have R (instead of Python) as a companion to Mathematica?" -- R and Mathematica are close paradigm-wise (both descend from LISP.) R is/was much more popular for people with more solid academic roots (both degrees and pedigrees) and Python is/was more popular with people with engineering and computer science backgrounds and predispositions. (cont.) $\endgroup$ – Anton Antonov Jan 11 at 17:07
  • 1
    $\begingroup$ @gwr (cont.) This is rapidly changing though, largely because universities try to satisfy the hunger of companies and prospective students with Data Science degrees, and that means using least denominator languages and paradigms. Hence Python. One of the easiest thing to automate with Machine Learning is Data Science workflows. For this Functional Programming (FP) languages are much suited for, hence Scala and R and has been utilized a lot in that direction. For example, we can say that RStudio saturated/saturates the R ecosystem with FP, monadic DSL's for different Data Science tasks. $\endgroup$ – Anton Antonov Jan 11 at 17:14
7
$\begingroup$

A recent lecture by Wolfram himself at an MIT Open Courseware class on AGI and neural nets opened my eyes to some of Mathematica's advantages for one aspect of Machine Learning, visualizing and debugging neural networks. It's a lot easier to visualize, manipulate, and explore neural nets with the magic under the hood of Mathematica. So for learning about deep learning and iterating on neural network designs, Mathematica may be a good companion to Python or R. You can interact with your "production" Python neural nets within a Mathematica session using the StartProcess command to share data/models with Mathematica.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.