Mathematica Stack Exchange is a question and answer site for users of Mathematica. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

I am very new to mathematica but it looks like an interesting language and I would like to explore it for a future project to create a web search engine. It seems like it would be fairly well suited for this as it seems to already be powering wolfram alpha.

I was wondering if the only databases you can hook into are SQL like databases. I can't see anywhere that talks about hooking into Mongo or a graph database like Neo4j. Why is this? Does this not make sense for some reason?

Any thoughts would be great. Thanks.

Edit:

So ya. If you have examples of ways to connect Mathematica to MongoDB that would be awesome. If possible I would also like to see an example of connecting it to Neo4j. (Or reasons why that would not make sense). I guess I am wondering why everything for Mathematica is being used to connect to SQL databases thus far? Is there a reason for that?

share|improve this question
    
You can connect MongoDB to Mathematica but it is not as straight forward as using the built in SQL connectors. A possible advantage of Mongo, and possibly Couch and similar, is that in theory you should be able to access these from a free CDF. – Mike Honeychurch Jan 19 '14 at 22:17
4  
If you re-word your question I'll dig up some MongoDB set up instructions and code and post later today. – Mike Honeychurch Jan 19 '14 at 22:28
    
Cool. Thanks. I updated it. Why are NoSQL databases less popular with Mathematica? And what about using a graph database like neo4j. The one way I was thinking of going about this could be using Clojuratica (call mathematica from clojure), and use clojure to access the database. – user2433617 Jan 21 '14 at 2:44
up vote 22 down vote accepted

Here are the two ways I have successfully used MongoDB with Mathematica. AFAIK much of this would also apply to similar DBs such as CouchDB.

A. Easy way

One working example used a third party Mongo supplier such as 28msec.io and use URLFetch e.g.

enter image description here

With this supplier you can both read from and write to the database using HTTP GET and simple query strings. While I have not tested it, it seems to me that you should be able to make calls to a set up such as this from a free CDF (it is my understanding that you can import from a URL from free CDF).

B. Stand alone implementation

I have installed MongoDB on my Mac and got it working with Mathematica using this method:

enter image description here

I do not imagine that the set up would be too different on other systems. To interface with Mathematica I have used the REST API. For me as a non-Java programmer this was far more straight forward than trying to make a JLink connector. I have no idea of performance issues of the REST API vs purpose built JLink connector. For the intended application there were no noticeable problems, slowdowns etc.

1 Download and install MongoDB.

2 Start the webserver

$ sudo apachectl start

3 Start Mongo DB

$ mongod --rest

4 Check that everything is working on port 28017

127.0.0.1:28017

enter image description here

The problem with the install is that the bundled REST interface does not allow inserting etc. Full REST capabilities can be achieved with an external tool such as sleepy mongoose.

5 Install pymongo

$ easy_install pymongo

6 Download and install sleepy.mongoose.

You are now in business and can read from and write to your local MongoDB from Mathematica using URLFetch.

enter image description here

Edit

Forgot to mention that you use ImportString[ string, "JSON"] to convert the JSON to rules. Also forgot to mention that for easy option A, or similar database providers, if you can enter the login and password as part of the query string then you can use Import to read from, and write to, the database.

Edit 2

I've been using RESTHeart for the past few months and find that a better API. Add your instructions using the "Body" option for URLFetch.

share|improve this answer
    
Wow this is awesome. Thanks! – user2433617 Jan 21 '14 at 21:32
1  
Since 10.2, ImportString[jsonText, "RawJSON"] is perfect to this job. Transforming JSON into an Association. – Murta Oct 28 '15 at 17:53

Mathematica can also connect to Neo4J via the REST API. Instructions for use on OS X:

1. Install Neo4j

2. start the webserver

$ sudo apachectl start

3. start Neo4j (installation in usr/local/bin)

$ neo4j start

4. Make sure all is working

127.0.0.1:7474

enter image description here

Examples:

URLFetch["http://localhost:7474/db/data/node",
 {"StatusCode", "Content"},
 "Method" -> "POST"]

Neo4J has a cypher query language that can be used from Mathematica in an analogous way to how we use SQL queries from SQLExecute:

enter image description here

tmp = URLFetch["http://localhost:7474/db/data/cypher",
  {"StatusCode", "Content"},
  "Method" -> "POST",
  "Parameters" -> {"query" -> 
     "MATCH (a)-[r]->(b)RETURN DISTINCT head(labels(a)) AS This, type(r) AS To,   
head(labels(b)) AS That LIMIT 100"}
]

(*

{200, "{
   \"columns\" : [ \"This\", \"To\", \"That\" ],
   \"data\" : [ [ \"BusinessLine\", \"LOCATED_IN\", \"Building\" ], [ \
\"BusinessProcess\", \"CONTAINS\", \"Process\" ], [ \"BusinessProcess\
\", \"BUSINESSPROCESS_HAS_RTO\", \"RTO\" ], [ \"Process\", \
\"USED_BY\", \"BusinessLine\" ], [ \"Process\", \"PROCESS_HAS_RTO\", \
\"RTO\" ], [ \"Process\", \"USES\", \"Application\" ], [ \"RTO\", \
\"PRECEDES\", \"RTO\" ] ]
 }"}

*)

The output is JSON so

tmp2 = ImportString[tmp[[2]], "JSON"];
TableForm["data" /. tmp2]

enter image description here

Things get a bit tricky for more complex queries that return graphs but above gives the structure for connecting and querying.

share|improve this answer

For MongoDB instead of using the Rest interface using urls as suggested by Mike, you can use the Java driver.

Here's the proof of concept that should allow a lot of fruitful things ...
I'm still amazed by the possibilities offered by JLink.

Prerequisites

Install MongoDB, for example on Windows http://docs.mongodb.org/manual/tutorial/install-mongodb-on-windows/
Follow the getting started http://docs.mongodb.org/manual/tutorial/getting-started/
Download the Java driver http://docs.mongodb.org/ecosystem/drivers/java/

Some useful links as a start

http://www.mkyong.com/mongodb/java-mongodb-convert-json-data-to-dbobject
http://howtodoinjava.com/2014/06/03/java-mongodb-insert-documents-in-collection-examples/
http://docs.mongodb.org/manual/reference/sql-aggregation-comparison/
http://www.mathematica-journal.com/issue/v9i1/contents/UI_With_JLink/UI_With_JLink_2.html

Proof of concept

Installing the Java driver

Needs["JLink`"]
ReinstallJava[ClassPath->"yourPath\\mongo-2.10.1.jar"];

Connecting to the database

mongo=JavaNew["com.mongodb.Mongo","127.0.0.1",27017];

Here's an example if you need to authenticate, for example on the cloud with MongoLab

mongo=JavaNew["com.mongodb.MongoClient",JavaNew["com.mongodb.MongoClientURI","mongodb://user:pass@xxxxx.mongolab.com:63140/mydb"]]

Defining a collection

db=mongo@getDB["mydb"];
collection=db@getCollection["testData"];

Converting an association to an insertable document. You could just use rules instead of Associations.

assocToJson[expr_]:=expr//.a_Association:>Normal[a]//ExportString[#,"JSON"]&;
json=<|"x"->3.`,"y"-><|"z"->48|>|>//assocToJson;
LoadJavaClass@"com.mongodb.util.JSON";
dbObject=JSON`parse[json];
dbObject@toString[] (*not necessary, just for display*)

Other way to create a document

document=JavaNew["com.mongodb.BasicDBObject"];
document@put["keyTest",MakeJavaObject@"valueTest2"];
document@toString[]

Inserting two documents

collection@insert[{dbObject,document}];

Query and converting the results back to Associations

cursorDoc=collection@find[];
Needs["GeneralUtilities`"]; (*for ToAssociations*)
While[cursorDoc@hasNext[],
    Print[(cursorDoc@next[])@toString[]//ImportString[#,"JSON"]&//ToAssociations]
]
share|improve this answer
    
2 votes ! Finally ! Lol – faysou Jan 7 '15 at 7:11
    
I only noticed this yesterday! +1 of course – Mike Honeychurch Jan 8 '15 at 2:53
    
Could something similar to @Murta method be employed for this as well? – Mike Honeychurch Jan 8 '15 at 2:54
    
I've switched to Couchbase recently, it's similar to MongoDB but better (for me). It's much faster on a Windows machine and there's a query language called N1QL that is an SQL adapted to documents. The same ideas as in my post above can be used for this db. – faysou Dec 12 '15 at 13:22
    
I've been using a different REST API for Mongo since my post. It works fine but I'd like to try other approaches but cannot get the java driver to work – Mike Honeychurch Dec 12 '15 at 23:32

I just made my MongoDB Mathematica driver open-source here: https://github.com/zbjornson/MongoDBLink. There are quick setup instructions and an overview in the readme.

It's built on the official Java driver, like @faysou's answer, but it's optimized to reduce the JLink overhead when serializing/deserializing many documents. It also transparently maps some of the Wolfram Dataset/Query operations to MongoDB aggregation operations, so you can do things like myCollection[Total, "myKey"].

share|improve this answer
    
Nice! Couchbase is very good also especially since the introduction of N1QL, I switched to it recently from MongoDb. In case you happen to use it, I wish to see a similar driver for it! Thanks again – faysou Jan 1 at 20:28
    
Are their any simple instructions on how to upgrade to use this with Mongo V3.2. I downloaded the latest driver but presumably the class files needs updating ?? – Mike Honeychurch Jan 13 at 5:42
    
The java will need to be recompiled. It doesn't look like any methods I used are removed so it should be easy, will do it in the morning. Github issue for this. – ZachB Jan 13 at 8:32
    
@MikeHoneychurch evidently didn't need recompiling. All tests pass with 3.2 driver; re-download the package from github to update. – ZachB Jan 13 at 20:02
    
@ZachB I had already updated the driver yesterday but continued to get errors which is why I thought other parts of the package needed updating. So today I downloaded Github again and I still get errors when running your tests. FindDocuments does not work for me. I am having the same problems as this user here: mathematica.stackexchange.com/questions/103925/… – Mike Honeychurch Jan 13 at 22:49

Here is one example of how to create a neo4j connection using JDBC driver from this git.

The steeps are:

1- Put the JAR file (neo4j-jdbc-2.0.1-SNAPSHOT-jar-with-dependencies.jar) together with other jar drivers in: FileNameJoin@{ $InstallationDirectory, "Contents/SystemFiles/Links/DatabaseLink/Java"}

2- Restart you Mathematica

3- Now, using Movie Graph as example (that comes with neo4j).

Needs["DatabaseLink`"]
conn=OpenSQLConnection[ JDBC["org.neo4j.jdbc.Driver", "jdbc:neo4j://localhost:7474"]];
sql="
    MATCH (tom:Person {name: \"Tom Hanks\"})-[:ACTED_IN]->(movies)
    RETURN tom.name,movies.title
    LIMIT 10
";
SQLExecute[conn,sql,"ShowColumnHeadings"-> True]//TableForm
CloseSQLConnection[conn];

enter image description here

UPDATE

The maven repository sounds to be the best place to download the latest JDBC jar version. https://maven-repository.com/artifact/org.neo4j/neo4j-jdbc

I retested the 2.1.4 file (with dependency), and worked just nice.

share|improve this answer
1  
+1 very useful! – Mike Honeychurch Jan 8 '15 at 2:52
    
I get this working fine and also get @faysou example working but cannot get a mongodb connection using OpenSQLConnection. I know the driver is fine because I can reproduce faysou example. – Mike Honeychurch Dec 13 '15 at 3:25

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.