# Export to MX seems to corrupt the in-memory Dataset

I have a function that does a large number of calculations (by calling out to other smaller functions), and then builds a large Dataset object. Then I do two things with the object:

(a) I write it to disk as an MX file using Import; (b) I write the rows of the MX file to a SQL Server database (by using DatabaseLink).

So, the sequence of calls is something like this:

mainFunction[param1_, param2_] := Module[{dbConn, result},
result = Composition[
innerFunctionN,
.
.
.
innerFunction2,
innerFunction1
][];
(* At this point, result is a 250 MB Dataset object *)
(* We now write this object to disk *)
Export["result.mx", result]; (* This step happens correctly *)
(* Set dbConn to a valid SQLConnection object *)
DatabaseLinkSQLExecute[
dbConn,
queryBuilder[result]
]; (* This step fails; JDBC parse error from string to float *)
];


But curiously, the following succeeds:

mainFunction[param1_, param2_] := Module[{dbConn, result, result1},
result = Composition[
innerFunctionN,
.
.
.
innerFunction2,
innerFunction1
][];
(* At this point, result is a 250 MB Dataset object *)
(* We now write this object to disk *)
Export["result.mx", result]; (* This step happens correctly *)
(* Set dbConn to a valid SQLConnection object *)
(* NEW STEP: NOW READ RESULT BACK FROM DISK *)
result1 = Import["result.mx"];
dbConn,
queryBuilder[result1]
]; (* This succeeds! *)
];


This has been driving me crazy. The only conclusion I can draw is that something about the MX write is corrupting result in memory.

I am currently using the second method as a workaround. But do you have a better suggestion? Can you please help me understand the problem?

• In the second case, is result == result1? – MarcoB May 22 '15 at 21:12
• And also result === result1? – Mr.Wizard May 22 '15 at 21:13
• Unfortunately, I don't have a ready answer to your questions. The sequence of calls innerFunction1, innerFunction2, ..., take about four hours to complete. I will have to set off the computations once again. But is there any reason why they should not be equal? – Shredderroy May 22 '15 at 21:23
• Not that I can think of; it sounds like a strange problem indeed, but it wouldn't be the first time there was an explanation for something I thought was surely a bug. If a smaller working example is not possible I think you should contact Wolfram support about this. – Mr.Wizard May 22 '15 at 22:26
• I have no solution, but I have seen similar behavior when I was writing (relatively) large Datasets in an earlier version 10 (I think it was 10.0.1). I have never investigated what it was and just reloaded as you do in the second example (which was almost immediate in my case). I just checked and found it does not happen for my case with 10.0.2 on Windows anymore. Which version are you using? – Albert Retey May 24 '15 at 8:45