# Export to MX seems to corrupt the in-memory Dataset

I have a function that does a large number of calculations (by calling out to other smaller functions), and then builds a large Dataset object. Then I do two things with the object:

(a) I write it to disk as an MX file using Import; (b) I write the rows of the MX file to a SQL Server database (by using DatabaseLink).

So, the sequence of calls is something like this:

mainFunction[param1_, param2_] := Module[{dbConn, result},
result = Composition[
innerFunctionN,
.
.
.
innerFunction2,
innerFunction1
][];
(* At this point, result is a 250 MB Dataset object *)
(* We now write this object to disk *)
Export["result.mx", result]; (* This step happens correctly *)
(* Set dbConn to a valid SQLConnection object *)
DatabaseLinkSQLExecute[
dbConn,
queryBuilder[result]
]; (* This step fails; JDBC parse error from string to float *)
];


But curiously, the following succeeds:

mainFunction[param1_, param2_] := Module[{dbConn, result, result1},
result = Composition[
innerFunctionN,
.
.
.
innerFunction2,
innerFunction1
][];
(* At this point, result is a 250 MB Dataset object *)
(* We now write this object to disk *)
Export["result.mx", result]; (* This step happens correctly *)
(* Set dbConn to a valid SQLConnection object *)
(* NEW STEP: NOW READ RESULT BACK FROM DISK *)
result1 = Import["result.mx"];
dbConn,
queryBuilder[result1]
]; (* This succeeds! *)
];


This has been driving me crazy. The only conclusion I can draw is that something about the MX write is corrupting result in memory.

I am currently using the second method as a workaround. But do you have a better suggestion? Can you please help me understand the problem?

• In the second case, is result == result1? – MarcoB May 22 '15 at 21:12
• And also result === result1? – Mr.Wizard May 22 '15 at 21:13
• Unfortunately, I don't have a ready answer to your questions. The sequence of calls innerFunction1, innerFunction2, ..., take about four hours to complete. I will have to set off the computations once again. But is there any reason why they should not be equal? – Shredderroy May 22 '15 at 21:23