Return to Revisions

This really bugged me so I used the opportunity to learn a bit more about LibraryLink. I have to say that having a close look at the internals of Szabolcs' LTemplate package was significantly more helpful than the actual LibraryLink documentation...

Setting up the C++ code:

Needs["CCompilerDriver`"];
Needs["GeneralUtilities`"];

LibraryLink`BoilerPlate[funname_String] := 
 StringJoin["#include \"math.h\"
  
  #include \"WolframRTL.h\"
  
  static WolframCompileLibrary_Functions funStructCompile;
  
  static mbool initialize = 1;
  
  DLLEXPORT int Initialize_", funname, "(WolframLibraryData libData)
  {
    if( initialize)
    {
        funStructCompile = libData->compileLibraryFunctions;
        initialize = 0;
    }
    return 0;
  }
  
  DLLEXPORT void Uninitialize_", funname, "(WolframLibraryData libData)
  {
    if( !initialize)
    {
        initialize = 1;
    }
  }
  
  "]


funname = "writeRealMatrixToFile";
preamble = "
  #include <iostream>
  #include <fstream>
  using namespace std;
  ";

code = StringJoin[
   preamble, "\n\n",
   LibraryLink`BoilerPlate[funname],
   "extern \"C\" DLLEXPORT int ", funname, 
   "(WolframLibraryData libData, mint Argc, MArgument * Args, \
MArgument Res)
   {
    int err = 0;
       const char * file = MArgument_getUTF8String(Args[0]);
       const MTensor a0 = MArgument_getMTensor(Args[1]);
       const mreal * a = libData->MTensor_getRealData(a0);
       const char * prefix = MArgument_getUTF8String(Args[2]);
       const char * infix = MArgument_getUTF8String(Args[3]); 
       const char * suffix = MArgument_getUTF8String(Args[4]);
       const char * linesep = MArgument_getUTF8String(Args[5]);
       mint prec = MArgument_getInteger(Args[6]);
    
       mint const* dims;
       dims = libData->MTensor_getDimensions(a0);
    mint i, j;
   
    ofstream outputstream;
   
    try
    {
        outputstream.open(file, ofstream::out | ofstream::app);
        outputstream.precision(prec);
        outputstream.setf(ios::fixed, ios::floatfield);
   
        for( i=0; i < (dims[0]-1); i++)
        {
            outputstream << prefix;
            for( j=0; j < dims[1]-1; j++)
            {
                outputstream << *a << infix; 
                a++;
            }
            outputstream << *a << suffix;
            a++;
            outputstream << linesep;
        }
        {
            outputstream << prefix;
            for( j=0; j < dims[1]-1; j++)
            {
                outputstream << *a << infix;
                a++;
            }
            outputstream << *a << suffix;
        }
           
        outputstream.close();
        MArgument_setInteger(Res, err);
    }
    catch (...)
    {
        return LIBRARY_FUNCTION_ERROR;
    }
   
    return LIBRARY_NO_ERROR;
   }
   "];

Creating the library in a temporary directory. (If you'd like to reuse the library, just specify a value of choice for "TargetDirectory".)

libfile = CreateLibrary[
   code, funname,
   "Language" -> "C++",
   "TargetDirectory" -> $TemporaryDirectory
   ];

Loading the library function and creating a wrapper function as rudimentary user interface.

If[TrueQ[Head[writeRealMatrixToFile] === LibraryFunction], 
  LibraryFunctionUnload[writeRealMatrixToFile]];
ClearAll[writeRealMatrixToFile];
If[TrueQ[FindLibrary[libfile] =!= $Failed],
  writeRealMatrixToFile = LibraryFunctionLoad[libfile, funname,
    {
     "UTF8String", {Real, 2, "Constant"}, "UTF8String", "UTF8String", 
     "UTF8String", "UTF8String", Integer
     },
    Integer]
  ];

Options[WriteRealMatrixToFile] = {
   "Prefix" -> "",
   "Infix" -> "\t",
   "Suffix" -> "",
   "RowSep" -> "\n",
   "Precision" -> 16
   };
WriteRealMatrixToFile[file_String, 
   A_?(MatrixQ[#, Developer`MachineRealQ] &), OptionsPattern[]] := 
  writeRealMatrixToFile[file, A,
   OptionValue["Prefix"],
   OptionValue["Infix"],
   OptionValue["Suffix"],
   OptionValue["RowSep"],
   OptionValue["Precision"]
   ];

Running the test example.

A = RandomReal[{-1, 1}, {1000, 1000}];
file = "test.txt";
Put[file];
WriteRealMatrixToFile[file, A]; // AbsoluteTiming // First
B = Import[file, "Table"]; // AbsoluteTiming // First
Max[Abs[A - B]]

0.618298

1.40147

5.55112*10^-17

This is almost 25 times faster than using Export with "Table" as export format:

Export["A.txt", A, "Table"]; // AbsoluteTiming // First

16.0703

Final remark:

I wrote writeRealMatrixToFile such that it appends to existing files (creating a new one if needed). This way, it can be used to export multiple data sets with other chunks of code in between. Typical file formats where this might be helpful are, e.g., povray and obj.

In particular, povray needs lists of vectors specified in the format < x1, y1, z1 >,< x2, y2, z2 >,... and to put < and > is the purpose of the "Prefix" and "Suffix" option. "Infix" specifies the column separator (e.g. ","; default is a tabulator ("\t")) and "RowSep" specifies the row seperators (default is newline ("\n")).