11
$\begingroup$

Bug introduced in 6.0 and fixed in 10.2 (July 29, 2015)


To be honest, I don't know if the problem is with Mathematica, or with the file; it's been generated by PicoScope, a PC-based oscilloscope software, not Matlab.
The error I get in Mathematica 9 when I try Import[] is

Insufficient data found for MAT format

PicoScope claims it save .mat files version 4, which should be readable by Mathematica. Can anybody else give this file a try (Maybe in Mathematica 10)?


more details (in answer to Szabolcs' comment)
I'm not interested in MAT files, but while PicoScope can save text files, or CSV, both are limited in the number of samples they can save. Only MAT files can save large sets of data in PicoScope. Pretty dumb, as far as I'm concerned...

$\endgroup$
  • $\begingroup$ v10.0.2 result: During evaluation of In[66]:= Import::nodta: Insufficient data found for MAT format. >> During evaluation of In[66]:= Import::fmterr: Cannot import data as MAT format. >> Out[66]= $Failed $\endgroup$ – Mr.Wizard Mar 23 '15 at 17:43
  • $\begingroup$ Do you need the data extracted from this particular .mat file? MATLAB reads it. Do you have MATLAB? $\endgroup$ – Szabolcs Mar 23 '15 at 18:09
  • $\begingroup$ @Szabolcs - No, I don't have MATLAB. I have Mathematica, and my pockets aren't that deep :-). $\endgroup$ – stevenvh Mar 23 '15 at 18:38
  • $\begingroup$ Updated my answer, it's a bug. $\endgroup$ – Szabolcs Mar 23 '15 at 19:01
12
$\begingroup$

Update: Successfully debugged!! I don't know why I do this ...

This is a bug in Mathematica's MAT v4 reader that can easily be fixed on the user-side.

First, let's try to do the import, which will trigger loading the .mat-importer functionality. Now let's look at the following function System`Convert`MATDump`myImportMAT4:

Mathematica graphics

Notice the Switch listing various datatypes. These correspond to those from pages 1-36 and 1-37 of the .mat file v4 specification. 1 is associated with "Integer64". It should be "Real32" instead. Indeed, if we change this function, then it can successfully import your file.


Original answer

This is a version 4 .mat file. The specification is documented here.

I looked into the file with a hex viewer, and I can't really see anything wrong with it. Of course I might have missed something, but I suspect a bug. Please report it to Wolfram, just in case.

Version 4 .mat files have a very simple structure. The matrices are stored sequentially, each one having a 20-byte header. It should be fairly straightforward to implement a v4 mat reader in Mathematica in 1-2 hours based on the documentation. But it's likely better to just use some free tool to read the file (Octave?) and then re-save it. Hopefully Mathematica will read it then. It does read it if it's re-saved with MATLAB (though MATLAB doesn't create a file with identical data types, it converts integers and 32-bit floats to 64-bit floats).


Solution

First trigger loading the .mat import functionality by trying to import a mat file (sorry, didn't have time to figure out how to do this properly). Then evaluate the code below, which has the single correction described above. Tested only in v10.0.2

System`Convert`MATDump`myImportMAT4[
  System`Convert`MATDump`stream_InputStream, 
  System`Convert`MATDump`tmp_, System`Convert`MATDump`opts___] := 
 Module[{System`Convert`MATDump`b1, System`Convert`MATDump`b2, 
   System`Convert`MATDump`b3, System`Convert`MATDump`b4, 
   System`Convert`MATDump`mtype, System`Convert`MATDump`mrows, 
   System`Convert`MATDump`ncols, System`Convert`MATDump`imagf, 
   System`Convert`MATDump`namlen, System`Convert`MATDump`name, 
   System`Convert`MATDump`zero, System`Convert`MATDump`data, 
   System`Convert`MATDump`s, System`Convert`MATDump`mach, 
   System`Convert`MATDump`z1, System`Convert`MATDump`type, 
   System`Convert`MATDump`z2, System`Convert`MATDump`form, 
   System`Convert`MATDump`bo}, 
  If[Length[System`Convert`MATDump`tmp] =!= 4, 
   Return[$Failed]]; {System`Convert`MATDump`b1, 
    System`Convert`MATDump`b2, System`Convert`MATDump`b3, 
    System`Convert`MATDump`b4} = System`Convert`MATDump`tmp; 
  System`Convert`MATDump`mtype = 
   System`Convert`MATDump`b1 + 256 System`Convert`MATDump`b2 + 
    256 System`Convert`MATDump`b3 + 
    System`Convert`MATDump`b4; {System`Convert`MATDump`mach, 
    System`Convert`MATDump`z1, System`Convert`MATDump`type, 
    System`Convert`MATDump`z2} = 
   IntegerDigits[System`Convert`MATDump`mtype, 10, 4]; 
  If[System`Convert`MATDump`mach > 2 || 
    System`Convert`MATDump`z1 =!= 0 || 
    System`Convert`MATDump`type > 5, Return[$Failed]]; 
  System`Convert`MATDump`bo = 
   If[System`Convert`MATDump`mach === 1, 
    1, -1]; {System`Convert`MATDump`mrows, 
    System`Convert`MATDump`ncols, System`Convert`MATDump`imagf, 
    System`Convert`MATDump`namlen} = 
   BinaryReadList[System`Convert`MATDump`stream, "Integer32", 4, 
    ByteOrdering -> System`Convert`MATDump`bo]; 
  If[! IntegerQ[System`Convert`MATDump`mrows] || 
    System`Convert`MATDump`mrows < 0 || ! 
     IntegerQ[System`Convert`MATDump`ncols] || 
    System`Convert`MATDump`ncols < 0 || ! 
     IntegerQ[System`Convert`MATDump`imagf] || 
    System`Convert`MATDump`imagf < 
     0 || ! (IntegerQ[System`Convert`MATDump`namlen] || 
       System`Convert`MATDump`namlen < 1), Return[$Failed]]; 
  System`Convert`MATDump`name = 
   StringJoin[
    BinaryReadList[System`Convert`MATDump`stream, "Character8", 
     System`Convert`MATDump`namlen - 1]]; 
  System`Convert`MATDump`zero = 
   BinaryRead[System`Convert`MATDump`stream, "Byte"]; 
  If[System`Convert`MATDump`zero =!= 0, Return[$Failed]]; 
  System`Convert`MATDump`s = 
   If[System`Convert`MATDump`imagf =!= 0, 
    System`Convert`MATDump`mrows System`Convert`MATDump`ncols 2, 
    System`Convert`MATDump`mrows System`Convert`MATDump`ncols]; 
  If[System`Convert`MATDump`s === 0, Return["Data" -> {}]]; 
  System`Convert`MATDump`form = Switch[System`Convert`MATDump`type,
    0, "Real64",
    1, "Real32",
    2, "Integer32",
    3, "Integer16",
    4, "UnsingedInteger16",
    5, "Byte",
    _, "Byte"]; 
  System`Convert`MATDump`data = 
   BinaryReadList[System`Convert`MATDump`stream, 
    System`Convert`MATDump`form, System`Convert`MATDump`s, 
    ByteOrdering -> System`Convert`MATDump`bo]; 
  If[Length[System`Convert`MATDump`data] =!= System`Convert`MATDump`s,
    Message[Import::nodta, "MAT"]; Return[$Failed]]; 
  If[System`Convert`MATDump`imagf =!= 0, 
   System`Convert`MATDump`data = 
    Take[System`Convert`MATDump`data, System`Convert`MATDump`s/2] + 
     I Drop[System`Convert`MATDump`data, System`Convert`MATDump`s/2]];
   System`Convert`MATDump`name -> 
   Developer`ToPackedArray[
    Transpose[
     Partition[System`Convert`MATDump`data, 
      System`Convert`MATDump`mrows]]]]
$\endgroup$
  • 2
    $\begingroup$ Nice sleuthing. +1 $\endgroup$ – ciao Mar 23 '15 at 20:59
  • $\begingroup$ +1 Did you send a bug report to WRI? $\endgroup$ – Sjoerd C. de Vries Mar 24 '15 at 7:29
  • 1
    $\begingroup$ @SjoerdC.deVries Yes, I did. But the bug is still present in version 10.2 (I tried the trial). I mean the fix was handed to them on a plate nearly 4 months ago! $\endgroup$ – Szabolcs Jul 16 '15 at 17:12
  • 1
    $\begingroup$ @Szabolcs I checked but couldn't find an internal report about this issue. I've filed one now. $\endgroup$ – Stefan R Jul 16 '15 at 22:03
  • 1
    $\begingroup$ Update: I found your original report to tech support. Hopefully we can get this fixed soon. $\endgroup$ – Stefan R Jul 16 '15 at 22:13

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.