11
$\begingroup$

Bug introduced in 6.0 and fixed in 10.2 (July 29, 2015)


To be honest, I don't know if the problem is with Mathematica, or with the file; it's been generated by PicoScope, a PC-based oscilloscope software, not Matlab.
The error I get in Mathematica 9 when I try Import[] is

Insufficient data found for MAT format

PicoScope claims it save .mat files version 4, which should be readable by Mathematica. Can anybody else give this file a try (Maybe in Mathematica 10)?


more details (in answer to Szabolcs' comment)
I'm not interested in MAT files, but while PicoScope can save text files, or CSV, both are limited in the number of samples they can save. Only MAT files can save large sets of data in PicoScope. Pretty dumb, as far as I'm concerned...

$\endgroup$
4
  • $\begingroup$ v10.0.2 result: During evaluation of In[66]:= Import::nodta: Insufficient data found for MAT format. >> During evaluation of In[66]:= Import::fmterr: Cannot import data as MAT format. >> Out[66]= $Failed $\endgroup$
    – Mr.Wizard
    Mar 23, 2015 at 17:43
  • $\begingroup$ Do you need the data extracted from this particular .mat file? MATLAB reads it. Do you have MATLAB? $\endgroup$
    – Szabolcs
    Mar 23, 2015 at 18:09
  • $\begingroup$ @Szabolcs - No, I don't have MATLAB. I have Mathematica, and my pockets aren't that deep :-). $\endgroup$
    – stevenvh
    Mar 23, 2015 at 18:38
  • $\begingroup$ Updated my answer, it's a bug. $\endgroup$
    – Szabolcs
    Mar 23, 2015 at 19:01

1 Answer 1

12
$\begingroup$

Update: Successfully debugged!! I don't know why I do this ...

This is a bug in Mathematica's MAT v4 reader that can easily be fixed on the user-side.

First, let's try to do the import, which will trigger loading the .mat-importer functionality. Now let's look at the following function System`Convert`MATDump`myImportMAT4:

Mathematica graphics

Notice the Switch listing various datatypes. These correspond to those from pages 1-36 and 1-37 of the .mat file v4 specification. 1 is associated with "Integer64". It should be "Real32" instead. Indeed, if we change this function, then it can successfully import your file.


Original answer

This is a version 4 .mat file. The specification is documented here.

I looked into the file with a hex viewer, and I can't really see anything wrong with it. Of course I might have missed something, but I suspect a bug. Please report it to Wolfram, just in case.

Version 4 .mat files have a very simple structure. The matrices are stored sequentially, each one having a 20-byte header. It should be fairly straightforward to implement a v4 mat reader in Mathematica in 1-2 hours based on the documentation. But it's likely better to just use some free tool to read the file (Octave?) and then re-save it. Hopefully Mathematica will read it then. It does read it if it's re-saved with MATLAB (though MATLAB doesn't create a file with identical data types, it converts integers and 32-bit floats to 64-bit floats).


Solution

First trigger loading the .mat import functionality by trying to import a mat file (sorry, didn't have time to figure out how to do this properly). Then evaluate the code below, which has the single correction described above. Tested only in v10.0.2

System`Convert`MATDump`myImportMAT4[
  System`Convert`MATDump`stream_InputStream, 
  System`Convert`MATDump`tmp_, System`Convert`MATDump`opts___] := 
 Module[{System`Convert`MATDump`b1, System`Convert`MATDump`b2, 
   System`Convert`MATDump`b3, System`Convert`MATDump`b4, 
   System`Convert`MATDump`mtype, System`Convert`MATDump`mrows, 
   System`Convert`MATDump`ncols, System`Convert`MATDump`imagf, 
   System`Convert`MATDump`namlen, System`Convert`MATDump`name, 
   System`Convert`MATDump`zero, System`Convert`MATDump`data, 
   System`Convert`MATDump`s, System`Convert`MATDump`mach, 
   System`Convert`MATDump`z1, System`Convert`MATDump`type, 
   System`Convert`MATDump`z2, System`Convert`MATDump`form, 
   System`Convert`MATDump`bo}, 
  If[Length[System`Convert`MATDump`tmp] =!= 4, 
   Return[$Failed]]; {System`Convert`MATDump`b1, 
    System`Convert`MATDump`b2, System`Convert`MATDump`b3, 
    System`Convert`MATDump`b4} = System`Convert`MATDump`tmp; 
  System`Convert`MATDump`mtype = 
   System`Convert`MATDump`b1 + 256 System`Convert`MATDump`b2 + 
    256 System`Convert`MATDump`b3 + 
    System`Convert`MATDump`b4; {System`Convert`MATDump`mach, 
    System`Convert`MATDump`z1, System`Convert`MATDump`type, 
    System`Convert`MATDump`z2} = 
   IntegerDigits[System`Convert`MATDump`mtype, 10, 4]; 
  If[System`Convert`MATDump`mach > 2 || 
    System`Convert`MATDump`z1 =!= 0 || 
    System`Convert`MATDump`type > 5, Return[$Failed]]; 
  System`Convert`MATDump`bo = 
   If[System`Convert`MATDump`mach === 1, 
    1, -1]; {System`Convert`MATDump`mrows, 
    System`Convert`MATDump`ncols, System`Convert`MATDump`imagf, 
    System`Convert`MATDump`namlen} = 
   BinaryReadList[System`Convert`MATDump`stream, "Integer32", 4, 
    ByteOrdering -> System`Convert`MATDump`bo]; 
  If[! IntegerQ[System`Convert`MATDump`mrows] || 
    System`Convert`MATDump`mrows < 0 || ! 
     IntegerQ[System`Convert`MATDump`ncols] || 
    System`Convert`MATDump`ncols < 0 || ! 
     IntegerQ[System`Convert`MATDump`imagf] || 
    System`Convert`MATDump`imagf < 
     0 || ! (IntegerQ[System`Convert`MATDump`namlen] || 
       System`Convert`MATDump`namlen < 1), Return[$Failed]]; 
  System`Convert`MATDump`name = 
   StringJoin[
    BinaryReadList[System`Convert`MATDump`stream, "Character8", 
     System`Convert`MATDump`namlen - 1]]; 
  System`Convert`MATDump`zero = 
   BinaryRead[System`Convert`MATDump`stream, "Byte"]; 
  If[System`Convert`MATDump`zero =!= 0, Return[$Failed]]; 
  System`Convert`MATDump`s = 
   If[System`Convert`MATDump`imagf =!= 0, 
    System`Convert`MATDump`mrows System`Convert`MATDump`ncols 2, 
    System`Convert`MATDump`mrows System`Convert`MATDump`ncols]; 
  If[System`Convert`MATDump`s === 0, Return["Data" -> {}]]; 
  System`Convert`MATDump`form = Switch[System`Convert`MATDump`type,
    0, "Real64",
    1, "Real32",
    2, "Integer32",
    3, "Integer16",
    4, "UnsingedInteger16",
    5, "Byte",
    _, "Byte"]; 
  System`Convert`MATDump`data = 
   BinaryReadList[System`Convert`MATDump`stream, 
    System`Convert`MATDump`form, System`Convert`MATDump`s, 
    ByteOrdering -> System`Convert`MATDump`bo]; 
  If[Length[System`Convert`MATDump`data] =!= System`Convert`MATDump`s,
    Message[Import::nodta, "MAT"]; Return[$Failed]]; 
  If[System`Convert`MATDump`imagf =!= 0, 
   System`Convert`MATDump`data = 
    Take[System`Convert`MATDump`data, System`Convert`MATDump`s/2] + 
     I Drop[System`Convert`MATDump`data, System`Convert`MATDump`s/2]];
   System`Convert`MATDump`name -> 
   Developer`ToPackedArray[
    Transpose[
     Partition[System`Convert`MATDump`data, 
      System`Convert`MATDump`mrows]]]]
$\endgroup$
10
  • 2
    $\begingroup$ Nice sleuthing. +1 $\endgroup$
    – ciao
    Mar 23, 2015 at 20:59
  • $\begingroup$ +1 Did you send a bug report to WRI? $\endgroup$ Mar 24, 2015 at 7:29
  • 1
    $\begingroup$ @SjoerdC.deVries Yes, I did. But the bug is still present in version 10.2 (I tried the trial). I mean the fix was handed to them on a plate nearly 4 months ago! $\endgroup$
    – Szabolcs
    Jul 16, 2015 at 17:12
  • 1
    $\begingroup$ @Szabolcs I checked but couldn't find an internal report about this issue. I've filed one now. $\endgroup$
    – Stefan R
    Jul 16, 2015 at 22:03
  • 1
    $\begingroup$ Update: I found your original report to tech support. Hopefully we can get this fixed soon. $\endgroup$
    – Stefan R
    Jul 16, 2015 at 22:13

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.