Doing local FFT on huge 3D vector data cell mesh and visualizing it spatially?

Simulation type:

I'm running a simulation with the OOMMF micromagnetics package

where are magnet is represented by a mesh of 3 million cells, it gets excited by a magnetic pulse, the single cells oscillate and data output for every cell is $x, y, z$ direction and absolute magnetic field value of every component over time, so a mini-magnet. Now I want to do a FFT on $m_z(t)$ for every cell and later visualize this magnet over time while cells with $m_z(t)$ showing common oscillation modes/eigenfrequencies get same color.

Data output format:

It is "data binary 4" and as said $x, y, z$ and absolute field value for every component is given for all time steps (below 300-500 time steps).

Download an example of a time-step data file from the middle of the simulation:

The manual says:

The binary representations are IEEE floating point in network byte order (MSB). To insure that the byte order is correct, and to provide a partial check that the le hasn't been sent through a non 8-bit clean channel, the first datum is a predened value: 1234567.0 (Hex: 49 96 B4 38) for 4-byte mode, and 123456789012345.0 (Hex: 42 DC 12 21 83 77 DE 40) for 8-byte mode. The data immediately follow the check value. The structure of the data depends on whether the \meshtype" declared in the header is \irregular" or \rectangular". For irregular meshes, each data element is a 6-tuple, consisting of the x, y and z components of the node position, followed by the x, y and z components of the field at that position. Ordering among the nodes is not relevant. The number of nodes is specied in the \pointcount" line in the segment header. For rectangular meshes, data input is field values only, in x, y, z component triples. These are ordered with the x index incremented first, then the y index, and the z index last. This is nominally Fortran order, and is adopted here because commonly x will be the longest dimension, and z the shortest, so this order is more memory-access ecient than the normal C array indexing of z, y, x. The size of each dimension is specied in the \xnodes, ynodes, znodes" lines in the segment header.

I use a rectangluar mesh here

Question:

The FFT can be done using something like MATHEMATICA or MATLAB for a single cell and ($m_z$/ number, time/number) data list .dat file, but a data binary 4 file for a single time step for 3 million cells is up to 20 MB big, workstation with 30 GB RAM and 12 Cores available. The manual above mentions FORTRAN (to my knowledge not so easy to learn language although very fast).

What is the way to achieve the results I'm looking for and described above? From my point of view I face the problem to do the FFT in reasonable time with a script process for around 3 million cells and then transforming that data into a spectrogram (to see what different eigenfrequencies and higher harmonics exist, also this can be done by using the average $m_z$ (summed over all single cells) and simply FFT it) but also have a output format of that data binary 4 format after FFT that can be used to feed a visualization software that can read such mesh data and color code it, show single 2D slices of the 3D object.

I read that MATHEMATICA can deal with binary 4 and offers high performance, but as this is completely new land to me, I first want to ask given above constraints and wishes, what is the best way and software/programming language to achieve this in reasonable time (few weeks)?

-
You need to at least upload the 20MB file and share a download link here! Then someone may try to shade some light to your question... –  PlatoManiac Apr 12 '13 at 8:32
@PlatoManiac thanks, here is a link from one time step data file in the middle of the simulation, ge.tt/1QgOPqd/v/0 –  James Last Apr 12 '13 at 8:53
Your question describes the file as big-endian (what is meant by "network byte order (MSB)", if I understood the strange language of the manual correctly) and contains "data binary 4" (whatever that means). But, the file you provide is little-endian and contains "data binary 8" (again, with unclear meaning). These contradictions will make answering this question very difficult--plus, you don't mention the number of time steps involved, which is surely one of the most important parameters. I'm also unconvinced that reading some file and doing an FFT is really a Mathematica question. –  Oleksandr R. Apr 12 '13 at 11:22
A quick search returned this public-domain code for interpreting the file format and reading its contents. We see that "data binary 4" is just an array of single-precision floating point values (IEEE754 binary32), while "data binary 8" is an analogous array of doubles (IEEE binary64). While this library can certainly be linked into Mathematica if that is what you really desire, as you imply that you don't actually know Mathematica, I wonder if there is any point in doing so, rather than using a language you do know? –  Oleksandr R. Apr 12 '13 at 11:39
@OleksandrR. Thanks for your comment. Overall time steps are 300-500 as written above. I know mathematica, but didn't use it for such purposes, mainly simple data (number, number) table files. Here scicomp.stackexchange.com/a/6826/3873 it is actually recommended, I just was not sure if I can do everything of it in MATHEMATICA, as in older paper FORTRAN was used to do this. If it is doable with high performance in MATHEMATICA I would prefer it, as I know the syntax, but then have to dig into big data analysis or maybe someone here can recommend some functions or kind of preprocessing of –  James Last Apr 12 '13 at 12:04