Interpolation The coordinates do not lie on a structured tensor product grid

Question

I have a big array of multidimentional data 50 Mb, 500 000 elements. and I want to interpolate it and make a normal function.

Please do not mark this question as a duplicate straightaway, because I read many answers and they actually do not answer my question, they are a little bit different. I do not ask how to fix this problem, but more about definition and a way to find where my data is wrong.

Data example is this:

{
  {{-0.5, -0.4898989898989899, 1., 0., 0.1}, 0.10120921662021362},
  {{-0.5, -0.4898989898989899, 1., 0.12566370614359174, 0.1}, 0.1012092163684851},
  {{-0.5, -0.4898989898989899, 1., 0.25132741228718347, 0.1}, 0.10120921611675658},
  {{-0.5, -0.4898989898989899, 1., 0.37699111843077515, 0.1}, 0.10120921586502805},
  {{-0.5, -0.4898989898989899, 1., 0.5026548245743669, 0.1}, 0.10120921561329953},
  {{-0.5, -0.4898989898989899, 1., 0.6283185307179586, 0.1}, 0.101209215361571}
}

This small example works fine, but when I try to interpolate all my data, I get this error

Interpolation::indim: The coordinates do not lie on a structured tensor product grid.

My questions are:

1. What is "structured tensor product grid"? Can someone give me example of "structured tensor product grid" and "NOT structured tensor product grid"? Might be I will be able to change my data accordingly, but I could not find definition of this term anywhere.

2. Is there any way to find in my big data list (50 Mb) this place, where deviation (something wrong) happens? In other words how to find where my data is wrong?

Answer 1

Interpolation requires that the data be laid out on a multidimensional (in this case, 4D) grid that is the product of 1D grids. A quick check is provided by comparing the number of data points, here,

Length[data[[All, 1]]]
(* 6 *)

with the size of the product of the four 1D grids, here

Times @@ (Length@Union[data[[All, 1, #]]] & /@ Range[4])
(* 6 *)

As observed in the question, Interpolation returns an InterpolatonFunction for data.

Suppose, however, that one instance of 1. is replaced by 1.0000000001; call this new array data1. Then,

Times @@ (Length@Union[data1[[All, 1, #]]] & /@ Range[4])
(* 12 *)

which is greater than the number of data points. Thus, the grid in this case is incomplete, and Interpolation fails. Finding the offending data points can be tedious for large arrays. Here are the 1D grids for data1.

Gather[data1[[All, 1, #]]] & /@ Range[4]
(* {{{-0.5, -0.5, -0.5, -0.5, -0.5, -0.5}}, 
    {{-0.489899, -0.489899, -0.489899, -0.489899, -0.489899, -0.489899}}, 
    {{1., 1., 1., 1., 1.}, {1.}}, 
    {{0.}, {0.125664}, {0.251327}, {0.376991}, {0.502655}, {0.628319}}} *)

The final {1.} really is ``1.0000000001`, but not all the digits are shown in the notebook output. Looking for things like this can help to identify the offending grid points.