Problem during the analysis and plotting of a large data

Question

I am dealing with a large dataset and I need to plot the dataset as a contour plot. The code is as follows:

q = Table[i, {i, 1, 1020}];
p = Table[i, {i, 1, 4092}];
testdata1 = Table[{i, N[Sin[i]]}, {i, 1, Length[p]}];
processeddata1 = 
  Partition[Flatten@Table[Table[Flatten@Join[{j, j*testdata1[[i]]}], {i, 
   1, Length[p]}], {j, 1, Length[q]}], 3]
ListContourPlot[processeddata1]

Whenever I try to plot the dataset, unfortunately it takes enormously long time and stuck ultimately. So, has there any way to get rid of that and to plot of my dataset smoothly? I am using Mathematica 12.0

Please help me.

Answer 1

When I try to plot only 1 single contour line with a reduced data set (you have over 4 millions points) I get:

ListContourPlot[processeddata1[[1 ;; 4173840 ;; 50]], Contours -> 1]

A black triangle? As this is only 1 contour line, this line must wiggle to and fro to fill the whole triangle. A Plot3D may help solve the mystery:

This looks at first rather innocent, but if we look at it from the right side:

we see that the surface is highly corrugated. This together with over 4 million data points makes it a tremendous job to draw several contour lines. Anyway, contour lines are not of much help if they fill the whole picture.

Answer 2

Do you really need all points plotted? In principle you are parametrizing a 2D manifold $$ \{x,y,z\} = \{j, i j, j \sin i\} $$ over the ranges $i\in[1,4092]$ and $j\in[1,1020]$. We can plot it directly with

ParametricPlot3D[{j, i j, j Sin[i]}, {i, 1, 4092}, {j, 1, 1020},
  BoxRatios -> {1, 1, 1}, PlotPoints -> 100, AxesLabel -> {x, y, z}]

Playing with the options of this ParametricPlot3D may give you exactly what you need without going through the list construction of processeddata1.