Why is saving my notebook taking a long time and how can I fix it?

Question

I have a notebook that I do my calculations in. I can't really copy it here because it's both pretty big, and I can't share the content. But, when I save it, the process will often take several minutes... it stops responding and does the "window not responding" thing Windows does when it thinks it's frozen, though it eventually does finish saving.

I'm guessing it's because I have some fairly large data sets (matrices of dimensions 200x200x5) and also have them plotted (maybe that's the real problem?).

If I do:

In[128]:= N@MemoryInUse[]/10^6
N@MaxMemoryUsed[]/10^6

Out[128]= 55.4787

Out[129]= 74.8118

That doesn't seem that large...

If I look in Windows Task manager, Mathematica is currently using 45MB memory. If I look at the .nb file in Windows Explorer, it's 122MB large.

I've read about setting $HistoryLength=1, so I've done that just now. (Just to be clear, that shouldn't affect variables, right? Just the output that gets automatically saved so you can access it with % ?)

Is there anything else I'm missing? For example, would it make sense to, instead of plotting 40,000 points/graphics objects on a plot, raster that plot as an image, and ditch the data or something? It's just very frustrating having MMa constantly lag, hang, and freeze.

edit:

Let me include a more concrete example. I start off by doing $HistoryLength=1, by the way. Let's say I want to produce this horrible looking image:

nSquares = 200;
sqArray = 
  Flatten[#, 1] &@
   Table[Table[{XYZColor[{Sin[.1 i] i^.8, i*j/nSquares, 
         Sin[.1 j] j^.9}/nSquares], 
      Rectangle[{i, j}, {i + 1, j + 1}]}, {i, nSquares}], {j, 
     nSquares}];
Graphics[sqArray, ImageSize -> 800]

I don't think I can downsample it because it's 800 pixels wide, and there are only 200 color squares in each direction... I'd like high resolution for these images and I can already see granularity in this one.

What's strange is that after plotting this, MemoryInUse[] tells me only 47MB is being used, but Windows Task Manager says MMa is using 597MB... In fact, even after doing a Quit[] (which I thought would delete anything using memory?), task manager still says the same... So is this just from drawing that Graphics with the ~400k rectangles?

If so, what can I do here? I can certainly import pictures with better resolution that this easily... is the smart thing to Export[] it, then import it?

edit2: So it's definitely due to drawing the Graphics[]... If I put a semicolon so it doesn't draw it, the memory in Task Manager doesn't go high and stay there. But then I obviously want to see the image, so I tried the answer I got in this post, by doing:

testgraphics = Graphics[sqArray, ImageSize -> 800];
rast = Rasterize[#, "Image", RasterSize -> 300, ImageSize -> 600] &@
  testgraphics

And besides taking like... a full minute to Rasterize it (does that seem right?), it then takes 360MB in Task Manager that doesn't go away after doing Quit[] (similar to when displaying the Graphics).

So this is confusing still. Clearly just having an image in the notebook doesn't take much memory (I just imported one to test that's at a much higher resolution, and it didn't add any noticeable memory usage). This is significantly slowing down my workflow... what can I do? Thank you.

one more thing: I tried just exporting it, so that it never even displays in my notebook. It STILL increased the memory MMA is using in WTM to 350MB (after it's done exporting).