JavaGraphics` and CreateDialog don't work with ListPlot in Mathematica 11.3/12

Question

Bug introduced in 11.2 and persists through 12.2, case 4331202

I'm trying to see some plots based on data from a remote cluster as it runs various Mathematica 11.3/12 scripts, so I want to use the JavaGraphics` package to render plots that I can in turn X11 forward to my computer. The X11 forwarding is definitely working, as I can Plot and ListPlot3d just fine and get nice plots. However, when I try to ListPlot, it fails. I'm able to reproduce this same behavior in my local Mathematica 12 Notebook. Here's how:

<<"JavaGraphics`"
Plot[Sinc[x],{x,0,20}]
Pause[5];
ListPlot3D[Table[Mod[x,y],{x,0,20},{y,0,20}]]
Pause[5];
ListPlot[Table[Prime[n],{n,1,20}]]

But meanwhile, when I go back into the internet sands of time, I see evidence of it working at some point. Somewhat infuriatingly, ListPlot is the function I need, and I can't see how I could replace it for making a scatter plot.

As a potential foray into figuring this out, I noticed that I get the same non-functionality with the option DisplayFunction->CreateDialog.

Plot[Sinc[x],{x,0,20},DisplayFunction->CreateDialog]
Pause[5];
ListPlot3D[Table[Mod[x,y],{x,0,20},{y,0,20}],DisplayFunction->CreateDialog]
Pause[5];
ListPlot[Table[Prime[n],{n,1,20}],DisplayFunction->CreateDialog]

I've confirmed both of these bugs over multiple Mathematica installations on multiple devices running both 11.3 and 12.

Does anyone have any experience with this, or any hunches as to why this isn't working?

Answer 1

As noted in the comments, this bug appears to have been introduced in version 11.2, when they moved a lot of the plot post-processing into System`ListPlotsDump`postProcessLayout. The issue occurs because they temporarily set DisplayFunction->Identity (to prevent multiple applications of the DisplayFunction), and then the wrong DisplayFunction setting is forwarded to `postProcessLayout. The code below fixes this issue by prepending the correct setting to the relevant list of options:

ListPlot[{}];
Begin[If[$VersionNumber >= 11.3, "System`ListPlotsDump`", 
   "System`ProtoPlotDump`"]];
With[{sym = If[$VersionNumber >= 12.1, `preProcessLayout, `iListPlot]},
  SubValues[sym] = SubValues[sym] /. 
    HoldPattern[lhs : _["PreProcessedOptions"] = `opts] :>
      (lhs = Prepend[`opts,
        DisplayFunction -> OptionValue[{`fn, `iListPlot}, `o, DisplayFunction]
      ])
  ];
End[];

Update

They seem to have moved stuff around again in 12.1, now the line to be patched is in System`ListPlotsDump`preProcessLayout. The variable displayfunction also doesn't exist anymore, so we have to extract it ourselves. I have patched the code above to account for this.