Why a simple Plot on Raspberry Pi 3 with Mathematica takes 20 seconds?

Question

I'm with ssh -X raspberrypi logged in. So $DISPLAY points to a X-Server. I use a very simple script file script.m with

Plot[{x}, {x, 0, 1}, PlotLabels -> "Expressions"] 

I run this like

$ time wolfram -script script.m

, which gives me

real    0m21.314s
user    0m4.100s
sys     0m1.090s

If I run

Plot[{x}, {x, 0, 1}]

, it gives me:

real    0m2.930s
user    0m2.320s
sys     0m0.230s

, so almost 20 seconds faster than the plot with the option. I see a process WolframFrontEnd popping up in top, when i set the option PlotLabels.

So how to avoid that wolfram talks to the X Server? And why is this option dependent in Plot?