I ended up adapting the last response for Windows, that is, deleting the folder in the init file.

Specifically, Mathematica 11 on Windows 7 creates the folder MyDocuments\Wolfram Mathematica each time it starts. Consider editing 

C:\ProgramData\Mathematica\Kernel\init.m

adding the command

    With[{dir = $UserDocumentsDirectory <> "\\Wolfram Mathematica"}, If[DirectoryQ[dir], DeleteDirectory[dir]]]

to delete it automatically. Note the DOUBLE backslash as folder separator. Alternatively, use FileNameJoin.