Bug reproduced in 10.3.1 and persisting through 11.2.0
From:
Note:
Modal
dialogs will prevent users from accessing other notebooks in the front end while the dialog is open. If there is no mechanism for closing the dialog, the Wolfram System will effectively freeze.tutorial/CreatingDialogBoxes
But try to evaluate this code: (all at once)
CreateDialog[{TextCell["ordinary 1"], DefaultButton[]},
WindowMargins -> 100]
CreateDialog[{TextCell["modal"], DefaultButton[]}, Modal -> True,
WindowMargins -> 200]
CreateDialog[{TextCell["ordinary 2"], DefaultButton[]},
WindowMargins -> 300]
Both "ordinary" dialogs are expected to be frozen till the Modal one is closed. It's not the case on Windows. Works correctly on Mac OS though.
I don't think documentation leaves any room for exceptions, right?
Related: What really makes a Dialog expresion behave as a modal dialog?