OSX on Mac now allows two apps to split the screen. https://support.apple.com/en-us/HT204948 Doesn't seem to work in Mathematica, even though the full-screen button displays an icon suggesting that it should.
Update: Mathematica 12.0 now supports this.
You can't. This feature only works with certain programs and as of version 10.4 Mathematica is not one of them. Maybe the coming Mac front end improvement mentioned in this presentation will fix that, but this is purely a guess on my part.
As a workaround you can use window control software such as Spectacle, which makes it easy to arrange notebooks in a similar way to how the split-screen feature would.