You may be able to hack together something with Text
. Look up how Scaled
, ImageScaled
and Offset
work for positioning. (The latest version of MaTeX has a short tutorial on these in its documentation: "Preparing Figures to Size".) Make enough room around the frame by setting ImagePadding
manually (unfortunately, this is rather inconvenient). Turn off PlotRangeClipping
so that elements outside the frame will be rendered (this may be a problem for some plots).
Plot[x, {x, -1, 1}, Frame -> True, PlotTheme -> "Detailed",
PlotLegends -> None,
ImagePadding -> {{30, 5}, {30, 5}},
PlotRangeClipping -> False,
Epilog -> {Text[TraditionalForm[y],
Offset[{-20, 0}, Scaled[{0, 1}]], {1, 1}],
Text[TraditionalForm[x],
Offset[{0, -15}, Scaled[{1, 0}]], {1, 1}]}
]

The default colour of frames/labels is not black, but grey. You may want to figure out what shade it is exactly and match your own labels.
Unfortunately this method is a lot of trouble. If you are going to put this much work into it anyway, take a look at SciDraw. I recommend it for when you are creating publication-quality figures (rather than just a quick visualization).