I asked this same question at the Wolfram Community.

Ian Hojnicki of Wolfram gave the following piece of advice. Give the command

CurrentValue[$FrontEndSession, "NotebookBrowseDirectory"] = 
 NotebookDirectory[EvaluationNotebook[]]

I tested it today, and it works as prescribed. I guess I will be copy/pasting that command to the relevant notebooks as an initialization cell as explained in Nasser's answer.