Assigning probabilities to an Autocomplete function

Question

I have created an autocomplete function based on WordList.

auto = Autocomplete[WordList["KnownWords"]]

And if I use this on the letters "ga", it will provide me with a long list of possible words. Is it possible to create a probability list from these to choose the most commonly used one?

I am aware this is more of a theoretical question and am happy with any answers that point me in the right direction.

Answer 1

QUICK TAKE

You can make use of WordFrequencyData and define

auto=Autocomplete[WordList["KnownWords"]];
hint[word_]:=First[Keys[ReverseSort[DeleteMissing[WordFrequencyData[auto[word],IgnoreCase->True]]]]]

It will give you most popular hint, for examplle:

hint["gath"]

"gathered"

DETAILS

So starting from hints for say "gath"

auto = Autocomplete[WordList["KnownWords"]];
hints = auto["gath"]

{"gather", "gatherer", "gathered", "gathering"}

you can find and sort reversely their frequencies in typical English:

freq=ReverseSort[DeleteMissing[WordFrequencyData[auto["gath"],IgnoreCase->True]]]

<|gathered->0.000013772,gathering->8.98098*10^-6,gather->7.46439*10^-6,gatherer->3.42559*10^-7|>

Then get already order hints as

Keys[freq]

{gathered,gathering,gather,gatherer}

or just the most frequent one

First[Keys[freq]]

"gathered"

NOTE

WordFrequencyData can slow down for too many words as it accesses Wolfram servers to download data and needs internet. One trick to speed this up - is to pre-download and save WordFrequencyData for your set of words. It might take a long time for many 1000s of words, but is doable if you split it in cycles by, say, 100 words.