# Conditional Probability of first and second letters in a data set of strings

I have a dataset of strings and I would like to take only the first two letters of each string and find the conditional probability P(Second Letter|First Letter) while only looking at the words of the dataset. I can split the words and make another list with only the first two letters of each word but I'm having difficulty figuring out a way to say for example, all of the two-letter word snippets that start with 'a' what is the probability that a,b,c...z will follow as the second letter. I want to do the probability for every letter pair and put it into a 3D histogram where the x-axis will be the first letter chosen, the y-axis will be the second letter and the height will be the probability that that pair occurs within the chosen dataset.

Example: Original dataset {cat,dog,human} dataset split to only include the first two letters {ca,do,hu}

I've been trying to figure out a way to do it and I just can't seem to get it. I attempted to search to see if anyone has asked a question like mine but I didn't see anything like it. Any help is thoroughly appreciated.

• Build a matrix of frequencies of digrams? Commented Apr 12, 2021 at 21:20
• @flinty Let me explore that. I appreciate it the help. Commented Apr 12, 2021 at 21:22

(* English words, lowercase, length > 2 *)
words = ToLowerCase@Select[WordList[], StringLength[#] > 2 &];

(* Get first two chars, throw out any pairs containing non-letters/punctuation *)
pairs = Select[Characters[words][[All, ;; 2]], AllTrue[#, LetterQ] &];

(* Probability function *)
countProbs[list_] := Map[#/Length[list] &, Counts[list]];

(* Gather the pairs by common first letter and prob-tally each grouping *)
probAssoc = Association[#[[1,1]] -> countProbs[#[[All,2]]]& /@ GatherBy[pairs, First]];

(* Generate the 3D histogram *)
hist = Histogram3D[LetterNumber[pairs]]

(* Get the probability of a H following an S *)
probAssoc["s"]["h"]


• Also if you want the height to scale with probability then do: Histogram3D[LetterNumber[pairs], Automatic, "Probability"] - I used LetterNumber to convert the letters to numbers that the histogram would like - I'm not sure how you'd change the ticks to a,b,c,d,....,z but that would be nice. Commented Apr 12, 2021 at 21:47
• This is exactly what I was looking for! I appreciate the time you put into this. I will figure out how to incorporate it for every letter combination as well as convert the labels from number value to alphabet values. Again, thank you! I also have to do it for other languages but I will incorporate that as well. I just needed a beginning. Commented Apr 12, 2021 at 22:02