What I want to do is to recognize keys in the image and create a list of keys around every key that fall within a pentagonal (or similar) region. These are the ones that can be accidentally pressed instead of the intended key. For the following case, the keys around 'y' will be y->{6,7,t,u g h}. For keys at the edges there would be fewer adjacent keys. Any convenient output format will do.

enter image description here

I have seen 27045, but could not get it to work with this keyboard image. To get a fresh image:


Your help is appreciated. Thanks.

  • $\begingroup$ Just in case someone has the knowledge to take it further... The functions ImageContents and ImageCases seemed promising to me, but we'd need an entity concept for "letter" or "alphabetic character" or something like that. Both functions (without the qualifier as a second argument) find the keyboard and nothing else. I don't know if that means it can't possibly find anything else, or if it would use a different search strategy if we could tell it to look for "letter". Anyway, if we could find the letters this way, then you can find their location and use a distance function etc etc. $\endgroup$
    – lericr
    Nov 3, 2023 at 17:06
  • $\begingroup$ Will the image always be a QWERTY keyboard? In which case it might not be necessary to recognize characters. $\endgroup$
    – MelaGo
    Nov 3, 2023 at 19:52
  • $\begingroup$ @Melago, It is easy enough to create a list by hand, but the general task is about recognizing characters that may not be static. $\endgroup$
    – Syed
    Nov 4, 2023 at 1:25
  • $\begingroup$ Speaking of which here is a post on the Puzzling SE. $\endgroup$
    – Syed
    Nov 4, 2023 at 2:38
  • 2
    $\begingroup$ maybe we can start with something like ds = Dataset[ TextRecognize[Binarize@ImageResize[img, 3000], "Character", {"Text", "BoundingBox", "Image"}, RecognitionPrior -> "Column"], MaxItems -> All]? $\endgroup$
    – kglr
    Nov 4, 2023 at 13:41

2 Answers 2

kb = Reverse @ 
  TakeList[Characters["1234567890qwertyuiopasdfghjklzxcvbnm"], {10, 10, 9, 7}]; 

 vLabels = Join @@ 
  MapIndexed[{1, Sqrt[3]/2} Reverse@#2 - {(#2[[1]] - 1)/2, 0} -> # &, kb, {2}];

qwertyGraph = 
 VertexReplace[#, vLabels] & @ 
  NearestNeighborGraph[Keys @ vLabels, 
  VertexLabels -> Placed["Name", Center], 
  VertexSize -> Large, 
  VertexLabelStyle -> 14]

enter image description here

VertexComponent[qwertyGraph, "t", 1]
{"t", "f", "g", "r", "y", "5", "6"}
VertexComponent[qwertyGraph, "b", 1]
{"b", "v", "n", "g", "h"}
Select[DictionaryWordQ] @
 MapApply[StringJoin] @
  Tuples @  
   Map[VertexComponent[qwertyGraph, #, 1] &] @ 
{"hello", "brook", "jello"}


With the dictionary available and the task enunciated in the post mentioned in the comments:

adjChecker[k_String] := 
 And @@ (MemberQ[VertexComponent[qwertyGraph, #2, 1], #1] & @@@ 
    Partition[Characters@k, 2, 1])

Select[DictionaryLookup[], adjChecker] // SortBy[StringLength]

{"a", "I", "as", "dc", "er", "es", "lo", "oi", "re", "uh", "we",
"ass", "awe", "dew", "ere", "err", "ewe", "huh", "loo", "lop", "oik", \ "poi", "pol", "pop", "red", "saw", "see", "sew", "was", "wed", "wee", \ "WWW", "awed", "awes", "deed", "deer", "dews", "drew", "ewer", \ "ewes", "free", "kook", "loll", "look", "loop", "plop", "poll", \ "polo", "pool", "poop", "reds", "reed", "sass", "saws", "seed", \ "seer", "sees", "sere", "serf", "sews", "tree", "weds", "weed", \ "weer", "wees", "were", "asses", "deeds", "deres", "dress", "erred", \ "freed", "freer", "frees", "reeds", "resew", "rewed", "sawed", \ "seeds", "sered", "serer", "seres", "sewed", "sewer", "swede", \ "treed", "trees", "tress", "trews", "wases", "weeds", "assert", \ "assess", "deeded", "desert", "lollop", "qwerty", "redder", "redrew", \ "reseed", "resews", "reweds", "sassed", "sasses", "seeded", "seeder", \ "seesaw", "swedes", "wedded", "wedder", "weeded", "weeder", \ "dessert", "dressed", "dresser", "dresses", "redress", "reseeds", \ "resewed", "seesaws", "tresses", "assessed", "assesses", "reseeded", \ "rewedded", "seesawed", "redressed", "redresses"}

  • $\begingroup$ Thanks @kglr. The graph show upside down on v12.2.0 on Win7-x64. $\endgroup$
    – Syed
    Nov 4, 2023 at 16:14
  • $\begingroup$ @Syed, I am using "13.3.1 for Linux x86 (64-bit) (July 24, 2023)". Does removing Reverse from definition of kb fix the issue? $\endgroup$
    – kglr
    Nov 4, 2023 at 16:16
  • 1
    $\begingroup$ how about adding the option VertexCoordinates -> Keys[vLabels] in NearestNeighborGraph? $\endgroup$
    – kglr
    Nov 4, 2023 at 16:26
  • 1
    $\begingroup$ It is fixed now. The DictionaryWordQ application is concise and amazing. $\endgroup$
    – Syed
    Nov 4, 2023 at 16:38
  • 1
    $\begingroup$ @kglr: Ahhh... I see. There is a link in one comment of OP under OP question. - There is that requirement. $\endgroup$ Nov 4, 2023 at 17:07

Since I consider image recognizing code as a waste of time and typing keys of keyboard in sequence as pretty easy I produced the following code.

keyboard = 
    "Q W E R T Y U I O P\n A S D F G H J K L\n  Z X C V B N M", 
    "\n"];(*keys of keyboard separated and aligned with spaces*)
keys = DeleteCases[Flatten[keyboard], " " | "\n"];
coo = ({1, -2}*Reverse@First@Position[keyboard, #] & /@ keys);
gr = NearestNeighborGraph[coo, {All, 3}, 
   VertexLabels -> Thread[coo -> (Placed[#, Above] & /@ keys)]];
gr = VertexReplace[gr, Thread[coo -> keys]]
# -> AdjacencyList[gr, #] &@"Y"
# -> AdjacencyList[gr, #] &@"A"
# -> AdjacencyList[gr, #] &@"D"

enter image description here

(* "Y" -> {"T", "U", "G", "H"} *)

(* "A" -> {"Q", "W", "S", "Z"} *)

(* "D" -> {"E", "R", "S", "F", "X", "C"} *)


keyboard = 
    "1 2 3 4 5 6 7 8 9 0\n Q W E R T Y U I O P\n  A S D F G H J K L\n \
  Z X C V B N M", "\n"];

enter image description here

"Y" -> {"6", "7", "T", "U", "G", "H"}

"A" -> {"Q", "W", "S", "Z"}

"D" -> {"E", "R", "S", "F", "X", "C"}
  • $\begingroup$ I would appreciate a pointer on how to use it. $\endgroup$
    – Syed
    Nov 4, 2023 at 14:37
  • 1
    $\begingroup$ I just updated :-) $\endgroup$ Nov 4, 2023 at 14:39

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.