How to detect and split sections of an image

Question

I'm new to Mathematica and I was trying to split the following image into horizontal sections, just above every square box. I have done some pre-processing but I can't figure out how to "locate" the boxes and split them at their topmost boundaries, until the next box's topmost boundary.

EDIT - 2 ( Better info about previous edits can be gleamed from the accepted answer )

This image has the proper text segment I wish to extract, I just need to create a corpus with the sentences with the corresponding boxes, which I'm choosing as a guide for comp. vision.

I need to chunk it to components segments

1. The sentence lengths and thus their spacing on a page is variable.
2. There are variable number of such sentence groups in a page. ( As shown in Full Page )

Could you help me out?

Answer 1

Update 2:

We'll work this case by first finding the most common left-most pixel. Note that this won't work for the previous update, as that vertical separator will be it. However it should be possible to detect and strip that vertical separator.

img = Import["https://i.sstatic.net/APXIy.png"];

norgba = ColorCombine@Take[ColorSeparate@img, UpTo[3]];
binarized = Binarize@norgba;
data = ImageData@binarized;

leftmosts =
  FirstPosition[#, 0, Nothing] & /@ data // Counts // ReverseSort // 
   KeyMap[First];

blockstart = First@Keys@leftmosts;

Then we find the maximum contiguous block of black pixels (in our binarized image) starting from there.

contiguous =
  Split[
   Position[#, 0, {1}] & /@ data // Flatten // Sort // 
    DeleteDuplicates,
   Abs[# - #2] == 1 &
   ];

pixelBlock = SelectFirst[contiguous, MemberQ[blockstart]];

Then we'll apply the same row splitting algorithm, but this time allowing much longer stretches of white pixels (so that the wrapped lines are captured). Then we'll find the same set of rows defined by these blocks, then select out those that don't have a marker:

splits =
  Block[{block = False, counts = 0, countmax = 50}, 
   SplitBy[data, (If[Length@Counts[#] > 1, block = True;
       counts = 0;, counts++;
       block = (block && counts <= countmax)];
      block) &]];

rowsRaw = 
  Partition[Riffle[Most@#, Rest@#] &@Accumulate[Length /@ splits], 2];

rows =
  Select[
   rowsRaw,
   MemberQ[
     Take[data, #][[All, pixelBlock]] // Flatten,
     0
     ] &
   ];

Finally we take from the maximum right position of a marker and crop out whitespace:

fullcolordata = ImageData@img;

chunks =
  ImageCrop@Image@
      Take[fullcolordata, #][[All, Last@pixelBlock + 1 ;; -1]] & /@ 
   rows;

chunks

Update

This is for the updated version of the question.

First we need to get an appropriate image to work on. First we'll binarize and then take only the chunk to the left of the vertical divider and below the horizontal one:

img = Import@"https://i.sstatic.net/s8eZx.png";

norgba = ColorCombine@Most@ColorSeparate@img;

binarized = Binarize@norgba;
data = ImageData@binarized;
flipped = Transpose@data;
rowseparator =
  Max@
   Position[
    data,
    Alternatives @@ MinimalBy[data, Total],
    {1}
    ];
rowdropped = 
 Take[data, {
  rowseparator + 5 (* a fudge factor could be much bigger, too*), 
  -1}];
columnseparator =
  Min@
   Position[
    flipped,
    Alternatives @@ MinimalBy[flipped, Total],
    {1}
    ];

chopped =
  Transpose@Take[Transpose@rowdropped, columnseparator - 1];

Take[chopped, 1000] // Image

Then we split as before:

splits =
  Block[{block = False, counts = 0, countmax = 3},
   SplitBy[chopped,
    (If[Length@Counts[#] > 1, block = True;
       counts = 0;, counts++;
       block = (block && counts <= countmax)];
      block) &]];

Then we need the row spans each of the chunks is in (rows) and the vertical range that the indicator is in (subblocks):

rows =
  Partition[
   Riffle[Most@#,
      Rest@#] &@
    Accumulate[
     Length /@ splits
     ],
   2
   ];

subblocks =
  If[Length@# > 0,
       {0, 10} + Through[{Min, Max}[Keys@#]],
       None] &@
     (Function[Through[{Min, Max}[#]]] /@

       GroupBy[First -> Last]@Position[#, 0]
      ) & /@ splits;

Then we thread over these together, first taking the row range, then taking the subrange:

fullcolordata = ImageData@img;

chunks =
  MapThread[
   Take[
     Take[fullcolordata, rowseparator + #],
     #2
     ] &, {
    rows,
    DeleteCases[subblocks, None]
    }];

And then we view:

ImageCrop@Image[#, ColorSpace -> "RGB"] & /@ chunks

And if you just want the line text you can use this:

fullcolorrowchopped =
  Transpose@
   Take[
    Transpose@Take[fullcolordata, {rowseparator + 5, -1}],
    {columnseparator + 5, -1}
    ];

fchopped =
  Select[
   MapThread[
    Take[
      Take[fullcolorrowchopped, #],
      #2
      ] &, {
     rows,
     DeleteCases[subblocks, None]
     }],
   Length@Counts[#] > 1 &
   ];

ImageCrop@Image[#, ColorSpace -> "RGB"] & /@ fchopped

Original

So we can do this by looking at the pixel values and just splitting where we see whitespace.

First get the image and its pixel rows:

img = Import["https://i.sstatic.net/TPX7R.png"];
data = ImageData@img;

Then we'll use a procedural style loop to chunk our data by the following criterion:

Either 1. There's more than a single color pixel in it

Or 1. Up to n rows previous there was more than a single pixel color

Here's my loop. It's not optimized and a pretty lazy, off the top of my head way to do it, but hopefully it's good enough:

splits =
  Block[{
    block = False,
    counts = 0,
    countmax = 3
    },
   SplitBy[data,
    (
      If[Length@Counts[#] > 1,
       block = True;
       counts = 0;,
       counts++;
       block = (block && counts <= countmax)
       ];
      block
      ) &
    ]
   ];

Then we'll crop whitespace and format images:

chunks = ImageCrop@*Image /@ splits;

And then delete those images that are just whitespace:

imgs =
  Select[chunks,
   Length@Counts[Flatten[ImageData[#], 1]] > 1 &
   ];

And let's see how we did:

imgs[[;; 10]]

There's probably a cleaner way to do this, but this was the first thing that came to mind.

Answer 2

To extract the sentences, you could use TextRecognize if the text is what you really want.

In:

img = Import["https://i.sstatic.net/s8eZx.png"] // Rasterize;
ts = Block[{$Language = #}, 
     TextRecognize[img]] & /@ {"ChineseTraditional", "Chinese"};
ts // Map[StringSplit[#, "\n"] &] // Map[MatrixForm]
Transpose[{ts, {{5, 12}, {8, 15}}}] // 
 Map[StringSplit[First@#, "\n"][[Last@#]] &]

Out:

{{"他在廚房 。 他在煮束西 口", 
  "你踩 到 我的 腳 了 。\[LongDash] 對不起"}, 
{"他在厨房 \[Degree] 他正在做饭 。", 
  "你踩着我脚了 口 一对不起 口"}}