How can I fix Mathematica's text line-breaking algorithm not to line-break so soon?

Question

I have some custom plotting functions that simplify the placement of footnotes and source notes under the plot. It is important that the line length of the footnotes fills up the available space and lines up neatly with the plot, which has a known, fixed width.

Below is a somewhat simplified version of my code for creating a grid containing these notes, with some Frames added so you can see the relationship of the length of the text lines to the width of the Grid cells.

My problem is that, no matter what I do with LinebreakAdjustments, the lines in the footnote text break far too "soon", and leave far more whitespace than one would see in any other program that lays out text in a text box. How can I fix this? It's the sort of thing my persnickety user base will consider a show-stopper.

Options[SourceNote] = {FootnoteMargin -> 5.7, NoteFontSize -> 14, 
   LeftCellSize -> 10, TotalSize -> 45, NotesMargin -> 0.8};
Options[GraphNotesGrid] = {FootnoteMargin -> 5.7, NoteFontSize -> 14, 
   LeftCellSize -> 10, TotalSize -> 45, NotesMargin -> 0.8};

SourceNote[x : {__String}, opts : OptionsPattern[]] := 
  With[{fnm = OptionValue[FootnoteMargin], fs = OptionValue[NoteFontSize]}, 
    If[Length[x] ==  1, 
 {DisplayForm[AdjustmentBox[Style["Source:", fs, FontFamily -> "Arial", Black], 
            BoxMargins ->  {{fnm, 0}, {0, 0}}]], 
        Style[StringTrim[x[[1]] ], fs, FontFamily -> "Arial", Black]},
 {DisplayForm[AdjustmentBox[Style["Sources:", fs, FontFamily -> "Arial", Black], 
            BoxMargins ->  {{fnm, 0}, {0, 0}}]],
 (* delete empty strings AFTER counting length to allow "Sources: company reports" case *)
   Style[ StringJoin @@ Riffle[ Union[DeleteCases[ StringTrim /@ x, ""]], "; "], fs, 
          FontFamily -> "Arial", Black], " "}] ]

GraphNotesGrid[cf_?MatrixQ, cs : {__String}, 
   opts : OptionsPattern[]] /; Dimensions[cf][[2]] == 2 := 
 With[{fnm = OptionValue[FootnoteMargin], 
   fs = OptionValue[NoteFontSize], l = OptionValue[LeftCellSize], 
   ts = OptionValue[TotalSize], topm = OptionValue[NotesMargin]}, 
  Grid[Join[(Map[{DisplayForm[AdjustmentBox[
       Style[#[[1]], fs, FontFamily -> "Arial", Black], 
       BoxMargins -> {{fnm, 0}, {0, 0}}]], 
       Framed@Pane[ Style[#[[2]], fs, Black, LineIndentMaxFraction -> 0, 
        LinebreakAdjustments -> {1.2, 200, 12, 1, 500}, (* I tried playing with this *)
        FontFamily -> "Arial"], (ts - l)* CurrentValue["FontMWidth"]]} &, cf]), 
      {SourceNote[cs]}], (* real version does do FilterRules*)
   Spacings -> {{0, {}, 0}, {topm, {}, 0}}, 
   ItemSize -> {{l, ts - l}, Automatic}, Alignment -> {Left, Top}, 
   Frame -> All, FrameStyle -> Blue] ]

Here is an example:

GraphNotesGrid[{{"*", 
   "This is a test footnote that is designed to go over several lines \
and demonstrate the crappy line-breaking algorithm. This might or \
might not be long enough. We shall see. "}, {"**", 
   "This is another very long footnote which you do not need to read \
but do need to notice the line-breaking"}}, {"ABC", "CBS", "NBC"}]

Answer 1

From the documentation,

LinebreakAdjustments is specified as a list of five numbers: LinebreakAdjustments->{a, b, c, d, e}. Here, a specifies the optimal line width to aim for, expressed as a percentage of the total line width; b is the penalty assigned for deviating from the line width specified by a; c is a penalty based on the depth of the box tree (the deeper in the box tree the line break would occur, the greater the penalty for line breaks); d is a penalty based on how bad the break was at the previous line; e is a penalty based on the total number of lines (the larger the number d, the smaller the number of lines used to format the expression).

The key seems to be in the parenthetical phrase in the last line: the effectiveness of e, the penalty for number of lines, depends on the size of d, a penalty based upon how bad the previous line break was. So, they both need to be large to work effectively.

Both

LinebreakAdjustments -> {1, 2, 12, 500, 500}

which is based off of the documentation example, and

LinebreakAdjustments -> {1.2, 200, 12, 500, 500}

based off of yours gives the reasonable result

The first one looks a little worse, but that is because "demonstrate" is to large for the line.