So I can show the pic:
CurrentValue[{StyleDefinitions, "Text", CellMargins}]
{{66, 10}, {7, 8}}
CurrentValue[{StyleDefinitions, "Text", CellFrameMargins}]
8
Cell[
CellGroupData[
{
Cell[
"top",
"Text",
FontColor -> White,
Background -> Pink,
CellMargins -> {{66, 10}, {0, 0}},
CellFrameMargins -> {{8, 8}, {7, 8}}
],
Cell[
"bottom",
"Text",
FontColor -> White,
Background -> Pink,
CellMargins -> {{66, 10}, {0, 0}},
CellFrameMargins -> {{8, 8}, {7, 8}}
]
},
Open
]
] // CellPrint

With different colors
Cell[
CellGroupData[
{
Cell[
"top",
"Text",
FontColor -> White,
Background -> Pink,
CellMargins -> {{66, 10}, {0, 0}},
CellFrameMargins -> {{8, 8}, {7, 8}}
],
Cell[
"bottom",
"Text",
FontColor -> Black,
Background -> LightBlue,
CellMargins -> {{66, 10}, {0, 0}},
CellFrameMargins -> {{8, 8}, {7, 8}}
]
},
Open
]
] // CellPrint

This obviously admits a simple stylesheet solution too
Per the comment, Mathematica doesn't resize content in the CellFrameLabels
(you can look up on the site the many, many attempts to make that content resize properly). This is a problem with numbered equations. There are three straightforward solutions.
The first is less flexible but works with documents where the cell heights can be parametrized
Cell[CellGroupData[{
Cell["blah", "DisplayFormulaNumbered", FontColor -> White,
Background -> Pink,
CellMargins -> {{66, 3}, {0, 0}},
CellFrameMargins -> {{8, 8}, {4, 4}},
CellFrameLabelMargins -> 0,
CellFrameLabels -> {
{None,
Cell[
TextData[{"(", CounterBox["DisplayFormulaNumbered"], ")"
}], "DisplayFormulaEquationNumber",
CellMargins -> {{0, 0}, {0, 0}},
CellFrameMargins -> {{5, 5}, {4, 4}},
TextAlignment -> {Center, Center},
CellSize -> {Automatic, 26}
]
},
{None, None}
}
],
Cell["top", "FigureCaption", FontColor -> Black, Background -> LightBlue,
CellMargins -> {{66, 3}, {0, 0}},
CellFrameMargins -> {{8, 8}, {4, 4}}
]
}, Open]
] // CellPrint

You'll notice it spans fine, but this is fragile because I hard-coded the height and it also floats oddly to the top-right corner and this is basically unfixable.
The second option is to remove the background on the labels and make sure the label widths are identical
Cell[CellGroupData[{
Cell["blah", "DisplayFormulaNumbered", FontColor -> White,
Background -> Pink,
CellMargins -> {{66, 3}, {0, 0}},
CellFrameMargins -> {{8, 8}, {4, 4}},
CellFrameLabels -> {
{None,
Cell[
TextData[{"(", CounterBox["DisplayFormulaNumbered"], ")"}],
"DisplayFormulaEquationNumber",
FontColor -> Pink,
Background -> None,
CellSize -> {25, Automatic},
CellMargins -> {{0, 0}, {0, 0}}
]
},
{None, None}
}
],
Cell["top", "FigureCaption", FontColor -> Black, Background -> LightBlue,
CellMargins -> {{66, 3}, {0, 0}},
CellFrameMargins -> {{8, 8}, {4, 4}},
CellFrameLabels -> {
{None,
Cell[
"",
"DisplayFormulaEquationNumber",
CellSize -> {25, Automatic},
Background -> None,
CellMargins -> {{0, 0}, {0, 0}}
]
},
{None, None}
}
]
}, Open]
] // CellPrint

This is a much more robust solution. It is worth noting that the width is hard coded.
A final possibility, again, where the label width is hard coded, is to use a negative value for CellFrameLabelMargins
. To do this, again, both the upper and lower cells need a label.
Cell[CellGroupData[{
Cell["blah", "DisplayFormulaNumbered", FontColor -> White,
Background -> Pink,
CellMargins -> {{66, 3}, {0, 0}},
CellFrameMargins -> {{8, 8}, {4, 4}},
CellFrameLabelMargins -> -20
],
Cell["top", "FigureCaption", FontColor -> Black, Background -> LightBlue,
CellMargins -> {{66, 3}, {0, 0}},
CellFrameMargins -> {{8, 8}, {4, 4}},
CellFrameLabelMargins -> -20,
CellFrameLabels -> {
{None,
Cell[
"",
"DisplayFormulaEquationNumber",
Background -> None,
CellMargins -> {{0, 0}, {0, 0}}
]
},
{None, None}
}
]
}, Open]
] // CellPrint

If you're interested in using these cells, make sure the starting content will allow you to type what you want (i.e. ""
for text, BoxData[""]
for code, and BoxData@FormBox["", TraditionalForm]
for equations). Here's an example of this with equations
Cell[
CellGroupData[{
Cell[
BoxData@FormBox["", TraditionalForm],
"DisplayFormulaNumbered",
TextAlignment -> Center, FontColor -> White, Background -> Pink,
CellMargins -> {{66, 3}, {0, 0}}, CellFrameMargins -> {{8, 8}, {4, 4}},
CellFrameLabelMargins -> -20],
Cell["top", "FigureCaption", FontColor -> Black, Background -> LightBlue,
CellMargins -> {{66, 3}, {0, 0}}, CellFrameMargins -> {{8, 8}, {4, 4}},
CellFrameLabelMargins -> -20,
CellFrameLabels -> {{None,
Cell["", "DisplayFormulaEquationNumber", Background -> None,
CellMargins -> {{0, 0}, {0, 0}}]}, {None, None}}]},
Open]
] // CellPrint
then I can type math in the top cell

Advanced>Open Option Inspector>Notebook Options>Display Options>Background
that changes the whole page background. But it sounds like you just want it behind the certain cells. Maybe there's an option to reduce the padding at the top and bottom of these special cells to 0, so adjacent cells appear as one long cell? $\endgroup$CellMargins -> {{r, l}, {0, 0}
andCellFrameMargins->{{0, 0}, {bottom, top}}
which visually merges the two but retains the padding $\endgroup$