I view this as both a mathematics and a Mathematica question -- so apologies if it is thought I should have sent it alternatively to the mathematics stack exchange.
I want to perform the two-dimensional integration (or, possibly, reduce to a one-dimensional integration)
Integrate[Y^(-1 + d)
Hypergeometric2F1Regularized[d/2, -k, (2 + d)/
2, ((Y^2 - e^2 Subscript[r, 14]^2) (-1 + e^2 Subscript[r, 14]^2))/(
e^2 (Y^2 - Subscript[r, 14]^2) (-1 + Subscript[r, 14]^2))] (1/(
e Subscript[r, 14]))^(
1 + d) (1 + Y^2 (1 - 1/Subscript[r, 14]^2) - Subscript[r, 14]^2)^
k (1 - e^2 Subscript[r, 14]^2)^(
d/2) (-Y^2 + e^2 Subscript[r, 14]^2)^(d/2), {Subscript[r, 14], 0,
1}, {Y, e Subscript[r, 14]^2, e Subscript[r, 14]},
Assumptions -> d >= 1 && k >= 0 && 0 < e <= 1]
So, $ d $ and $ k $ are parameters, and $ Y $ and $ r_{14} $ the variables of integration, with $ e $ being a free variable.
In $ \mathrm\TeX $, the integrand is the product of
\begin{equation} Y^{d-1} \left(\frac{1}{r_{14} \epsilon }\right){}^{d+1} \left(1-r_{14}^2 \epsilon ^2\right){}^{d/2} \left(\left(1-\frac{1}{r_{14}^2}\right) Y^2-r_{14}^2+1\right){}^k \left(r_{14}^2 \epsilon ^2-Y^2\right){}^{d/2} \end{equation}
and
\begin{equation} \, _2\tilde{F}_1\left(\frac{d}{2},-k;\frac{d+2}{2};\frac{\left(r_{14}^2 \epsilon ^2-1\right) \left(Y^2-r_{14}^2 \epsilon ^2\right)}{\left(r_{14}^2-1\right) \epsilon ^2 \left(Y^2-r_{14}^2\right)}\right) . \end{equation}
The two-dimensional domain of integration is
\begin{equation} r_{14} \in [0,1], \hspace{.25in} Y \in [\varepsilon r_{14}, \varepsilon^2 r_{14}] . \end{equation}
For $ k=0 $, the integral evaluates (as can be confirmed by setting $ d $ to a positive integer -- even integers evaluate more readily) to
1/4 e^(-1 + d)
Gamma[d/2] Gamma[
d] HypergeometricPFQRegularized[{-(d/2), d/2, d}, {1 + d/2,
1 + (3 d)/2}, e^2]
That is, \begin{equation} \frac{1}{4} \epsilon ^{d-1} \Gamma \left(\frac{d}{2}\right) \Gamma (d) \, _3\tilde{F}_2\left(-\frac{d}{2},\frac{d}{2},d;\frac{d}{2}+1,\frac{3 d}{2}+1;\epsilon ^2\right). \end{equation}
I've been trying all possible integration-by-parts combinations with no success to this point in time.
For even, nonnegative $ d $ and nonnegative $ k $, the integrand evaluates to a polynomial in $ e $. For odd $ d $, logs and polylogs appear.
The question stated here pertains to the issue discussed in sec. IX.B of my posting, Qubit-qudit separability/PPT-probability investigations, including Lovas-Andai formula advancements, of finding an "extended Lovas-Andai master formula", denoted there by $ \tilde{\chi}_{d,k}(\varepsilon) $.