I am trying to calculate a moving-window of compounded returns on a large 2D-array. (ie some kind of MovingMap on geometric returns)
The Rows are time-steps and the Columns are different financial instruments. The array contains returns at that time-step.
I want to get back a 2D array of compounded returns that has the same size/dimensions as the 2D array input.
The returns array I am working with has some Missing[]
data as not all instruments start at the same time
The method I have come up with doesn't feel as fast as it can be and I would like to optimize it.
Before we start, this is an example of calculating compound returns on a single financial instrument.
compoundReturns[returns_List, window_Integer] :=
Module[
{
movingWindow = Partition[returns + 1., window, 1],
unpaddedReturns
},
unpaddedReturns = Times@@@movingWindow - 1;
PadLeft[unpaddedReturns, Length[returns]]
];
So
window = 3;
returns = {0.0634779`, -0.0624394`, 0.00844932`,
0.0400948`, -0.0154299`};
compoundReturns[returns, 3]
{0, 0, 0.00549958, -0.0166087, 0.0326987}
I would like to perform this calculation on a list of financial instruments, preferably in parallel or threaded.
Let's make some small data to work with:
SeedRandom[1];
$rows = 5;
$cols = 3;
$return = RandomReal[{-0.1, 0.1}, {$rows, $cols}];
(*make data with missing*)
$return[[2, 2]] = Missing[];
$return[[1, 2 ;; 3]] = Missing[];
$return
{{0.0634779, Missing[], Missing[]}, {-0.0624394, Missing[], -0.0868522}, {0.00844932, -0.0537691, -0.0207988},
{0.0400948, -0.0576348, 0.0497314}, {-0.0154299, -0.050501, 0.0954344}}
Given that this type of problem is entirely numerical my first instinct is to Compile
it. I convert the Missing[]
data in $return
to 0.
so that it can be handled by Compile
.
$returnWithZero = $return /. Missing[] -> 0.;
I keep track of when the data starts
$dataStart = Flatten[FirstPosition[#, _Real] & /@ (Transpose@$return)]
{1, 3, 2}
compoundReturnArrayC = Compile[
{
{array, _Real, 2},
{window, _Integer},
{startRows, _Integer, 1}
},
With[
{
$totalColumns = Length[startRows],
$totalRows = Length[array],
returnArrayByInstruments = Transpose[array] + 1.
},
Module[
{
returnsByInstrument,
returnsByInstrumentShortened,
windows, compoundedReturns,
compoundedReturnsByInstrument,
startRow
}
,
compoundedReturnsByInstrument =
Table[
returnsByInstrument = returnArrayByInstruments[[c]];
startRow = startRows[[c]];
returnsByInstrumentShortened =
returnsByInstrument[[startRow ;; $totalRows]];
windows = Partition[returnsByInstrumentShortened, window, 1];
compoundedReturns = Fold[Times, #] & /@ windows;
PadLeft[compoundedReturns - 1., $totalRows]
,
{c, $totalColumns}
];
Transpose[compoundedReturnsByInstrument]
]
]
,
CompilationTarget -> "C",
Parallelization -> True
];
So in our small example we get the correct result
compoundReturnArrayC[$returnWithZero, window, $dataStart]
{{0., 0., 0.}, {0., 0., 0.}, {0.00549962, 0., 0.}, {-0.0166087, 0., -0.061377}, {0.0326987, -0.153336, 0.125995}}
However for large arrays, increasing the window size slows down the code quite a bit.
SeedRandom[1];
$rows = 10000;
$cols = 100;
$return = RandomReal[{-0.1, 0.1}, {$rows, $cols}];
$return[[2, 2]] = Missing[];
$return[[1, 2 ;; 3]] = Missing[];
$dataStart = Flatten[FirstPosition[#, _Real] & /@ (Transpose@$return)];
$returnWithZero = $return /. Missing[] -> 0.;
compoundReturnArrayC[$returnWithZero,
500, $dataStart]; // RepeatedTiming (*{3.19958, Null}*)
compoundReturnArrayC[$returnWithZero,
5, $dataStart]; // RepeatedTiming (*{0.209141, Null}*)
How can I increase the speed of this pretty trivial calculation? I feel there is some way to operate on all the lists in parallel, and then padding the final outcome with zeroes in the correct place, instead of looping through with table.
Edit
For reference, the fastest ways I have found to do a moving-map of Times
to a list:
SeedRandom[1];
rng = RandomReal[{0.95, 1.05}, 10000];
window = 1000;
a = MovingMap[Apply[Times], rng, Quantity[window, "Events"]]; // RepeatedTiming
b = Times @@@ Partition[rng, window, 1]; // RepeatedTiming
c = Apply[Times][Transpose@Partition[rng, window, 1]]; // RepeatedTiming
d = Fold[Times, Transpose@Partition[rng, window, 1]]; // RepeatedTiming
{0.679459, Null}
{0.858843, Null}
{0.12836, Null}
{0.144564, Null}