It seems that set is the index you need for your $k_i$ in your model. If this is the case then you don't need Which.
ClearAll[fitmodel];
fitmodel[set_, k_List, m_, n_][t_] := model[set][k[[set]], m, n][t]
fitmodel could then be called with as many $k_i$ as needed as they are contained in a list. You just need to add additional $k_i$ to the list.
kVars = {k1, k2, k3, k4, k5};
fitmodel[s, kVars, m, n][t]
Hope this helps.