# Finding a fit to a multi-dimensioned function

I have a model function $f:\mathbb{R}^2\rightarrow\mathbb{R}^2$, and a bunch of data points for which I'd like Mathematica to fit for me. Unfortunately FindFit seems to only deal with functions $\mathbb{R}^n\rightarrow\mathbb{R}$. I guess I could make my own square difference from the data to the model and use NMinimize on that, but I wondering if there was an easier way?

### Edit - toy example with answers

$g\left(x,y\right)=\left<abx+(2c+d)y,(2a+b)x+cdy\right>$

g[{x_, y_}, {a_, b_, c_, d_}] := {a b x + (2 c + d) y, (2 a + b) x + c d y};
points = Flatten[Table[{{x, y}, g[{x, y}, {1, 2, 3, 4}]},
{x, -1, 1, 0.5}, {y, -1, 1, 0.5}], 1];


Finding a fit for each component separately doesn't work:

data1 = points /. {{x_, y_}, {u_, v_}} -> {x, y, u};
data2 = points /. {{x_, y_}, {u_, v_}} -> {x, y, v};
FindFit[data1, g[{x, y}, {a, b, c, d}][[1]], {a, b, c, d}, {x, y}]
FindFit[data2, g[{x, y}, {a, b, c, d}][[2]], {a, b, c, d}, {x, y}]


{a -> 0.729723, b -> 2.74077, c -> 3.35659, d -> 3.28681}

{a -> 1.48555, b -> 1.02891, c -> 2.81936, d -> 4.25629}

Jens suggestion works well:

data3 = points /. {{x_, y_}, {u_, v_}} -> Sequence[{x, y, 0, u}, {x, y, 1, v}];
FindFit[data3, g[{x, y}, {a, b, c, d}].{1 - s, s}, {a, b, c, d}, {x, y, s}]


{a -> 1., b -> 2., c -> 3., d -> 4.}

My original suggestion (which is similar to user840's links)

error = Plus @@ (points /.
{{x_, y_}, {u_, v_}} -> Norm[g[{x, y}, {a, b, c, d}] - {u, v}]^2);
NMinimize[error, {a, b, c, d}]


Gives a different, but still correct, solution (or at least a close approximation thereof)

{1.41754*10^-13, {a -> 1.00004, b -> 1.99993, c -> 2., d -> 6.}}

-
How about using NonlinearModelFit? Take a look into the documentation and the examples. You will see that under Scope the first example shows how one can fit a model of more than one variable. If you provide example data, it would be possible to help you more detailed. – partial81 May 9 '12 at 5:53
@partial81 More than one variable, not but not a vector value. That example is for $\mathbb{R}^2 \rightarrow \mathbb{R}$ while the OP wants $\mathbb{R}^2 \rightarrow \mathbb{R}^2$. NonlinearModelFit doesn't seem to support this. – Szabolcs May 9 '12 at 8:11
ok, I assume you are right. I would love to try it with NonlinearModelFit, but without example data, I do not want to spend more time on this problem. – partial81 May 9 '12 at 10:54
Why not two fits, one for each dependent variable? – Daniel Lichtblau May 9 '12 at 18:30
@DanielLichtblau Each variable depends on all the model parameters, so I end up with different parameter values for the different fits. – wxffles May 9 '12 at 21:55

Without a specific example, I can only make general suggestions here.

The first thing that comes to mind is that you could fit each of the two components of your vector field independently. I.e., if $\vec{f}: \mathbb{R}^2\to\mathbb{R}^2$ is split up into $\vec{f} = \{f_x, f_y\}$ with $f_{1,2}: \mathbb{R}^2\to\mathbb{R}$, then FindFit would work on each of these component functions.

If you don't want the fits to be determined independently, it could still be possible to use FindFit by introducing an auxiliary variable $s$ that labels the two component functions above, $f_s$, and then provide the fitting data with this variable s included. Here, s can only have two discrete values (I borrowed the idea from spin-half quantum mechanics). Let's choose s = 0 for the function $f_x$ and s = 1 for $f_y$. So you'd have data in the form

data = {{x1, y1, 0, fx1}, {x1, y1, 1, fy1}, ...}


where fx1 is the first value of $f_x$ and fy1 the first value of $f_y$, both at the point x1, y1.

Your model could then look something like this:

model[x_, y_, s_] := modely[x, y]*s + modelx[x, y]*(1 - s)


where modely[x, y] is the model for $f_y$ and modelx[x, y] is the model for $f_x$. The variables in the FindFit call would be x, y, s, and the parameters in the models modelx, modely could be the same (or different).

There are probably many other reasonable ways to do such a fit, but these are some ideas.

-
Thanks. This is working well for my problem. FindFit[data /. {{a_, b_}, {p_, q_}} -> Sequence[{a, b, 0, p}, {a, b, 1, q}], model.{1 - s, s}, ...] – wxffles May 9 '12 at 21:33
This is a very nice example, I'm also trying to fit to a vector field, and this is much easier than trying to build a multivariate regression function. – wil3 Nov 18 '15 at 3:30

You can check Fitting Multiple-Response Data from Bruce Miller. He gives a step-by-step example of how to perform a simultaneous fit to multiple functions, and provides a fairly general function for performing such fits.

Alternately, Multiple-Response Fitting notes from wolframThis package contains functions for simultaneously fitting multiple functions to data with a possibly shared set of parameters. It has flexible output options and the first part of the accompanying demonstration notebook can serve as an introduction to writing one's own fitting codes using FindMinimum.

-

Multi-layer perceptrons have this ability. The Neural network package from Wolfram offers this functionality.

Neural Networks

-
Since that isn't part of the standard suite of packages, could you provide a link? – rcollyer May 9 '12 at 13:14
True, but now you have the additional problem of structuring the neural net appropriately (which is by no means trivial if you don't have any experience with neural nets). And after that you have to train it, which is equivalent to the original problem... – Oleksandr R. May 9 '12 at 13:46
Yes there is some effort involved, but it offers a different approach to the solution, which I thought might fill in parts of the landscape of the large selection of approximation techniques that might be deployed outside of the half a dozen or so that FindFit offers and MLPs have inherent support for output vectors in higher dimensions. – image_doctor May 9 '12 at 15:35
The URL link is broken. – shrx Feb 13 '15 at 8:44