Doing same operation on multiple files

Question

Is there a way in mathematica to do same operation on multiple files at once? And preferably have the output in a table or list form?

I want to do the following operation preferably in few non-labor intensive steps as possible. (As before this, I did it allll by hand, pluggging in each number for each file. AND IT WAS NO FUN;;-( )

I have files, a001, ... a100, which contains two field(column) of numerical data that look like the following

2 4

5 6

3 7

...
..
.

For each file, I need to import/use the data in a variable in an equation and locate the global maximum which boundaries.

f:= Beta[2+a,4+b]Beta[5+a,6+b]Beta[3+a,7+b]......../(Beta[a,b]^n)]

To make the problem more complicated n is the number of rows in each file.

Next I would use NMaximize function with ristriction on 100>a>0, 100>b>0 like the following

 NMaximize[{Log[f], 100 > a > .1^10, 100 > b > .1^10}, {a, b}, 
 Method -> "NelderMead", MaxIterations -> 10000]

And if possible I would like to get the results of all NMaximize methods.

 NMaximize[{Log10[f], 100 > a > .1^10, 100 > b > .1^10}, {a, b}, 
 Method -> "NelderMead", MaxIterations -> 10000]

 NMaximize[{Log10[f], 100 > a > .1^10, 100 > b > .1^10}, {a, b}, 
 Method -> "DifferentialEvolution", MaxIterations -> 10000]

 NMaximize[{Log10[f], 100 > a > .1^10, 100 > b > .1^10}, {a, b}, 
 Method -> "SimulatedAnnealing", MaxIterations -> 10000]

 NMaximize[{Log10[f], 100 > a > .1^10, 100 > b > .1^10}, {a, b}, 
 Method -> "RandomSearch", MaxIterations -> 10000]

The wanted result would look something like this : ( header not necessary ) ( Method name not necessary, but I want 4 results for each file. )

filename Method a b

f1 NM 1 4

f1 DE .2 .5

f1 SA .1 .5

f2 DE .2 .3

f2 NM 1 4

f2 DE 3 4

f2 SA .2 .1

f2 DE .5 .6

...

..

.

PLEASE HELP. IS THIS IN ANYWAY POSSIBLE?

EDIT: SampleData

Answer 1

Here is my quick answer.

Import the data from the files:

files = FileNames["/sample_data/1_5ormore_methy_00*"];
data = Import[#, "Table"] & /@ files;

Define a function to compute your values:

f[s1_, s2_, vals_] := 
   Times @@ (Beta[First@# + s1, Last@# + s2] & /@vals)/Beta[s1, s2]^Length@vals

Apply the function to the data for each of the desired methods:

res = Table[
    NMaximize[{Log[f[a, b, #]], 100 > a > .1^10, 100 > b > .1^10}, {a,
       b}, Method -> m, MaxIterations -> 10], 
     {m, {"NelderMead", "DifferentialEvolution","SimulatedAnnealing", "RandomSearch"}}] & 
    /@ data

Extract just the values for a and b and reshape to a list of a,b pairs:

Flatten[{a, b} /. #[[All, 2]] & /@ res , 1]
Export["/sample_data/1_5ormore_methy_all.csv", %]

Answer 2

Of course this is possible. But I suggest that you break the problem down in smaller steps. Let's start with the problem of reading many files. If you have them in one directory, it can be as simple as

   files = FileNames["a*"];
   data = Import[#,"Table"] &/@ files;

This fills the list data with sub-lists, that contain the data n each file. Now you can create the expressions (no need to use functions, you can, if you want to, though) that you want to maximize

f = Function[u,Product[Beta[ #[[k,1]]+a, #[[k,2]]+b],{k,Length[ u] ]}]/Beta[a,b]^Length[u]] &/@ data

In simple words, this creates the 'Beta...Beta/Beta' function for one file. We assume that the data is stored in an array called u. We run through each row of the array (numbered k) and multiply everything (Product). Then we divide by the approriate power of the Beta-function (n is here simply the number of rows in the array u). We run this over all the top-level components of data (i.e. over all files), which is why this is a Function that is then mapped (/@) onto the data array. This gives an array of expressions f, one for each file.

And now you can maximize:

max=Table[ NMaximize[{Log[#], 100 > a > .1^10, 100 > b > .1^10}, {a, b}, Method -> m, MaxIterations -> 10000], {m,{"NelderMead", "DifferentialEvolution", "SimulatedAnnealing","RandomSearch"}}] &/@ f;

Run the maximizer of each component of f once for each method (this is what the Table does, it replaces m by the methods one by one), yielding an array with each of entry containing 4 solutions, one for each of the methods.

Now you just need to print them. The filename is still in files, so you want a table a bit like this

files[[1]], {a,b}/. max[[1,1]], {a,b}/. max[[1,2]], {a,b}/. max[[1,3]], {a,b}/. max[[1,4]]
.
.
.
files[[n]], {a,b}/. max[[n,1]], {a,b}/. max[[n,2]], {a,b}/. max[[n,3]], {a,b}/. max[[n,4]]

Use Map (or the /@ shorthand) to replace all four solutions and then a MapThread to simultaneously run through the files

 tt = MapThread[ {#1 , ({a,b}/. #2 ) } & ,{files, max}];

Now print it:

TableForm[ tt];

Or export it into a file for future use

Export[ tt, "myfile.dat", "Table"];

Done.