1
$\begingroup$

I have a long table of data (the number of rows 795,000) and I need to create a code in Mathematica that can calculate the minimum value for every 5000 rows and then I need to know the positions of these minima.Is there any way to do this?

$\endgroup$
1
  • $\begingroup$ Do you need something like this (76121), i.e the sequential running minima? Or must it be by chunks? $\endgroup$
    – rhermans
    Commented May 15, 2017 at 9:28

2 Answers 2

3
$\begingroup$

First, you split yuor big data into chunks of size 5000 rows you will have a matrix like this:

 chunkOfMatrix = Table[RandomReal[], {i, 1, 5000}, {j, 1, 100}];

Then applying this will give the Min and its position in each row:

{Min[#], Position[#, Min[#]]} & /@ chunkOfMatrix

Hope this will help

$\endgroup$
6
  • $\begingroup$ Thank you for your answer, but please can you explain more if my data is already as a matrix and its name is data, how can I use your command? $\endgroup$
    – Ghady
    Commented May 12, 2017 at 14:27
  • $\begingroup$ Welcome, so if your data is smth. like this : data = Table[RandomReal[], {i, 1, 150000}, {j, 1, 100}]; and you can partition this data into chunks of 5000 rows using Partition function: dataChunks = Partition[data, 5000]; and then use the code above for every chunk. For example for the first one would be {Min[#], Position[#, Min[#]]} & /@ dataChunks[[1]]. Did I get you right? $\endgroup$ Commented May 12, 2017 at 15:07
  • $\begingroup$ I tried your way, but I didn't get what I need. For more details my data is below $\endgroup$
    – Ghady
    Commented May 12, 2017 at 17:56
  • $\begingroup$ Sorry, my data is huge! $\endgroup$
    – Ghady
    Commented May 12, 2017 at 18:05
  • $\begingroup$ Ohh, in this way you maybe need to write your data to a file and read it by portions. $\endgroup$ Commented May 12, 2017 at 19:54
1
$\begingroup$

If you are looking for a running minima, then the answer already exists here. If you need it by chunks, then

Some data as an example (you use your own)

data = RandomReal[1, 795000];

The function

MinPositionAndValueByChunks[data_List, chunks_Integer] := MapIndexed[
  {First@FirstPosition[#1, Min[#1]] + chunks (First[#2] - 1), 
    Min[#1]} &,
  Partition[data, chunks]
  ]

Example

Short@MinPositionAndValueByChunks[data, 5000]
{{2042,0.0000435445},{6324,0.000265437},<<155>>,{787796,0.000635954},{793848,0.000139004}}
$\endgroup$
2
  • $\begingroup$ I tried to use your command, but my data has specific numbers. It is the result of another code that computes the differential cross section, so do you think I can use RandomReal.? I should get stability minima. $\endgroup$
    – Ghady
    Commented May 16, 2017 at 0:27
  • 1
    $\begingroup$ @Ghady You must be kidding, the random data I offer is just for testing. For the benefit of the people that do not have access to your data, as you did not share it. $\endgroup$
    – rhermans
    Commented May 18, 2017 at 8:12

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.