3
$\begingroup$

I have many text files numbered from 1 to 1 million in numerical order. Within each text file is 5 columns of data (TAB spaced) with many rows. I effectively want to take the minimum value from each column from each file and then plot them on a graph in order of the file name so that i can see how the minimum value of each column changes as the file name increases. This is my current code

This seems to work for me however it is plotting the results in the wrong order as i think it is reading the file names not in a numerical order but numerical order of each character so this order 1,10,1000,10000,2,20,30. Which is muddling up the order of the alldata matrix i'm creating. I am pretty new to mathematica so i apologise if anything i have done seems an odd way of going about it. My second issue is speed somewhere i think it is changing the values to floating point numbers with many .0000s which is slowing down the process.

My files are named 1.txt, 2.txt, ect...

Thank you for your time.

$\endgroup$
3
  • 1
    $\begingroup$ Control the order that the file names are listed in using Join[FileNames[RegularExpression["[0-9].txt"]], FileNames[RegularExpression["[0-9][0-9].txt"]], FileNames[RegularExpression["[0-9][0-9][0-9].txt"]], FileNames[RegularExpression["[0-9][0-9][0-9][0-9].txt"]], FileNames[RegularExpression["[0-9][0-9][0-9][0-9][0-9].txt"]], FileNames[RegularExpression["[0-9][0-9][0-9][0-9][0-9][0-9].txt"]]] $\endgroup$
    – Bill
    Feb 13, 2016 at 18:28
  • 1
    $\begingroup$ the cleanest way i think is something like SortBy[FileNames[..],ToExpression[First@StringCases[#,__~~n:NumberString~~__:->n]]&] (sorry cant test here but that should be close ) $\endgroup$
    – george2079
    Feb 13, 2016 at 19:19
  • $\begingroup$ Why did you delete your code? $\endgroup$
    – user1722
    Feb 13, 2016 at 20:19

1 Answer 1

3
$\begingroup$

The problem here stems from the filenames being in alphabetical order. If you separate the numbers from the ".txt" suffix with StringSplit, you can use ToExpression to get Mathematica to treat them as numbers and put them in numerical order. Then use Ordering to put the original list of filenames in the same order as the list of numbers.

filelist = {"1.dat", "10.dat", "2.dat", "100.dat", "20.dat",
   "3.dat", "1000.dat"}

{"1.dat", "10.dat", "2.dat", "100.dat", "20.dat", "3.dat", "1000.dat"}

numberlist = 
 ToExpression[StringSplit[filelist, ".dat"][[All, 1]]]

{1, 10, 2, 100, 20, 3, 1000}

orderednumbers = {filelist, numberlist}[[All, 
    Ordering@numberlist]][[1]]

{"1.dat", "2.dat", "3.dat", "10.dat", "20.dat", "100.dat", "1000.dat"}

I think that part is really the crux of your problem. From there you can call whatever part of the file you want. Hope that helps!

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.