0
$\begingroup$

This question asks how Rest operates with Part in lists of lists.

I’m trying to use Rest to drop headers from .csv files, and can’t seem to get it to select the right parts. I’m clearly misunderstanding something about it. Here are 4 .csv files, called F1 to F4, which all have the first line as a header:

enter image description here

I get the filenames and load them:

allFiles = FileNames["*.*", "Data\\"];
pData = Import[#, "Table"] & /@ allFiles;

A display of pData, formatted, gives:

{
  {
    {"A1","A2","A3","A4"}, {"1,2,3,"a"}, {"4,5,6,"b"}, {"7,8,9,"c"}, {"10,11,12,"d"}, {"13,14,15,"e"}
  },
  {
    {"A1","A2","A3","A4"},{21,22,23,"f"},{24,25,26,"g"},{27,28,29,"h"},{210,211,212,"i"},{213,214,215,"j"},{216,217,218,"k"}
  },
  {
    {"A1","A2","A3","A4"},{31,32,33,"l"},{34,35,36,"m"},{37,38,39,"n"}
  },
  {
    {"A1","A2","A3","A4"},{41,42,43,"o"},{44,45,46,"p"},{47,48,49,"q"},{410,411,412,"r"}
  }
}

So there is a list with 4 sublists, each representing a file. Each of the sublists has a number of subsublists, each one representing a line in the file. I want to get rid of the {"A1","A2","A3","A4"} subsublists, the header record in each file, so naturally assumed the Rest would be appropriate.

If I do this:

pData[[All, 1]]

I get this:

{{"A1","A2","A3","A4"},{"A1","A2","A3","A4"},{"A1","A2","A3","A4"},{"A1","A2","A3","A4"}}
  • the first record of each file, precisely as expected.

Now I try Rest. So:

Rest[pData[[All]]]

Gives:

{
  {
    {"A1","A2","A3","A4"},{21,22,23,"f"},{24,25,26,"g"},{27,28,29,"h"},{210,211,212,"i"},{213,214,215,"j"},{216,217,218,"k"}
  },
  {
    {"A1","A2","A3","A4"},{31,32,33,"l"},{34,35,36,"m"},{37,38,39,"n"}
  },
  {
    {"A1","A2","A3","A4"},{41,42,43,"o"},{44,45,46,"p"},{47,48,49,"q"},{410,411,412,"r"}
  }
}
  • the first file's data have disappeared, again as expected.

So logically, Rest[pData[[All,All]]] should drop the first line in each file’s list. It doesn’t.

It gives the same result as Rest[pData[[All]]]: the first file is dropped. It states in the documentation that "In Part and related functions, All specifies all parts at a particular level.”, and that’s what I’m using. Rest gives no indication in the docs that it only works at the top level, so I’m obviously missing something.

$\endgroup$
  • $\begingroup$ pData[[All, All]] is the same thing as pData[[All]] and pData. There's no difference between these 3. Therefore, applying Rest to them yields the same result for all of these expressions. $\endgroup$ – Sjoerd Smit Aug 14 at 15:50
  • 1
    $\begingroup$ You want Rest /@ pdata or pdata[[All, 2 ;;]] $\endgroup$ – flinty Aug 14 at 15:51
  • 1
    $\begingroup$ You have syntax errors in your posted pData. You should Map Rest onto pData, i.e., either Rest /@ pData or Map[Rest, pData] $\endgroup$ – Bob Hanlon Aug 14 at 15:51
  • $\begingroup$ Thanks Sjoerd, flinty, Bob - I was misunderstanding the level-handling. That was very helpful. $\endgroup$ – Slaughterteddy Aug 16 at 13:04

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.