Take the 2-minute tour ×
Mathematica Stack Exchange is a question and answer site for users of Mathematica. It's 100% free, no registration required.

How can I get from data segment format to the final segment format with a pure function? The data fields are fixed length. There are no separators between the fields.

a) list

wordlength = {11, 20, 10, 9};
from = 2; to = 3;  

datasegment = {"        109Lemon Chiffon       25:22:29  6519218  ",
               "       2082Mint Cream          39:24:41  7314284  ",
               "       6790Indian Red          21:57:53  3368766  "};  s
finalsegment = {"Lemon Chiffon       25:22:29  ",
                "Mint Cream          39:24:41  ",
                "Indian Red          21:57:53  "};  

b) string

dataline = {"       1556Floral White        29:48:49  7856900  "};
finalline = {"Floral White        29:48:49  "};

Update

Final solutions were found for the task. I like to share it here.

inlist = RandomChoice[
{"       2524Lemon Chiffon       22:51:28  1881779  ",
 "       3085Teal                19:22:24  9744812  ",
 "       5935Light Yellow        03:32:54  8374029  ",
 "       6128Powder Blue         27:20:07  7315046  "}, 
400000];  

wordLength = {11, 20, 10, 9};
fromWord = 2;
numberOfWords = 2;  

fields = Length[wordLength];
borders = {{1, wordLength[[1]]}};
Do[
borders = 
FlattenAt[{borders, {borders[[i, 2]] + 1, 
borders[[i, 2]] + wordLength[[i + 1]]}}, 1], {i, fields - 1}];
grenz = {borders[[fromWord, 1]], 
borders[[fromWord + numberOfWords - 1, 2]] };
outlist = StringTake[inlist, grenz];// Timing  

{0.141, Null}  

The honor goes to the very concerned in this task. Every detail of their conception was vital to the solution. Keep'n rocking.

share|improve this question
2  
Are you certain you want to keep your data as strings? You may find it more convenient to structure dateline as a list, as in {{109, "Lemon Chiffon"}, {25, 22, 29}, 6519218}. Even if I were only interested in displaying the data, I would use a list rather than a string. –  David Carraher Dec 30 '12 at 14:14
    
I am certain about it. –  Hp Radojewski Schäfer Von Jan 1 '13 at 9:17
add comment

3 Answers

With a fixed length you can use StringTake and set the region you like

StringTake[#, {12, 11 + 20 + 10}] & /@ 
{"        109Lemon Chiffon       25:22:29  6519218  ", 
 "       2082Mint Cream          39:24:41  7314284  ", 
 "       6790Indian Red          21:57:53  3368766  "}

(*
Lemon Chiffon       25:22:29  
Mint Cream          39:24:41  
Indian Red          21:57:53  
*)

Update

You noted, that the creation of the appropriate region for StringTake is maybe tedious. For this, you can write a function which takes your word boundaries and the columns you want to extract and returns another function, an extractor which can take one string and extracts the appropriate columns. This has the advantage that you have to give the word boundaries and the columns only one time and can use the extractor as often as you like.

Taking your {11, 20, 10, 9} boundaries as example, the integral part of the function is a combination of Accumulate, Partition and a Map which adds 1 to the start position of each word region.

({#1+1,#2}&@@@Partition[Accumulate@{11,20,10,9},2,1])
(* {{12,31},{32,41},{42,50}} *)

As you see you get exactly the positions you need for StringTake, e.g. first word goes from 12 to 31, etc. Surely, you don't want all words from a string, because then you wouldn't need a selector, but wouldn't it be nice, if you could use the same syntax as you can use for Part, namely specify single elements {1,2,3,5} or ranges {1;;3, 5}? This is done in the second part of our selector, where we use a string which needs to be filled:

dataSelector[columns_, wordlengths_] := 
 With[{wordregions = ({#1 + 1, #2} & @@@
    Partition[Accumulate@Prepend[wordlengths, 0], 2, 1])},
  Function[string, StringJoin[StringTake[string, #] & /@ 
    (Part[wordregions, #] & /@ columns)]]]

The usage is now simple: with in being your sample data you can do

in={"        109Lemon Chiffon       25:22:29  6519218  ",
    "       2082Mint Cream          39:24:41  7314284  ",
    "       6790Indian Red          21:57:53  3368766  "};

dataSelector[{4,2;;3,1},{11,20,10,9}]/@in
(*
{6519218  Lemon Chiffon       25:22:29          109,
 7314284  Mint Cream          39:24:41         2082,
 3368766  Indian Red          21:57:53         6790}
*)

Update regarding comment

Yes, you are right, the first word could not be extracted but since it always ranges from 1 simply prepending 0 before calling Accumulate in the dataSelector function fixes the problem. Additionally, when you use the dataSelector to extract the say second and third word it returns a list of strings. When I look now at your sample, you maybe prefer to have one string back where all selected words are concatenated. I added the appropriate StringJoin to the function.

Depending on your level as Mathematica programmer, my dataSelector is a bad example since it's probably a bit harder to understand. When you want to create the output as in your last example with it, you have to use it like:

dataline = {"       1556Floral White        29:48:49  7856900  "};
sel = dataSelector[{2, 3}, {11, 20, 10, 9}];
sel[dataline]

(* "Floral White        29:48:49  " *)

or, if you want to extract the second and third word for a list of strings, you simply use sel /@ in. The important line to understand is the second. To create a selector function, you call for instance dataSelector[{2, 3}, {11, 20, 10, 9}] where {2,3} are the words you want to extract and the last parameter is the length of each word.

Important here is, that you can use the range specification ;; and All like you can use it with Part ([[]]). Test for instance the output of the following examples

dataSelector[{All}, {11, 20, 10, 9}][dataline]
dataSelector[{1, 3 ;; 4}, {11, 20, 10, 9}][dataline]
dataSelector[{1, All, 2 ;; 4}, {11, 20, 10, 9}][dataline]

Update 2

When your goal is speed, then you should call StringTake[{s1,s2,...},{spec1,spec2,...}], this means you input all your datasets {s1,s2,...} in one call and you put all wordboundary specs in a list. This should be reasonable fast.

Your update contains the undefined variable wb. Would you mind to measure your method against the method below with the 4000000 elements in data?

data = RandomChoice[
   {"        109Lemon Chiffon       25:22:29  6519218  ",
    "       2082Mint Cream          39:24:41  7314284  ", 
    "       6790Indian Red          21:57:53  3368766  "}, 4000000];

dataSelector[columns_, wordlengths_] := 
  With[{wordregions = (Part[({#1 + 1, #2} & @@@             
          Partition[Accumulate@Prepend[wordlengths, 0], 2, 
           1]) , #] & /@ columns)},
    Function[string, StringJoin @@@ StringTake[string, wordregions]] ]

f = dataSelector[{2, 3}, {11, 20, 10, 9}];

AbsoluteTiming[f[data]] // First
share|improve this answer
    
You are right. Difficult part is to go from wordlength {11, 20, 10, 9} to your selection list {12, 11 + 20 + 10} and integrate that into your function. And be generic. You like to take an arbitrary start word and an arbitrary end word to build the resuld. –  Hp Radojewski Schäfer Von Jan 1 '13 at 9:08
    
@HpRadojewskiSchäferVon I updated my answer. –  halirutan Jan 1 '13 at 9:59
    
OK. . .Dont you need the boundaries for the 1rst Word too?. . .And help me to use it. . .We need a result that looks the same as shown above in finalsegment. –  Hp Radojewski Schäfer Von Jan 2 '13 at 15:04
    
@HpRadojewskiSchäferVon I changed the function and added additional explanation at the end. I hope you can use it now. –  halirutan Jan 2 '13 at 16:03
    
OK. . .A fast algorithmic solution was found. . .(see Update). . .Timing for 4.000.000 records. . .This function 23 sec, the algorithmic solution 1.3 sec –  Hp Radojewski Schäfer Von Jan 3 '13 at 13:28
show 2 more comments

one possibility is:

fun = Function[x, 
   x /. s_String :> 
     StringReplace[StringTrim@s, 
      NumberString ~~ Longest[y__] ~~ "  " ~~ NumberString ~~ 
        EndOfString :> (y ~~ "  ")]];

though I find strange that it does not work if I leave out the " " ...

share|improve this answer
    
Can you give same elucidation where in your function to place the variables wordlength, from, to and datasegment. How to call your function. –  Hp Radojewski Schäfer Von Jan 1 '13 at 9:15
    
You can apply the function by fun@datasegment and fun@dataline and variable worldlength is just not needed. –  Rolf Mertig Jan 1 '13 at 14:21
    
OKWhat would you do if the number of "Words2 is 100. And you would like to take from 60 throgh 80? –  Hp Radojewski Schäfer Von Jan 2 '13 at 14:46
    
OK. . .What would you do if the number of "Words" in datasegment is 100. .(here we have 4). .and you would like to take from 60 through 80? –  Hp Radojewski Schäfer Von Jan 2 '13 at 14:50
    
I would read the documentation of ReadList and write a function to do what I need. –  Rolf Mertig Jan 2 '13 at 22:23
add comment

I know you said these strings were fixed position but here is a more generic approach:

Flatten@StringCases[datasegment, Whitespace ~~ NumberString ~~ z__ ~~ Whitespace
~~NumberString :> z]

(* {
"Lemon Chiffon       25:22:29 ", 
"Mint Cream          39:24:41 ", 
"Indian Red          21:57:53 "
} *)

This could be made more concise using Regular Expressions.

share|improve this answer
    
Unfortunately not generic. You like to take an arbitrary start word and an arbitrary end word to build the resuld. –  Hp Radojewski Schäfer Von Jan 1 '13 at 8:32
    
@HpRadojewskiSchäferVon hence I said "more generic" rather than "generic". –  Mike Honeychurch Jan 1 '13 at 21:15
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.