2
$\begingroup$

I scrape data from a public website about houses for rent. All pages have the same structure. But the pages can differ in the amount of information.

Below I give two examples:

page/house 1:

test = Cases[
  fundafiles1, 
  XMLElement["dl", {"class" -> "object-kenmerken-list"}, __], 
  ∞
];

Cases[test, {XMLElement[ "dt", {}, {"number rooms"}], ___}, ∞]

The output is:

{ { XMLElement["dt", {}, {"number rooms"}],    
    XMLElement["dd", {}, {"4 rooms (3 slepingrooms)"}],    
    XMLElement["dt", {}, {"number floors"}],    
    XMLElement["dd", {}, {"2 floors"}],    
    XMLElement["dt", {}, {"services"}],    
    XMLElement["dd", {}, {"Mechanische ventilatie, TV kabel, jacuzzi en schuifpui"}]
  }
}

for page/house 2 I use the same script:

test = Cases[
  fundafiles1, 
  XMLElement["dl", {"class" -> "object-kenmerken-list"}, __], 
  ∞
];

Cases[test, {XMLElement["dt", {}, {"number rooms"}], ___}, ∞]

but now the output is:

{ { XMLElement["dt", {}, {"Number rooms"}],    
    XMLElement["dd", {}, {"2 rooms (1 slepingroom)"}],    
    XMLElement["dt", {}, {"Number bathrooms"}],    
    XMLElement["dd", {}, {"1 bathroom en 1  toilet"}],   
    XMLElement["dt", {}, {"bathroomservices"}],    
    XMLElement["dd", {}, {"Douche"}],    
    XMLElement["dt", {}, {"number floors"}],   
    XMLElement["dd", {}, {"1 floor"}],    
    XMLElement["dt", {}, {"located"}],    
    XMLElement["dd", {}, {"2e floor"}],   
    XMLElement["dt", {}, {"services"}],    
    XMLElement["dd", {}, {"elevator"}]
  }
}

Because the number of attritutes can differ, the attributes don't have their own position. I want to collect the data and put them in a new table for further analysis.

For example: I want to select "Number rooms" end get as answer a number. Who has a suggestion?

$\endgroup$
  • 1
    $\begingroup$ Can you provide a small (made up) xml example to work with which exhibits your problems? I'm also not sure what is the desired result. $\endgroup$ – Kuba Jul 24 '18 at 12:56
  • 1
    $\begingroup$ Are "Number rooms" and "number of rooms" both supposed to match your search pattern? $\endgroup$ – Mr.Wizard Jul 24 '18 at 13:03
  • $\begingroup$ your are right, it must be 'number rooms' $\endgroup$ – Michiel van Mens Jul 24 '18 at 14:01
2
$\begingroup$

My guess as to what you want:

in1 = {{XMLElement["dt", {}, {"number of rooms"}], 
    XMLElement["dd", {}, {"4 rooms (3 slepingrooms)"}], 
    XMLElement["dt", {}, {"number floors"}], XMLElement["dd", {}, {"2 floors"}], 
    XMLElement["dt", {}, {"services"}], 
    XMLElement[
     "dd", {}, {"Mechanische ventilatie, TV kabel, jacuzzi en schuifpui"}]}};

in2 = {{XMLElement["dt", {}, {"Number rooms"}], 
    XMLElement["dd", {}, {"2 rooms (1 slepingroom)"}], 
    XMLElement["dt", {}, {"Number bathrooms"}], 
    XMLElement["dd", {}, {"1 bathroom en 1  toilet"}], 
    XMLElement["dt", {}, {"bathroomservices"}], XMLElement["dd", {}, {"Douche"}], 
    XMLElement["dt", {}, {"number floors"}], XMLElement["dd", {}, {"1 floor"}], 
    XMLElement["dt", {}, {"located"}], XMLElement["dd", {}, {"2e floor"}], 
    XMLElement["dt", {}, {"services"}], XMLElement["dd", {}, {"elevator"}]}};

target = StringMatchQ[#, "number*rooms", IgnoreCase -> True] &;

filter = Cases[{___, XMLElement["dt", {}, {_?target}], 
     XMLElement["dd", {}, {value_}], ___} :> value];

filter /@ {in1, in2}
{{"4 rooms (3 slepingrooms)"}, {"2 rooms (1 slepingroom)"}}
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.