3
$\begingroup$

I have strings like these:

animals1 = {"Cow and Milk, extracreamy and Chicken and Egg and Pig and Sheep"};
animals2 = {"Cow and Milk and Chicken and Egg and Pig and Sheep"};

Now I want to use StringSplitusing " and " as expression but I was wondering if it is possible to define some kind of universal Go-List which defines exceptions for StringSplit such as this one

whitelist = {"Cow and Milk", "Chicken and Egg", "Cow and Milk, extracreamy"}

So the outcome should be

{"Cow and Milk, extracreamy", "Chicken and Egg", "Pig", "Sheep"} 

and

{"Cow and Milk", "Chicken and Egg", "Pig", "Sheep"} 

respectively. More specifically, the problem is that Cow and Milk and Cow and Milk, extracreamy are defined in one Go-List....

$\endgroup$
5
  • 1
    $\begingroup$ StringSplit[animals[[1]], Prepend[Thread[whitelist -> whitelist], "and"]] should get you started. Note the nulls in the result - this is the documented behavior of StringSplit for adjacent delimiters, you can just apply DeleteCases on the result to remove them. $\endgroup$
    – ciao
    Feb 28, 2020 at 23:18
  • $\begingroup$ It seems that you are looking for some sort of Named Entity Recognition algorithm. $\endgroup$ Nov 24, 2020 at 23:30
  • $\begingroup$ @AntonAntonov: does this mean that there is no proper solution with a single whitelist? $\endgroup$
    – M.A.
    Nov 25, 2020 at 7:57
  • $\begingroup$ @M.A. "does this mean that there is no proper solution with a single whitelist?" -- I meant two things. 1) Your "whitelist" approach is basically NER's gazetteer approach. 2) There is a field in Information retrieval/extraction to which your question belongs. $\endgroup$ Nov 25, 2020 at 13:58
  • $\begingroup$ @AntonAntonov: Ok this suggests that there is no easy implementation in Mathematica (With easy I mean: without having advanced skills in machine learning...). $\endgroup$
    – M.A.
    Nov 25, 2020 at 14:15

2 Answers 2

3
$\begingroup$

There are many ways to do this, among others:

  whitelist = {"Cow and Milk", "Chicken and Egg"}  

     ReadList[StringToStream[
       "Cow and Milk and Chicken and Egg and Pig and Sheep"]
      , Word
      , RecordLists -> True
      , RecordSeparators -> whitelist
      , WordSeparators -> "and"
      , TokenWords -> whitelist]  

    Flatten[%]
    StringTrim /@ %
    DeleteCases[%, ""] 

{{"Cow and Milk", " ", " "}, {"Chicken and Egg", " ", " Pig ", " Sheep"}}
{"Cow and Milk", " ", " ", "Chicken and Egg", " ", " Pig ", " Sheep"}
{"Cow and Milk", "", "", "Chicken and Egg", "", "Pig", "Sheep"}
{"Cow and Milk", "Chicken and Egg", "Pig", "Sheep"}

Another approach :

whitelist = {"Cow and Milk", "Chicken and Egg"};  

StringSplit["Cow and Milk and Chicken and Egg and Pig and Sheep", # ->
     white[#] & /@ whitelist] 
If[Head[#] === white, #[[1]], StringSplit[#, "and"]] & /@ %  


Flatten[%]
StringTrim /@ % 
DeleteCases[%, ""]    

{white["Cow and Milk"], " and ", white["Chicken and Egg"], " and Pig and Sheep"}
{"Cow and Milk", {" ", " "}, "Chicken and Egg", {" ", " Pig ", " Sheep"}}
{"Cow and Milk", " ", " ", "Chicken and Egg", " ", " Pig ", " Sheep"}
{"Cow and Milk", "", "", "Chicken and Egg", "", "Pig", "Sheep"}
{"Cow and Milk", "Chicken and Egg", "Pig", "Sheep"}

$\endgroup$
1
$\begingroup$

A variation on ciao's suggestion in comments:

StringSplit["Cow and Milk and Chicken and Egg and Pig and Sheep", 
   {a : Alternatives @@ whitelist :> a, "and"}] // DeleteCases[" "] // StringTrim
 {"Cow and Milk", "Chicken and Egg", "Pig", "Sheep"}
$\endgroup$
1
  • $\begingroup$ But if you add "Cow and Milk, extracreamy" to whitelistand to the string respectively... put differently: is there a way that allows to distinguish between "Cow and Milk, extracreamy" and "Cow and Milk" when splitting the strings? $\endgroup$
    – M.A.
    Nov 25, 2020 at 10:45

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.