Skip to main content
improved answer and added comparison
Source Link
Syed
  • 59.5k
  • 5
  • 40
  • 95
data = Import[
  "http://www.deeplook.ir/wp-content/uploads/2021/11/data.txt", 
  "Text"]

Benefitting from this discussion (and don't ask me about the pattern):

tx = StringCases[data, 
   RegularExpression[
    "(?:[a-z0-9!#$%&'*+/=?^_`{|}~-]+(?:\\.[a-z0-9!#$%&'*+/=?^_`{|}~-]+)*\]+\
|\")*|\"(?:[\\x01-\\x08\\x0b\\x0c\\x0e-\\x1f\\x21\\x23-\\x5b\\x5d-\\x7f]|\\\\\x7f]|\
\\[\\x01\\\\[\\x01-\\x09\\x0b\\x0c\\x0e-\\x7f])*\")@(?:(?:[a-z0-9](?:[a-z0-9-]*[\]\
a*[a-z0-9])?\\.)+[a-z0-9](?:[a-z0-9-]*[a-z0-9])?|\\[(?:(?:(2(5[0-5]|[0-4]\\
[04][0-9])|1[0-9][0-9]|[1-9]?[0-9]))\\.){3}(?:(2(5[0-5]|[0-4][0-9])|1[0-9]\\
[09][0-9]|[1-9]?[0-9])|[a-z0-9-]*[a-z0-9]:(?:[\\x01-\\x08\\x0b\\x0c\\x0e-\\\x08\\x0b\\x0c\\\
x0e-\\x1f\\x21-\\x5a\\x53-\\x7f]|\\\\[\\x01-\\x09\\x0b\\x0c\\x0e-\\x7f]\\\
x7f])+)\
\\])
"]]"]];

list1 = Union@(StringTrim @tx)

This still has multiple email entries and some website strings that have not been filtered.


For Mathematica users, a more native (and better) way would be:

temail = StringCases[data, 
   Whitespace | StartOfLine ~~ Shortest[WordCharacter 
 ..] ~~ "@" ~~Shortest[("_" 
 | "-" | WordCharacter Shortest[WordCharacter| ".") ..] ~~ ".""@" ~~ 
    Shortest[__]Shortest[(WordCharacter | ".") ..] ~~ (Whitespace | "," | 
      EndOfLine | "&")];

list2 = Union@(StringTrim @temail);

and it is quite readable on its own.

StringTrim and Unionsimply do the required cleanup.


For comparison

{Length@list1, Length@list2} (*{652, 785} *)

As an exercise, do the following to verify the differences:

Complement[list1, list2]; (*25 ?entries due to uppercase etc*)
Complement[list2, list1];

data = Import[
  "http://www.deeplook.ir/wp-content/uploads/2021/11/data.txt", 
  "Text"]

Benefitting from this discussion (and don't ask me about the pattern):

StringCases[data, 
 RegularExpression[
  "(?:[a-z0-9!#$%&'*+/=?^_`{|}~-]+(?:\\.[a-z0-9!#$%&'*+/=?^_`{|}~-]+)*\
|\"(?:[\\x01-\\x08\\x0b\\x0c\\x0e-\\x1f\\x21\\x23-\\x5b\\x5d-\\x7f]|\\\
\\[\\x01-\\x09\\x0b\\x0c\\x0e-\\x7f])*\")@(?:(?:[a-z0-9](?:[a-z0-9-]*[\
a-z0-9])?\\.)+[a-z0-9](?:[a-z0-9-]*[a-z0-9])?|\\[(?:(?:(2(5[0-5]|[0-4]\
[0-9])|1[0-9][0-9]|[1-9]?[0-9]))\\.){3}(?:(2(5[0-5]|[0-4][0-9])|1[0-9]\
[0-9]|[1-9]?[0-9])|[a-z0-9-]*[a-z0-9]:(?:[\\x01-\\x08\\x0b\\x0c\\x0e-\
\\x1f\\x21-\\x5a\\x53-\\x7f]|\\\\[\\x01-\\x09\\x0b\\x0c\\x0e-\\x7f])+)\
\\])
"]]

This still has multiple email entries and some website strings that have not been filtered.


For Mathematica users, a more native (and better) way would be:

temail = StringCases[data, 
   Whitespace | StartOfLine ~~ Shortest[WordCharacter ..] ~~ "@" ~~ 
     Shortest[WordCharacter ..] ~~ "." ~~ 
    Shortest[__] ~~ (Whitespace | "," | EndOfLine)];

Union@(StringTrim @temail)

and it is quite readable on its own.

StringTrim and Unionsimply do the required cleanup.


data = Import[
  "http://www.deeplook.ir/wp-content/uploads/2021/11/data.txt", 
  "Text"]

Benefitting from this discussion (and don't ask me about the pattern):

tx = StringCases[data, 
   RegularExpression[
    "(?:[a-z0-9!#$%&'*+/=?^_`{|}~-]+(?:\\.[a-z0-9!#$%&'*+/=?^_`{|}~-]+\
)*|\"(?:[\\x01-\\x08\\x0b\\x0c\\x0e-\\x1f\\x21\\x23-\\x5b\\x5d-\\x7f]|\
\\\\[\\x01-\\x09\\x0b\\x0c\\x0e-\\x7f])*\")@(?:(?:[a-z0-9](?:[a-z0-9-]\
*[a-z0-9])?\\.)+[a-z0-9](?:[a-z0-9-]*[a-z0-9])?|\\[(?:(?:(2(5[0-5]|[0-\
4][0-9])|1[0-9][0-9]|[1-9]?[0-9]))\\.){3}(?:(2(5[0-5]|[0-4][0-9])|1[0-\
9][0-9]|[1-9]?[0-9])|[a-z0-9-]*[a-z0-9]:(?:[\\x01-\\x08\\x0b\\x0c\\\
x0e-\\x1f\\x21-\\x5a\\x53-\\x7f]|\\\\[\\x01-\\x09\\x0b\\x0c\\x0e-\\\
x7f])+)\\])
"]];

list1 = Union@(StringTrim @tx)

This still has multiple email entries and some website strings that have not been filtered.


For Mathematica users, a more native (and better) way would be:

temail = StringCases[data, 
   Whitespace | StartOfLine ~~  
    Shortest[("_" | "-" | WordCharacter | ".") ..] ~~ "@" ~~ 
    Shortest[(WordCharacter | ".") ..] ~~ (Whitespace | "," | 
      EndOfLine | "&")];

list2 = Union@(StringTrim @temail);

and it is quite readable on its own.

StringTrim and Unionsimply do the required cleanup.


For comparison

{Length@list1, Length@list2} (*{652, 785} *)

As an exercise, do the following to verify the differences:

Complement[list1, list2]; (*25 ?entries due to uppercase etc*)
Complement[list2, list1];

added Mathematica's way of doing it
Source Link
Syed
  • 59.5k
  • 5
  • 40
  • 95
data = Import[
  "http://www.deeplook.ir/wp-content/uploads/2021/11/data.txt", 
  "Text"]

This will do it. Clarification: My life is notBenefitting from this messed up...yet.discussion (and don't ask me about the pattern):

StringCases[data, 
 RegularExpression[
  "(?:[a-z0-9!#$%&'*+/=?^_`{|}~-]+(?:\\.[a-z0-9!#$%&'*+/=?^_`{|}~-]+)*\
|\"(?:[\\x01-\\x08\\x0b\\x0c\\x0e-\\x1f\\x21\\x23-\\x5b\\x5d-\\x7f]|\\\
\\[\\x01-\\x09\\x0b\\x0c\\x0e-\\x7f])*\")@(?:(?:[a-z0-9](?:[a-z0-9-]*[\
a-z0-9])?\\.)+[a-z0-9](?:[a-z0-9-]*[a-z0-9])?|\\[(?:(?:(2(5[0-5]|[0-4]\
[0-9])|1[0-9][0-9]|[1-9]?[0-9]))\\.){3}(?:(2(5[0-5]|[0-4][0-9])|1[0-9]\
[0-9]|[1-9]?[0-9])|[a-z0-9-]*[a-z0-9]:(?:[\\x01-\\x08\\x0b\\x0c\\x0e-\
\\x1f\\x21-\\x5a\\x53-\\x7f]|\\\\[\\x01-\\x09\\x0b\\x0c\\x0e-\\x7f])+)\
\\])
"]]

This still has multiple email entries and some website strings that have not been filtered.


For Mathematica users, a more native (and better) way would be:

temail = StringCases[data, 
   Whitespace | StartOfLine ~~ Shortest[WordCharacter ..] ~~ "@" ~~ 
    Shortest[WordCharacter ..] ~~ "." ~~ 
    Shortest[__] ~~ (Whitespace | "," | EndOfLine)];

Union@(StringTrim @temail)

and it is quite readable on its own.

StringTrim and Unionsimply do the required cleanup.


This will do it. Clarification: My life is not this messed up...yet.

StringCases[data, 
 RegularExpression[
  "(?:[a-z0-9!#$%&'*+/=?^_`{|}~-]+(?:\\.[a-z0-9!#$%&'*+/=?^_`{|}~-]+)*\
|\"(?:[\\x01-\\x08\\x0b\\x0c\\x0e-\\x1f\\x21\\x23-\\x5b\\x5d-\\x7f]|\\\
\\[\\x01-\\x09\\x0b\\x0c\\x0e-\\x7f])*\")@(?:(?:[a-z0-9](?:[a-z0-9-]*[\
a-z0-9])?\\.)+[a-z0-9](?:[a-z0-9-]*[a-z0-9])?|\\[(?:(?:(2(5[0-5]|[0-4]\
[0-9])|1[0-9][0-9]|[1-9]?[0-9]))\\.){3}(?:(2(5[0-5]|[0-4][0-9])|1[0-9]\
[0-9]|[1-9]?[0-9])|[a-z0-9-]*[a-z0-9]:(?:[\\x01-\\x08\\x0b\\x0c\\x0e-\
\\x1f\\x21-\\x5a\\x53-\\x7f]|\\\\[\\x01-\\x09\\x0b\\x0c\\x0e-\\x7f])+)\
\\])
"]]
data = Import[
  "http://www.deeplook.ir/wp-content/uploads/2021/11/data.txt", 
  "Text"]

Benefitting from this discussion (and don't ask me about the pattern):

StringCases[data, 
 RegularExpression[
  "(?:[a-z0-9!#$%&'*+/=?^_`{|}~-]+(?:\\.[a-z0-9!#$%&'*+/=?^_`{|}~-]+)*\
|\"(?:[\\x01-\\x08\\x0b\\x0c\\x0e-\\x1f\\x21\\x23-\\x5b\\x5d-\\x7f]|\\\
\\[\\x01-\\x09\\x0b\\x0c\\x0e-\\x7f])*\")@(?:(?:[a-z0-9](?:[a-z0-9-]*[\
a-z0-9])?\\.)+[a-z0-9](?:[a-z0-9-]*[a-z0-9])?|\\[(?:(?:(2(5[0-5]|[0-4]\
[0-9])|1[0-9][0-9]|[1-9]?[0-9]))\\.){3}(?:(2(5[0-5]|[0-4][0-9])|1[0-9]\
[0-9]|[1-9]?[0-9])|[a-z0-9-]*[a-z0-9]:(?:[\\x01-\\x08\\x0b\\x0c\\x0e-\
\\x1f\\x21-\\x5a\\x53-\\x7f]|\\\\[\\x01-\\x09\\x0b\\x0c\\x0e-\\x7f])+)\
\\])
"]]

This still has multiple email entries and some website strings that have not been filtered.


For Mathematica users, a more native (and better) way would be:

temail = StringCases[data, 
   Whitespace | StartOfLine ~~ Shortest[WordCharacter ..] ~~ "@" ~~ 
    Shortest[WordCharacter ..] ~~ "." ~~ 
    Shortest[__] ~~ (Whitespace | "," | EndOfLine)];

Union@(StringTrim @temail)

and it is quite readable on its own.

StringTrim and Unionsimply do the required cleanup.


Source Link
Syed
  • 59.5k
  • 5
  • 40
  • 95

This will do it. Clarification: My life is not this messed up...yet.

StringCases[data, 
 RegularExpression[
  "(?:[a-z0-9!#$%&'*+/=?^_`{|}~-]+(?:\\.[a-z0-9!#$%&'*+/=?^_`{|}~-]+)*\
|\"(?:[\\x01-\\x08\\x0b\\x0c\\x0e-\\x1f\\x21\\x23-\\x5b\\x5d-\\x7f]|\\\
\\[\\x01-\\x09\\x0b\\x0c\\x0e-\\x7f])*\")@(?:(?:[a-z0-9](?:[a-z0-9-]*[\
a-z0-9])?\\.)+[a-z0-9](?:[a-z0-9-]*[a-z0-9])?|\\[(?:(?:(2(5[0-5]|[0-4]\
[0-9])|1[0-9][0-9]|[1-9]?[0-9]))\\.){3}(?:(2(5[0-5]|[0-4][0-9])|1[0-9]\
[0-9]|[1-9]?[0-9])|[a-z0-9-]*[a-z0-9]:(?:[\\x01-\\x08\\x0b\\x0c\\x0e-\
\\x1f\\x21-\\x5a\\x53-\\x7f]|\\\\[\\x01-\\x09\\x0b\\x0c\\x0e-\\x7f])+)\
\\])
"]]