If I have a document, any document, it can be a book, an article, etc, how could I extract people's names from there?
For example, if the input is an article, the output of the function would be the authors.
$\begingroup$
$\endgroup$
7
-
5$\begingroup$ This is an extremely broad question - please consider adding more detail and a specific example. In the meantime, my first thought would be to split the text into words (using TextWords) and checking against some very long dataset of names that I'm sure is available on the internet somewhere. $\endgroup$– Carl LangeCommented Aug 27, 2018 at 19:39
-
2$\begingroup$ ...and a difficult question that needs semantic document analysis for its solution. After all, suppose the name "Mark Stone" is listed. You'll never extract that name based on simple semantics as both names are standard words. You'll need to know where authors are listed on a document, that a sentence refers to a person, and so on. Much harder than the poser would imagine, I bet. $\endgroup$– David G. StorkCommented Aug 27, 2018 at 20:20
-
$\begingroup$ What if you trained a neural network to pull names out of text? $\endgroup$– ktmCommented Aug 27, 2018 at 20:24
-
2$\begingroup$ You would likely need hundreds of millions of training documents in which the target names were pre-identified. A tricky task worthy of a Masters Degree thesis or a small startup company, such as the companies that parse resumes and other documents. $\endgroup$– David G. StorkCommented Aug 27, 2018 at 20:28
-
2$\begingroup$ hundreds of millions might be a slight exaggeration. $\endgroup$– High Performance MarkCommented Aug 27, 2018 at 22:39
|
Show 2 more comments
1 Answer
$\begingroup$
$\endgroup$
You may use the functions in the Text Analysis guide to extract the "NounPhrase"s containing "ProperNoun"s. This gives a short list of proper nouns to evaluate for people names.
First collect some text that contains people's names in the form of a Wikipedia article using WikipediaData.
articleText = WikipediaData["Garrett Morgan"];
The article's text can be separated into sentences using TextSentences.
txtSentences = TextSentences[articleText];
Version 10.3 added the experimental function TextStructure that breaks a text down into its grammatical structure using TextElements.
txtStruc = TextStructure[First@txtSentences, "ConstituentTree"]
Using the nested structure of the "ConstituentTree" (evaluate FullForm@txtStruc) the "NounPhrase"s containing "ProperNoun"s can be extracted by using Cases.
properNounPrases =
StringRiffle /@
Cases[TextElement[w_String, _Association] :> w] /@
First /@
Cases[
TextStructure[txtSentences, "ConstituentTree"]
,
TextElement[
{TextElement[_String,
<|
"GrammaticalUnit" -> Entity["GrammaticalUnit", "ProperNoun" | "Punctuation"]
|>] ..},
<|"GrammaticalUnit" -> Entity["GrammaticalUnit", "NounPhrase"]|>
],
Infinity
];
This produces a short list of proper nouns to consider.
Multicolumn[DeleteDuplicates@properNounPrases,
Frame -> All,
FrameStyle -> LightGray,
ItemSize -> Full]
From properNounPrases you could then continue either select by hand or compare to some database or train some net to select the names for you.
Hope this helps.
