Version 12 has basic NER support for some entities, but how does one recognize a custom entity?
For example, I want to parse text describing products, and parse out three entities: prices, size, and color, as custom entities. Using TextCases and TextContents works nicely for price and color:
TextCases["A red tshirt costs $5 and is medium", {"Color",
"CurrencyAmount"}]
But there is no way to parse entities that are not listed in guide/TextContentTypes, like sizes:
TextCases["large women's leather jacket", {"Size"}]
And there is no way to add additional synonyms or spellings to an entity:
TextCases[# <> " feather", "Color"] & /@ {"violet", "lilac",
"lavender", "royal", "purpled", "plum", "grape", "maroon",
"magenta", "purplish"}
I want to extend the built-in NER model with custom training data, i.e. substring labels:
newTrainingData = <|"Who is Nishanth?"-> {8, 16, "Name"},
"Who is Kamal Khumar?" -> {8, 20, "Name"},
"I like London and Berlin." -> {{8,14, "City"}, {19,25,"City"}
|>
If this is not supported in 12.2, perhaps there is a 3rd party paclet, resource function, or some NN repo entry to extend? or maybe it's coming in 12.3?
Related:
- Custom NER can be done in python with spacy: https://towardsdatascience.com/train-ner-with-custom-training-data-using-spacy-525ce748fab7
- Original post: Named entity recognition
