1
$\begingroup$

In an ongoing effort to reorganize data from files downloaded as PDF's from the web and using the solution provided in MMSE 295606 (Difficulty using StringCases to obtain desired substring),

I tried to modify this using suggestions made in MMSE 47246 (StringCases using RegularExpression fails when there is missing data) to take into account errors that generated when some fields of interest are not found in every record. Ultimately, I wish to use the fields as headers for a dataset, creating a list of keys and values that can then be put into an association using AssociationThread[]. However, steps in this direction fail as the function slightly modified from MMSE 295606 returns errors for those elements for which the fields are absent (as are of course their prospective values). See below for the resultant list of strings containing {First[{}] in such cases.

Because the input is long, even when using only a three record my notebook, with the imported data is presented here:

https://www.dropbox.com/scl/fi/d2qtv7rx6nxnalr4t3ihy/ARCImportSebastesTest.nb?rlkey=4r8m2t4beuto2yc0xbzm8um4t&dl=0 .

In essence, the function I am trying to modify to work when missing fields are present is:

 hstr[x_ ;/ String_] := 
 First@StringCases[
 x, # ~~ Repeated[WhitespaceCharacter, {1, \[Infinity]}] ~~ 
   h : Shortest[___] ~~ EndOfLine | EndOfString :> h] &;
   

The fields of interest, not always present, are:

 fields = {"Institution code", "Institution ID", "Collection code", 
 "Occurrence ID", "Catalog number", "Basis of record", "Life stage",
  "Field number", "Collecting date", "Supplied as", 
 "Date precision", "Scientific name", "Genus", "Common name", 
 "Scientific name authorship", "Collector", "Identified by", 
 "Date identified", "Individual count", "Identified date", 
 "Verbatim event date", "Country", "State / Province", 
 "LocalityLocality", "Verbatim latitude", "Verbatim longitude", 
 "Latitude", "Longitude", "Geodetic datum", 
 "Minimum depth in meters", "Maximum depth in meters", 
 "Coordinate precision"};fields = {"Institution code", "Institution ID", "Collection code", 
 "Occurrence ID", "Catalog number", "Basis of record", "Life stage",
  "Field number", "Collecting date", "Supplied as", 
 "Date precision", "Scientific name", "Genus", "Common name", 
 "Scientific name authorship", "Collector", "Identified by", 
 "Date identified", "Individual count", "Identified date", 
 "Verbatim event date", "Country", "State / Province", 
 "LocalityLocality", "Verbatim latitude", "Verbatim longitude", 
 "Latitude", "Longitude", "Geodetic datum", 
 "Minimum depth in meters", "Maximum depth in meters", 
 "Coordinate precision"};
 

Converting the original input, which are lists of list, to a list of strings using:

 newinput = ToString[First@#] <> StringJoin[ToString /@ 
 ToString[Rest@#]] & /@ input;  
 

Then invoking the function one sublist of strings at a time:

 records =  Table[hstr[newinput[[i]]] /@ fields, {i, 1, 
 Length[newinput]}]

From which I get the following output:

{{"Atlantic Reference Centre", "ARC", "Atlantic Reference Centre Museum of Canadian Atlantic", "ARC_101486", "101486", "Preserved specimen", "Larvae", "H00443", "1977-05-12", "http://creativecommons.org/licenses/by/4.0/legalcode", "Day", "Sebastes Cuvier, 1829", "Sebastes", "rockcod", "Cuvier, 1829", First[{}], First[{}], First[{}], First[{}], First[{}], First[{}], "inferred from coordinates :f058 Passed", First[{}], First[{}], First[{}], First[{}], "43.033333", "-62.05", "EPSG:4326", "102.00", "102.00", "0.01667"}, {"Canadian Museum of Nature", "http://biocol.org/urn:lsid:biocol.org:col:34935", "Canadian Museum of Nature Fish Collection", "49a5b553-2753-7534-ddb6-11ba0a1fab74", "CMNFI 1980-0594.2", "Preserved specimen", First[{}], First[{}], "[date not supplied]", "http://creativecommons.org/licenses/by/4.0/legalcode", "Not Supplied", "Sebastes fasciatus Storer, 1854", "Sebastes", "Redfish", "Storer 1854", "Stevens, D.M.", "Lesage, Andrée", "1980-06", "5", "1980-06-", "(circa 1970)", "Canada", "Nova Scotia (Prov.)", First[{}], "47 03 50 N", "060 14 10 W", "47.0639", "-60.2361", "EPSG:4326", First[{}], First[{}], "Unknown"}, {"Canadian Museum of Nature", "http://biocol.org/urn:lsid:biocol.org:col:34935", "Canadian Museum of Nature Fish Collection", "49a5b553-2753-7534-ddb6-11ba0a1fab74", "CMNFI 1980-0594.2", "Preserved specimen", First[{}], First[{}], "[date not supplied]", "http://creativecommons.org/licenses/by/4.0/legalcode", "Not Supplied", "Sebastes fasciatus Storer, 1854", "Sebastes", "Redfish", "Storer 1854", "Stevens, D.M.", "Lesage, Andrée", "1980-06", "5", "1980-06-", "(circa 1970)", "Canada", "Nova Scotia (Prov.)", First[{}], "47 03 50 N", "060 14 10 W", "47.0639", "-60.2361", "EPSG:4326", First[{}], First[{}], "Unknown"}}

Along with errors indicating that First::nofirst: {} has zero length and no first element, for those records that have a particular field. As expected, each string in each element of record matches in length the length of fields, so that the two lists of strings can be combined with AssociationThread.

I've tried modify hstr[] to use the suggestion in 47246, but can't get it to work for my data as the conversion suggested in 295606 is more complicated than that in 47246.

How should I modify the function hstr[] so that it reports Missing rather than "First[{}]", when fields are misssing?

$\endgroup$
6
  • $\begingroup$ First has a default argument. Use hstr = First[#, {}] &@ StringCases[ arc110250, # ~~ Repeated[WhitespaceCharacter, {1, \[Infinity]}] ~~ h : Shortest[___] ~~ EndOfLine | EndOfString :> h] &;. You can also use Nothing or Missing as the default. $\endgroup$
    – Syed
    Commented Dec 29, 2023 at 7:56
  • $\begingroup$ I'm a bit confused. changed hstr to pass in the values to be StringCased'd. hstr[x_ ;/ String_] := First@StringCases[ x, # ~~ Repeated[WhitespaceCharacter, {1, [Infinity]}] ~~ h : Shortest[] ~~ EndOfLine | EndOfString :> h] &; to allow it to be used on more than one input. This gets me the output with the First[{}] values in the right place. But, if I do this hstr[x;/String_] := First[#, Missing[] &]@ StringCases[ x, # ~~ Repeated[WhitespaceCharacter, {1, [Infinity]}] ~~ h : Shortest[_] ~~ EndOfLine | EndOfString :> h] & It no longer works. $\endgroup$ Commented Dec 29, 2023 at 19:48
  • $\begingroup$ I can't understand what the x;/String_ construct is doing. The conditional statement uses /; to include a condition. Try: Clear[hstr]; hstr[x_String, field_String] := First[#, Missing[]] &@ StringCases[x, field ~~ Repeated[WhitespaceCharacter, {1, \[Infinity]}] ~~ h : Shortest[___] ~~ EndOfLine | EndOfString :> h]; Usage: hstr[x, #] & /@ fields $\endgroup$
    – Syed
    Commented Dec 30, 2023 at 0:47
  • $\begingroup$ Thought x :/String_ would restrict function from working only when x is a String. Now that I see how to pass # to the function, your code works as expected. Thanks for the help and the education. $\endgroup$ Commented Dec 30, 2023 at 22:00
  • $\begingroup$ For some reasons the system is regarding this as effectively the answer to the earlier question as there is no checkmark to accept your answer, even though this is a slightly different question to that you answered previously. Would vote to accept your answer, if I could. $\endgroup$ Commented Dec 30, 2023 at 22:02

0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Browse other questions tagged or ask your own question.