How do I compose a list from an Association such that the final product matches a fixed percentage of each of four subsets?

Question

My question is in two parts. The first is about picking discrete elements inside a list of Associations to match a fixed percentage of a final sub-list; the second is about overcoming the rounding problem the first part creates.

I have data in a List of Associations, a portion of which looks like this

{<|"Sector" -> "A", "ID" -> 1101, "Year" -> 1990, "AnnReturn" -> 0.0946|>, <|"Sector" -> "A", "ID" -> 1102,  "Year" -> 1990, "AnnReturn" -> 0.0901|>, <|"Sector" -> "A", "ID" -> 1103, "Year" -> 1990, "AnnReturn" -> 0.00876|>, <|"Sector" -> "A", "ID" -> 1104, "Year" -> 1990, "AnnReturn" -> 0.0242|>, <|"Sector" -> "A", "ID" -> 1105, "Year" -> 1990, "AnnReturn" -> 0.0783|>, <|"Sector" -> "A", "ID" -> 1106, "Year" -> 1990, "AnnReturn" -> 0.0433|>, <|"Sector" -> "A", "ID" -> 1107, "Year" -> 1990, "AnnReturn" -> 0.0253|>, <|"Sector" -> "A", "ID" -> 1108, "Year" -> 1990, "AnnReturn" -> 0.1144|>, <|"Sector" -> "A", "ID" -> 1109, "Year" -> 1990, "AnnReturn" -> 0.1036|>, <|"Sector" -> "A", "ID" -> 111, "Year" -> 1990, "AnnReturn" -> 0.1216|>, <<145845>>}

The first Key = Sector has four possibilities, A, B, C and D

I need to compose a sub-list OF UNIQUE VALUES which has a fixed (but uneven) percentage of each sector. Here is an example where the proportions correspond to "A","B","C" and "D"

{0.13, 0.19, 0.28, 0.4}

Note the decimals must add to unity.

So, to be completely tedious, a sub-list of, say, 1033 elements would have

Round[1033 {0.13, 0.19, 0.28, 0.4}] 

to get this

(*{134, 196, 289, 413}*)

...which adds to 1032. What I need should be the total sought less the first three proportions, producing the fourth. I am sure there is a better way than this, but this works

Join[Round[1033 #], {1033 - Plus @@ Round[1033 #]}] &[{0.13, 0.19, 
  0.28}]

@MarcoB suggested this

grupBySec = GroupBy[netLoA, #"Sector" &];
RandomChoice[{0.13, 0.19, 0.28, 0.4} -> grupBySec]

but it only randomizes the sector. It looks like the Documentation for weights but those examples do not use Associations

Answer 1

Let me first generate some fake data fakeList that has the same structure as your list:

n = 10000;
fakeList = MapThread[
   <|"Sector" -> #1, "ID" -> #2, "Year" -> #3, "AnnReturn" -> #4|> &,
   {
     RandomChoice[{"A", "B", "C", "D"}, n],
     Range[1101, 1101 + n - 1],
     RandomInteger[{1990, 2000}, n],
     RandomReal[{0, 0.2}, n]
   }
];

Let's now assign your weights to a variable for readability:

weights = {0.13, 0.19, 0.28, 0.4};

---

With those in hand, let's reorganize the list, grouping it by "Sector". We also sort it to make sure that the groups are listed in alphabetical order of sector, so the relative weights are correctly assigned to each sector later on:

organizedList = SortBy[#Sector &@*First]@GatherBy[fakeList, #Sector &]

Let's now generate the number of samples to pick for each group, based on the number of desired samples overall. I use your example of 1033 samples:

With[{samples = 1033},
 counts = {##, samples - Total[{##}]} & @@ Round[samples weights[[;; 3]]]
]

(* Out: {134, 196, 289, 414} *)

Now let's pick a random selection without replacement of samples according to the output above from each grouped list. We do that using RandomSample, which never samples any element more than once from its input.

picks =
  MapThread[
    RandomSample[#1, #2] &,
    {organizedList, counts}
  ] // Flatten

(* Out: 
{<|"Sector" -> "A", "ID" -> 1952, "Year" -> 1994, "AnnReturn" -> 0.163849|>, 
 <|"Sector" -> "A", "ID" -> 1822, "Year" -> 1998, "AnnReturn" -> 0.0230599|>, 
 <|"Sector" -> "A", "ID" -> 1670, "Year" -> 1992, "AnnReturn" -> 0.0859481|>, 
 
 <<1027>>,

 <|"Sector" -> "D", "ID" -> 1651, "Year" -> 1999, "AnnReturn" -> 0.102282|>, 
 <|"Sector" -> "D", "ID" -> 2074, "Year" -> 1996, "AnnReturn" -> 0.192641|>, 
 <|"Sector" -> "D", "ID" -> 1539, "Year" -> 1990, "AnnReturn" -> 0.00170126|>}
*)

Finally, let's check that we did indeed obtain the right number of samples from each "Sector" in the output:

CountsBy[#Sector &][picks]

(* Out: <|"A" -> 134, "B" -> 196, "C" -> 289, "D" -> 414|> *)

... and that there are no duplicates in the output:

DeleteDuplicates[picks] == picks
(* True *)