1
$\begingroup$

I have several data sets that I am working with in Mathematica. There are several columns of data but I am going through it one column at a time. Now I have some columns where respondents were able to click only one option, and some where they were able to click several options, for example:

sources = {"Tumblr", "Other", "Tumblr", "Tumblr", "Tumblr", "Tumblr", 
"Tumblr","Other", "Tumblr", "Tumblr", "Tumblr", "Tumblr", "Tumblr", 
"Other","Other", "Tumblr", "Discord server (please specify)", 
"Tumblr","Tumblr", "Tumblr", "Other", "Other", "Direct recommendation, such 
as by a friend or family member", "Tumblr"}

and

religions = {{"Jewish"}, {"Agnostic"}, {"Atheist"}, {"Jewish", "Agnostic",
"Spiritual"}, {"Atheist"}, {"Spiritual"}, {}, {"Atheist",
"Agnostic"}, {"Spiritual"}, {"Christian - Protestant"}, {"Agnostic"},
{"Jewish"}, {"Agnostic", "Pagan - Other"}}

These data sets are all 9,000 to 15,000 responses long, so manually changing the responses isn't an option. I managed to use NetEncoder but think One-Hot encoding would better serve my purposes to find networks/communities, but am having difficulty thinking about how to go from the Integer to One-Hot Encoding.

So for example, the religion options were:

religionOpt = {"Christian - Protestant", "Christian - Catholic", "Christian 
Other", "Jewish", "Muslim", "Buddhist", "Hindu", "African Traditional
and Diasporic", "Atheist", "Agnostic", "Pagan - Nordic", "Pagan -
Kemetic", "Pagan - Hellenic", "Pagan - Other", "Spiritual", "Other
ethnic or indigenous religion", "Other"}
religionOptEnc = {1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17}

and the encoded data set I end up getting is (I already have a method to get it in this form):

religionEnc = {{4}, {10}, {9}, {4, 10, 15}, {9}, {15}, {}, {9, 
  10}, {15}, {1}, {10}, {4}, {10, 14}}

I saw this question One Hot Encode columns with categorical data but when I (quickly) tried to adopt it for my data, for the sources, I was getting the same results as NetEncode, and for religion I was stumped, as in this example, the data has the form

{{a,b},{c,d},...}

whereas my data sets have variable length, with options such as

{{a},{b,c,d},...}

Any advice from where to go from here would be appreciated. I initially think about tackling the problem by writing some For[] loop, but I remember reading some response on this site that in Mathematica it's usually better to avoid For[] loops if possible?

EDIT: The desired output for the sources would be: enter image description here and an example of religions would be: enter image description here and so on. I cut a lot of the rows and options since I just wanted a quick example of what I was trying to get the data to look like. I already have all the data Integer Encoded, but I'm hitting a mental block about this final step.

$\endgroup$
  • $\begingroup$ Would religions /. Thread[religionOpt -> religionOptEnc] work for you? This returns your religionEnc set above. Indeed, if the encoding number is always sequential, you could use religions /. Thread[religionOpt -> Range@Length@religionOpt]. If not, can you show us what your desired output should be? $\endgroup$ – MarcoB Mar 29 at 15:36
  • $\begingroup$ @MarcoB Good idea, I'll update the post with what I desire the output to be. Thank you for pointing this out. $\endgroup$ – Jomy Blue Mar 29 at 15:38
2
$\begingroup$

The following helper function generates the unique encodings as a replacement list, starting from a flat list of possible values such as sources or religionOpt in your example:

ClearAll[generateOneHotEncoding]
generateOneHotEncoding[list_] := Module[
  {unique, len},
  unique = DeleteDuplicates[list];
  len = Length[unique];
  Thread[unique -> IntegerDigits[2^(Range@len - 1), 2, len]]
]

You can then use it as follows:

sources /. generateOneHotEncoding[sources] // MatrixForm

single choice output

When multiple choices are possible, the output becomes less pretty:

religions /. generateOneHotEncoding[religionOpt] // TableForm[#, TableDepth -> 2] &

multilevel output

$\endgroup$
  • $\begingroup$ Actually, I just noticed that you were putting in the options in your code, somehow I completely missed that. So my question is: just to confirm, but the columns for the output are in the same order as the fed in options, i.e., religionOpt? $\endgroup$ – Jomy Blue Mar 29 at 17:09

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.