How to generate names with a GAN?

Question

I'd like a tutorial for text generation using GANs, specifically for names. I have a list of company names and would like the network to generate novel ones.

This is one of the only few examples of GANs in MMA:

• Generative Adversarial Network

Here's the training data:

train = CompanyData[#, "Name"] & /@ CompanyData[];
Length[train] (* 77,563 *)
RandomSample[train, 3] (* {"Microsoft", "MST Investment", "Cyberaton"} *)

Here are the starting points and resources that I've found:

• https://becominghuman.ai/generative-adversarial-networks-for-text-generation-part-1-2b886c8cab10
• https://poloclub.github.io/ganlab/
• https://arstechnica.com/information-technology/2017/05/an-ai-invented-a-bunch-of-new-paint-colors-that-are-hilariously-wrong

Answer 1

Here is a simple example that hopefully can get you started.

For simplicity, we use the names that up to 25 characters and only contains the alphabet.

allCompanyNames = 
  Select[CompanyData[#, "Name"] & /@ CompanyData[], StringQ];

vocabulary = Alphabet[]~Join~ToUpperCase[Alphabet[]]~Join~{" "};

maxLen = 25;

companyNames = With[{
    extraVoc = 
     Complement[Flatten[Characters /@ allCompanyNames], vocabulary]
    },
   Select[allCompanyNames, 
    StringLength[#] <= 25 && StringFreeQ[#, extraVoc] &]
   ];

We will use "." as the end token and pad all names to the same length

charLen = maxLen + 1;
companyNamesPadded = 
  StringPadRight[#, charLen, "."] & /@ companyNames;
vocLen = Length[vocabulary] + 1;

This are what the padded names look like

RandomSample[companyNamesPadded, 3]
(*{"Azot PJSC.................", "Vinnytsyamyaso............", "Macofil..................."}*)

Now define the generator and the discriminator. To make it simple, we only use linear layers.

discriminator = 
 NetChain[{LinearLayer[256, "Input" -> {charLen, vocLen}], 
   ElementwiseLayer["SELU"], DropoutLayer[0.4], LinearLayer[128], 
   ElementwiseLayer["SELU"], DropoutLayer[0.4], LinearLayer[{}], 
   LogisticSigmoid}, "Input" -> {charLen, vocLen}]

latenLen = 100;
generator = NetFlatten@NetChain[{
        LinearLayer[{charLen, 512}], ElementwiseLayer["SELU"], 
    DropoutLayer[0.3], LinearLayer[{charLen, 256}], 
    ElementwiseLayer["SELU"], DropoutLayer[0.3], 
    LinearLayer[{charLen, 128}], ElementwiseLayer["SELU"], 
    LinearLayer[{charLen, vocLen}], ElementwiseLayer["SELU"], 
    SoftmaxLayer[]
    }, "Input" -> latenLen]

Define a data generator that yields random seeds and sampled names.

getRandomLatent[batchSize_] := 
 Map[NumericArray[#, "Real32"] &, 
  RandomVariate[NormalDistribution[], {batchSize, latenLen}]]
datagen = 
  Function[<|
    "Sample" -> 
     UnitVectorLayer[vocLen][
      NetEncoder[{"Characters", {vocabulary, "."}}][
       RandomSample[companyNamesPadded, #BatchSize]]], 
    "Latent" -> getRandomLatent[#BatchSize]|>];

Train the network

trained = NetTrain[NetGANOperator[{generator, discriminator}],
  {datagen, "RoundLength" -> Length[companyNamesPadded]},
  TrainingUpdateSchedule -> {"Discriminator", "Generator"},
  MaxTrainingRounds -> 60, TargetDevice -> "GPU", BatchSize -> 512
  ]

Extract the generator

trainedgen = NetExtract[trained, "Generator"]
dec = NetDecoder[{"Characters", {vocabulary, "."}}]

Generate names by sampling the latent space

Table[First@
  StringSplit[
   dec@trainedgen[RandomVariate[NormalDistribution[], latenLen]], 
   "."], {10}]
(*{"Lokeirrle", "Lracian", "Lofcia", "Meritian", "Siriar", "Palteo", "Garirlso ", "Loepia", "Lraedm a", "Demiiiaa"}*)