# Normalize data in feature engineering

Here is the dataset to be used to train the model.

data = RandomReal[{-1, 1}, {100, 10}]; (*100 samples with 10 features each*)

(*Normalize each feature to be standard gaussian N(0,1)*)

μ = Mean[data]
σ = StandardDeviation[data]

normaldata = ((# - μ)/σ) & /@ data;
(*Plot one the first feature, but the plot does not look like normal distribution!*)

ListPlot[normaldata[[1]], Joined -> True]


What is the problem?

• Additions and multiplications to Uniformly distributed data result in .... Uniformly distributed data – Dr. belisarius Feb 21 '16 at 5:31
• Shouldn't the values for the first feature correspond to normaldata[[All, 1]]? Also, if you want to generate normally distributed multivariate data, you should look into MultinormalDistribution: RandomReal generates uniformly distributed data, as Belisarius mentioned. – MarcoB Feb 21 '16 at 5:31
• The plot does look like points taken from a normal distribution, with mean $0$ and and $\sigma = 1$. What were you expecting? – David G. Stork Feb 21 '16 at 5:32
• Try: Graphics[{Point[{#, 0}] & /@ normaldata[[1]], {Red, PointSize[0.02], Point[{\[Mu][[1]], 0}]}}]. – David G. Stork Feb 21 '16 at 5:51

Way back in the days when I was programming 8-bit microprocessors in assembly language and I had to keep things down to simple arithmetic, I generated pretty good approximations to normal variates with code that translates into Mathematica as follows:

### Tools

 normalVariate[μ_, σ_] := Total[RandomReal[{μ - 4 σ, μ + 4 σ}, 5]]/5
stats[sample_] := Through[{Mean, StandardDeviation}[sample]]


### Application

SeedRandom[1];
sample = Table[normalVariate[0, 1], 1000];
stats[sample]


{0.052734, 1.03348}

Histogram[sample]


This divides the full sample up in 100 sub-samples with ten value in each.

data = Partition[sample, 10];


Perhaps you might use this approach to generate your training sets.

• +1 Which micros? We want details! – Dr. belisarius Feb 21 '16 at 7:41
• @Dr.belisarius. Started with 8080, then 8085, Z80, and 8088, – m_goldberg Feb 21 '16 at 7:42
• Ha! when men were men machines weren't a New Kind of anything :). I miss the 6502's too :) – Dr. belisarius Feb 21 '16 at 7:44
• @Dr.belisarius. The 6502 is one I never programmed. I worked in an Intel-oriented shop. The Z80 was in my home computer, a S100 bus thing that I built with my very own soldering iron and screw-driver. A very advanced machine with two 256-Kbyte floppy drives (yes, two!) and 24 Kbytes of ram. I was the envy of all my fellow nerds. I had a VT100 terminal attached to it. (This was in 1978 -- the 8080 was in '74) – m_goldberg Feb 21 '16 at 7:54
• +1 The man can be recreated as a programmer! – user36273 Feb 21 '16 at 10:05