Some scientific libraries, like Scikit-Learn for Python are able to create normalizing/standardizing functions based on data that is previously given. Looking at the documentation for Normalize[] and Standardize[], I don't see any type of functionality for creating a normalizing or standardizing function.

Is there any way that I can use Normalize[] or Standardize[] without needing to load all of the previously used data, in the effort to make my program more efficient?

To clarify, I would like to provide data for these functions that is (hopefully) representative of the other data I will be feeding the functions, and then use the result to create a function that normalizes/standardizes any new data that is sent in. Thanks!

  • 3
    $\begingroup$ If you want the new data to have the same mean and standard deviation as the old data, then standardize the new data to have a mean of 0 and standard deviation of 1, multiply that standardized new data by the old standard deviation and then add the old mean. If you don't have the summary statistics that you want to match, then you're out of luck. (And "representative" is a pretty loose term.) $\endgroup$ – JimB Jan 3 '20 at 19:45
  • $\begingroup$ Thanks, I think that should work for what I am doing. And good point about "representative" - I edited my question a little. $\endgroup$ – Jmeeks29ig Jan 3 '20 at 19:50

The following applies JimB's suggestion; it would work if you have enough data to be representative of the "old" distribution:

standardizeTo[oldData_, newData_] :=
   {oldMean, oldStdev},
   {oldMean, oldStdev} = Through[{Mean, StandardDeviation}[oldData]];
   oldStdev Standardize[newData] + oldMean

Let's create some play data from two normal distributions with markedly different parameters:

old = RandomVariate[NormalDistribution[10, 3], 100];
new = RandomVariate[NormalDistribution[12000, 1000], 100];

Apply the function to obtain an appropriately standardized data set from your second one:

restandardized = standardizeTo[old, new];

Check that the descriptors of the re-standardized data are indeed similar to the ones of the old data set:

Through[{Mean, StandardDeviation}[restandardized]]
(* Out: {9.85804, 2.75594} *)
  • $\begingroup$ Thank you, this seems like it will work really well for what I am trying to do. $\endgroup$ – Jmeeks29ig Jan 3 '20 at 23:44

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.