3
$\begingroup$

Some scientific libraries, like Scikit-Learn for Python are able to create normalizing/standardizing functions based on data that is previously given. Looking at the documentation for Normalize[] and Standardize[], I don't see any type of functionality for creating a normalizing or standardizing function.

Is there any way that I can use Normalize[] or Standardize[] without needing to load all of the previously used data, in the effort to make my program more efficient?

To clarify, I would like to provide data for these functions that is (hopefully) representative of the other data I will be feeding the functions, and then use the result to create a function that normalizes/standardizes any new data that is sent in. Thanks!

Improve this question
$\endgroup$
2
  • 3
    $\begingroup$ If you want the new data to have the same mean and standard deviation as the old data, then standardize the new data to have a mean of 0 and standard deviation of 1, multiply that standardized new data by the old standard deviation and then add the old mean. If you don't have the summary statistics that you want to match, then you're out of luck. (And "representative" is a pretty loose term.) $\endgroup$
    – JimB
    Commented Jan 3, 2020 at 19:45
  • $\begingroup$ Thanks, I think that should work for what I am doing. And good point about "representative" - I edited my question a little. $\endgroup$
    – Jmeeks29ig
    Commented Jan 3, 2020 at 19:50

1 Answer 1

Reset to default
4
$\begingroup$

The following applies JimB's suggestion; it would work if you have enough data to be representative of the "old" distribution:

ClearAll[standardizeTo]
standardizeTo[oldData_, newData_] :=
 Module[
   {oldMean, oldStdev},
   {oldMean, oldStdev} = Through[{Mean, StandardDeviation}[oldData]];
   oldStdev Standardize[newData] + oldMean
 ]

Let's create some play data from two normal distributions with markedly different parameters:

old = RandomVariate[NormalDistribution[10, 3], 100];
new = RandomVariate[NormalDistribution[12000, 1000], 100];

Apply the function to obtain an appropriately standardized data set from your second one:

restandardized = standardizeTo[old, new];

Check that the descriptors of the re-standardized data are indeed similar to the ones of the old data set:

Through[{Mean, StandardDeviation}[restandardized]]
(* Out: {9.85804, 2.75594} *)
Improve this answer
$\endgroup$
1
  • $\begingroup$ Thank you, this seems like it will work really well for what I am trying to do. $\endgroup$
    – Jmeeks29ig
    Commented Jan 3, 2020 at 23:44

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.