Why normalize data before training it in Azure ML?

Question

I've been experimenting with normalizing data before training a model, and I don't get it. I understand that training the model theoretically should work better when the data is first normalized, especially when using geometric algorithms like SVM.
But when I try to do this in Azure ML. I don't see any improvements. In fact, the results get worse half of the time.
For example, I tried creating a two-class prediction model using the German Credit Card UCI data set. According to this article, it's best to normalize the data using the tanh method:
http://azure.microsoft.com/en-us/documentation/articles/machine-learning-sample-credit-risk-prediction/
This is what I get when I try predicting column 21 in the data set:
No normalization --> UAC = 0.740
Normalize using Z-score --> UAC = 0.740
Normalize using MinMax --> UAC = 0.740
Normalize using Logistic --> UAC = 0.670
Normalize using LogNormal --> UAC = 0.733
Normalize using tanh --> UAC = 0.696
As you see, using the suggested tanh normalization results in almost the worst performance, and doing no normalization at all yields the best results.
Any suggestions what might be going on?

Answer 1

Hi Andreas,
Feature normalization/scaling indeed helps a lot with gradient descent convergence. However, it shouldn't be seen as a perf-optimization technique only. It is an important data pre-processing step for algorithms which optimize a distance function. In
those cases the data should be normalized so that features contribute proportionally to the distance computation. Classifiers like the Bayes Point Machine don't work this way and hence don't need feature normalization, but SVMs certainly do.
Here is a decent read on the topic (reviewed by Schölkopf himself).
-Y-