Hi,
I used max-min normalisation on a regression problem. After I trained the model, I now receive a prediction between 0 and 1 from the model. Is there a way that I can ask the model to give me a prediction as an un-normalised number?
Hi @JWalker,
Did you normalize the target column or only the input columns?
And it sounds like you may be using some activation function on the last layer, make sure to set activation to None so that it can predict any values 
Thanks @robertl,
Yes, I normalised the output in the data wizard but not the input images. Should I be doing that? Normalising both?
My model follows the default, an input - > convolution -> dense (128) -> dense (1) - > output
The dense layers have no activation functions, but the convolution layer works best with an activation function. In fact, if I don’t use an activation function on the convolution or I don’t normalise the data in the data wizard it will crash.
I also use centred RMSProp and the quadratic loss function.
Here is my training window when it has finished. You can see that the values on the left of the prediction are low. They are all between 0 and 1. However, my dataset has values from -200 to 200 (approx).
Hence, I need to reverse the normalisation to obtain the real prediction. Obv I could do it myself using the data’s min and max, but I am wondering if there is a way to ask Perceptilabs for it.
Right now, I use the following code to ask for my prediction:
prediction1 = model(image)
print('The prediction is ', (np.asarray(prediction1['Price'])))
Is there some way to alter that to obtain the un-normalised prediction?
Edit: Also in Spyder, I receive this warning when I try to call the prediction:
WARNING:tensorflow:No training configuration found in save file, so the model was *not* compiled. Compile it manually.
Oh good, I’m glad I saw this - I was wondering about the correct usage of Normalisation in case I want to use it shortly!
I hope @robertl will confirm, but I think he was saying that one should either normalise inputs and target or none of them.
With regard to the TF warning re compilation; I’ve had that too but it didn’t seem to affect the ability to predict
Stackoverflow has a bit more on this here.
Thanks, I normalised both, but still received a prediction between 0 and 1. Let me read stackflow and see if I can figure it out.
I think you probably don’t want to normalize the targets
Normalizing the inputs is great because it keeps the input in a close range, making training and inference easier.
However, if you normalize the targets you will bring them down into a range of 0 to 1, which you typically don’t want as you likely want the model to predict in the original data’s range. For example, if you are predicting prices you want the model to predict the actual dollar price and not some normalized value between 0 and 1.
As soon as you stop normalizing the targets you should see predictions that go outside the 0 and 1
1 Like
Thank you, let me try.
Yes, that has worked, but of course increased the loss over the run. I guess that was artificially low because the output was only because 1 and 0, so it was probably an illusion.
What is interesting, is that the batch normalisation that you have included with the update works better than normalising through the data wizard.
2 Likes
Batch normalization is a little different than just “normal” normalizing.
The batch normalization works by adding a layer that normalizes the output of that component - although I need to double check on this one, it might make more sense if it normalizes the input to a component rather than the output. You can read more about that layer here: https://www.tensorflow.org/api_docs/python/tf/keras/layers/BatchNormalization
The “normal” normalizing, or instance normalization, works by normalizing the entire dataset.
The results will very quite heavily depending on where the batch normalization is used, and thanks to the fact that it just does a local normalization within a batch.
I’m glad that you noticed batch norm works better for you in this case though
1 Like
Thanks Robert. That makes sense, it is basically supplying the z-score to the next layer, which means that the next layer ought to receive similar inputs, which ought to help it generalise better.
Thanks @JWalker and @robertl for clarifying various things…
In particular, I had never heard the process described as “z-score normalisation”, so I learned something useful there!
For any one else interested, the Wikipedia page on [Standard Score] explains the details of this:
Standard scores are most commonly called z -scores ; the two terms may be used interchangeably, as they are in this article. Other terms include z-values, normal scores , standardized variables …
1 Like
Sure, that’s just my own explanation of it. (x - mean) / sd is a z-score in stats, so that’s just how I summed it up.
1 Like