Getting a negative NLL from outputs.loss - with multi-target normalized data
Hello!
I followed the blog and created my own TimeSeriesTransformerForPrediction model, using preprocessing with GluonTS, and training using Pytorch Lightning.
For some reason, on some of my iterations and evaluations for my model I'm getting a negative outputs.loss. As this is negative log likelihood, this doesn't make sense.
Any idea why this can happen? Or maybe point me in a direction to figure out my problem?
I'm using multi-variate prediction (input_size=5).
My data is normalized (so in particular can have negative values). Of course, the output distribution is AffineTransformed(Independent(StudentT())) - and the AffineTransformed has loc=0.0 (single float), and scale is a torch tensor of ones, of size torch.Size([256, 1, 5]). (Hence - normalized batch of size 256)
Any ideas would be great. Thanks!
thanks,
@yuvalarbel
so the loss is - distrb.log_prob(ground-truth) and so as the model gets better and better this value will tend to negative infinity... thus negative values are actually good. Note that this is not the proper likelihood but rather the likelihood from the distribution and the important thing is that it gets smaller and smaller.
Great, thank you!