But how does Surprise lead to Kullback-Leibler Divergence?

If you are interested in math and neural networks, it is impossible that you haven’t heard of Kullback-Leibler Divergence or KL-Divergence as it is popularly called in the deep learning community. It plays a pivotal role, especially in Generative Adversarial Networks (GANs). While the definition and equation seem pretty straightforward, I have a question.

Visit Now