Gaussian Process (GP) is Easy to Understand In the beginning This is a follow-up blog of "Variantial AntoEncoder", in which I tried to explain VAE in a Neural Network way before relating it to Bayesian machine learning. As we know, VAE is a unsupervised learning method; now, I will try to explain the most important supervised learning method in Bayesian machine learning, called Gaussian Process. I have tried many many times to understand GP in different ways, through papers, books, videos, and had even written my own GP software in Theano.

Neural Network Infers Bayes Congratulations, you have made to the third and final part! Equipped with the Bayesian language, we can start to look at the "special" regularization term in the VAE loss function and try to make sense of it. Most articles talk about "variational inference" and derive the equations of variational lower bound and KL divergence. I encourage you to read this blog from Eric Jang for more details of variational inference.

Bayesian Language I have tried to conquer Bayesian modeling several times since 2010; read a lot paper, couple of books, and took some online classes. Yes, you can remember math terms, you may follow what they say in the paper while you are reading it, you may even be able to derive the equations just as they do. But what's hard is to really understand what's going on behind those equations, without which you are bound to forget what you think you know after a certain period.

Variational AutoEncoder (VAE) is Easy to Understand Before Everything I assume you, like me, know a bit of neural networks. I assume you, also like me, have attempted many times to understand Bayesian and have either failed or reached a state of "almost got it". After all, Bayesian people speak a different language from NN people, which can be counterintuitive at times. Among the hardest, there is no Andrej Karpathy yet on this topic.