Probability Distributions

Exponential family

b (x) e x p (θ^{T} T (x) - A (θ))

Includes: Bernoulli , Binomial, Gaussian, Poisson, Gamma, Exponential, Dirichlet

It shows a specific linear characteristic for $x$ .

α^{x} (1 - α)^{1 - x}, x \in {0, 1} = e x p (l o g (α^{x} (1 - α)^{1 - x})) = e x p (x l o g (α) + (1 - x) l o g (1 - α))) = e x p (x (l o g (\frac{α}{1 - α}) + l o g (1 - α)))

b (x) = 1 T (x) = x θ = l o g (\frac{α}{1 - α}) A (θ) = - l o g (1 - α) = l o g (\frac{1}{1 - α}) = l o g (1 + \frac{α}{1 - α}) = l o g (1 + e x p (θ))

\frac{1}{2 π σ} e x p (- \frac{( x - μ ) ^{2}}{2 σ ^{2}}) = \frac{1}{2 π} e x p (- l o g σ) \cdot e x p (- \frac{( x - μ ) ^{2}}{2 σ ^{2}}) = \frac{1}{2 π} e x p (- l o g σ - \frac{x ^{2}}{2 σ ^{2}} - \frac{μ ^{2}}{2 σ ^{2}} + \frac{xμ}{σ ^{2}}) = \frac{1}{2 π} e x p (\frac{xμ}{σ ^{2}} - \frac{x ^{2}}{2 σ ^{2}} - l o g σ - \frac{μ ^{2}}{2 σ ^{2}})

b (x) = \frac{1}{2 π} T (x) = [x, x^{2}] θ = [\frac{μ}{σ ^{2}}, \frac{- 1}{2 σ ^{2}}] A (θ) = l o g σ + \frac{μ ^{2}}{2 σ ^{2}}