Sungjoo Ha

Directed Grahpical Model

Bayesian network

encoding conditional dependency using graphical representation

DAG

$$p(x_1, x_2, \ldots, x_n) = \prod_{i = 1}^n p(x_i | pa(x_i))$$

ex) Markov assumption

naive bayes

$$p(x_1, x_2, \ldots, x_d, y) = p(x_1|y)p(x_2|y) \cdots p(x_d|y) p(y)$$

Markov chain

plate notation

$p(a, b) = p(a) p(b|a) = p(b) p(a|b)$

$p(a, b, c) = p(a) p(b|a) p(c|a ,b) = p(c) p(b|c) p(a | b, c)$

non-uniquuue.

h-to-t h-to-h (v-structure) t-to-t

D-separation

A path from a node to another node is said to be blocked by a set $A$ of nodes, if one of the following is true

the path contains a node $t \in A$ such that it is of head-tail or tail-tail type

there is a head-head node in the path such that neither the node itself nor any of its descendants is in $A$

if $A$ to $B$ is blocked by $C$ then $A \perp B | C$.

Bayes ball algorithm

inference

$p(x_h | x_v, \theta) = \frac{p(x_h, x_v | \theta)}{p(x_v|\theta)} = \frac{p(x_h, x_v|\theta)}{\sum_{x'_h} p(x'_h, x_v|\theta)}$

if only some of hidden

$p(x_q | x_v, \theta) = \sum_{x_n} p(x_q, x_n | x_v, \theta)$

learning. in the graphical models literature, it is common to distinguish between inference and learning

$$\hat{\theta} = \mathrm{argmax}_{\theta} \sum_{i = 1}^N \log p(x_{i, v}|\theta) + \log p(\theta)$$

Mixture models

EM for GMM

maximize $p(X|\theta)$ w.r.t. $\theta$