Model Parameter Estimation using EM Algorithm

Since the observed spectral density function $f(x)$, where $x$ denotes log-frequency, is considered to be generated from the model of multiple harmonic structures, the log-likelihood difference in accordance with an update of the model parameter $\theta$ to $\bar{\mbox{\boldmath $\theta$}}$ is

$$f(x)\log P_{\bar{\theta}}(x)-f(x)\log P_{\theta}(x)=\displaystyle f(x)\log \frac{P_{\bar{\theta}}(x)}{P_{\theta}(x)}.$$ (2)

Although Dempster formulated EM algorithm [8] in order to maximize the mean log-likelihood considering $f(x)$ as a probabilistic density function, it can also be formulated in a same way even if $f(x)$ is replaced with spectral density function. By taking expectation of both sides with respect to $P_{\theta}(n,k\vert x)$ which represents the probability of the $\{n,k\}$-labeled Gaussian distribution from which $x$ is generated, $Q$-function will be derived in the right-hand side. Given $Q$-function as

$$Q(\theta,\!\bar{\theta})\!=\!\!\displaystyle\sum_{k=1}^{K}\!\sum_... ...}\!\!\!\!\! P_{\theta}(n,\!k\vert x)f(x)\log P_{\bar{\theta}}(x\!,\!n\!,\!k)dx,$$ (3)

thus it yields

$$\displaystyle\int_{-\infty}^{\infty}\biggr\{f(x)\log P_{\bar{\theta}}(x)-f(x)\log P_{\theta}(x)\biggr\}dx$$

$$~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\ge Q(\theta,\bar{\theta})-Q(\theta,\theta).$$ (4)

By obtaining $\bar{\theta}$ which maximizes the $Q$ function, the log-likelihood of the model of multiple harmonic structures with respect to every $x$ will be monotonously increased. A posteriori probability $P_{\theta}(n,k\vert x)$ in equation (3) is given as

$$P_{\theta}(n,k\vert x) = \displaystyle\frac{P_{\theta}(x,n,k)}{P_{\theta}(x)},$$ (5)

$$= \displaystyle\frac{w_n^k\cdot g(x\vert\mu_k\!+\!\log n,\sigma^2)}{\displaystyle\sum_{n}\sum_{k}w_n^k\cdot g(x\vert\mu_k\!+\!\log n,\sigma^2)},$$ (6)

$$g(x\vert x_0,\!\sigma^2) = \displaystyle\frac{1}{\sqrt{2\pi \sigma^2}}\exp\bigg\{-\displaystyle\frac{(x-x_0)^2}{2\sigma^2}\bigg\},$$ (7)

where $g(x\vert x_0,\!\sigma^2)$ is a Gaussian distribution. By the iterative procedure of the two steps as follows, the model parameter $\theta$ locally converges to ML estimates.

Initial-step

Initialize the model parameter $\theta$.

Expectaion-step

Calculate $Q(\theta,\bar{\theta})$ with equation (3).

Maximization-step

Maximize $Q(\theta,\bar{\theta})$ to obtain the next estimate

$$\mbox{\boldmath$\theta$} =\mathop{\rm argmax}_{{\bar{\theta}}} Q(\theta,\bar{\theta}).$$ (8)

Replace $\bar{\mbox{\boldmath $\theta$}}$ with ${\mbox{\boldmath $\theta$}}$ and repeat from the Expectation-step.