{ "cells": [ { "cell_type": "markdown", "id": "sustainable-aging", "metadata": {}, "source": [ "# Time Series Analysis \n", "\n", "## Topics\n", "\n", "* Stylized Facts\n", " * Stylized Facts\n", " * Log-Returns\n", "* Stationarity\n", " * Stationarity\n", " * Asymptotics of Stationary Sequences\n", " * Standard Facts on Conditional Expectation\n", " * MDS\n", " * Wold Decomposition\n", "* AR Processes\n", " * ACF\n", " * Bartlett's Formula\n", " * Ljung-Box Test\n", " * AR(1)\n", " * Causal Processes\n", " * AR(2)\n", " * Weak Stationarity of AR(p)\n", " * Partial Correlation Coefficients\n", " * PACF\n", "* MA Processes\n", " * MA(q)\n", " * Invertibility of MA Processes\n", " * Formal Notations\n", "* ARMA Models\n", " * AMRA Models\n", " * ARMA(1, 1)\n", " * ARMA(p, q) Analysis\n", " * ARIMA, Differencing to Obtain Stationarity\n", " * Dickey-Fuller Test\n", " * Parameter Estimation\n", " * Yule Walker Equations\n", " * Likelihood Methods\n", " * statsmodels\n", " * Forecasting\n", "* Non-Stationary to Stationary\n", " * Box-Cox Transformation\n", " * Trend and Seasonal Components\n", " * Differencing\n", "* ARCH/GARCH Modeling\n", " * Motivation\n", " * ARCH(1)\n", " * AR(1)/ARCH(1)\n", " * ARCH(p)\n", " * ARCH Properties\n", " * ARCH and Stylized Facts\n", " * Weaknesses of ARCH Model\n", " * From ARCH to GARCH\n", " * GARCH(1, 1)\n", " * Fitting ARCH to S&P 500 Data\n", " * GARCH(p, q)\n", " * GARCH Forecasting\n", " * Engle Test for ARCH Effects\n", " * GARCH Forecasting Example in Risk Management\n", " * Other Volatility Models\n", "* Multivariate Time Series\n", " * Multivariate Time Series\n", " * Vector Autoregressioive Processes \n", " * Stationarity of VAR(1) Processes\n", "* Cointegration\n", " * Cointegration \n", " * Johansen Test\n", " * Cryptocurrency Example\n", "* State Space Modeling\n", " * State Space Models\n", " * Kalman Recursions: Kalman Prediction & Filtering\n", " * Example (Linear Regression)\n", " * AR, MA in State Space Form\n", " * Bayesian Background to Kalman Methods\n", " * Stochastic Volatility" ] }, { "cell_type": "markdown", "id": "happy-asian", "metadata": {}, "source": [ "## Notes\n", "\n", "* Stylized Facts\n", " * Stylized Facts\n", " * Log-Returns\n", "* Stationarity\n", " * Stationarity\n", " * Weakly stationary\n", " * mean and var are constant\n", " * Cov$(X_s, X_t)$ only depends on the lag $|s-t|$\n", " * weak stationarity $+$ jointly normal distributions $\\implies$ strict stationarity\n", " * WN$(0, \\sigma^2)$\n", " * weakly stationary process with mean 0\n", " * ACovF is $\\{\\sigma^2, 0, 0, \\ldots\\}$\n", " * Asymptotics of Stationary Sequences\n", " * Standard Facts on Conditional Expectation\n", " * MDS\n", " * Martingale: $E(X_{t+1}|\\mathcal F_t) = X_t\\quad\\forall t\\ge 0$\n", " * MDS: $E(X_{t+1}|\\mathcal F_t) = 0\\quad\\forall t\\ge 0$, hence $E(X_{t+1}) = 0$.\n", " * 3 types of noise processes: iid, MDS and weakly stationary processes\n", " * {iid, zero mean} $\\subset$ {MDS}\n", " * {Common Finite Variance MDS} $\\subset$ {White Noise Processes}\n", " * MDS with common finite variance has CLT\n", " * Wold Decomposition\n", " * If $\\cap_{j=1}^\\infty \\mathcal F_{t-j} = \\{\\phi, \\infty\\}$, every weakly stationary $X_t$ is MA($\\infty$)\n", " \\begin{align*}\n", " &X_t = \\mu + \\sum^\\infty_{j=0} \\psi_j\\epsilon_{t-j},\\\\\n", " &\\psi_0 = 1,\\quad\\sum_{j=0}^\\infty \\psi_j^2 < \\infty.\n", " \\end{align*}\n", "* AR Processes\n", " * ACF\n", " * Bartlett's Formula\n", " * Ljung-Box Test\n", " \\begin{align*}\n", " &H_0: \\rho(1)=\\rho(2)=\\cdots=\\rho(m)=0\\\\\n", " &H_1: \\text{at least one of $\\rho(i)$ is nonzero, $1\\le i\\le m$}\n", " \\end{align*}\n", " * AR(1)\n", " \\begin{align*}\n", " X_t = \\phi_0 + \\phi_1 X_{t-1} + \\epsilon_t\n", " \\end{align*}\n", " * stationary iff $|\\phi_1| < 1$\n", " * $E(X_t) = \\phi_0/(1-\\phi_1), \\quad |\\phi_1| < 1$\n", " * $Var(X_t) = \\phi_{\\epsilon}^2/(1-\\phi_1^2), \\quad |\\phi_1| < 1$\n", " * $\\gamma(h) = \\phi_1^{|h|}\\frac{\\sigma_{\\epsilon}^2}{1-\\phi_1^2}, \\quad |\\phi_1| < 1$\n", " * $\\rho(h) = \\phi_1^{|h|}, \\quad |\\phi_1| < 1$\n", " * if either $E(X_0)$ or $Var(X_0)$ differ from the stationary values but $|\\phi_1| < 1$ then the process is only asymptotically stationary\n", " * remove the mean: define $\\mu = \\phi_0/(1-\\phi_1), Y_t = X_t - \\mu$\n", " * Causal Processes\n", " * AR(2)\n", " \\begin{align*}\n", " X_t &= \\phi_0 + \\phi_1 X_{t-1} + \\phi_2 X_{t-2} + \\epsilon_t, \\\\\n", " \\mu &= \\frac{\\phi_0}{1 - \\phi_1 - \\phi_2}\\\\\n", " Y_t &= X_t - \\mu\n", " \\end{align*}\n", " * assume moment structure is constant\n", " * ACF: Multiply $Y_{t+h} = \\phi_1Y_{t+h-1} + \\phi_2Y_{t+h-2} + \\epsilon_{t+h}$ by $Y_t$, take expectation, and divide by $\\gamma(0)$: \n", " \\begin{align*}\n", " \\rho(h) = \\begin{cases}\n", " 1 &\\mbox{ if } h=0\\\\\n", " \\phi_1/(1-\\phi_2) &\\mbox{ if } h=1\\\\\n", " \\phi_1\\rho(h-1) + \\phi_2\\rho(h-2) &\\mbox{ if } h\\ge 2\n", " \\end{cases}\n", " \\end{align*}\n", " * AR polynomial: Plug $z=1/\\lambda$ into the chf of the recurrence relation\n", " \\begin{align*}\n", " \\phi(z) = 1 - \\phi_1 z - \\phi_2 z^2\n", " \\end{align*}\n", " * $X_t$ is stationary iff all roots of $\\phi(z) = 0$ (the characteristic roots) have modulus strictly greater than 1\n", " * recurrence relation is $\\phi(B)\\rho(h) = 0$\n", " * matrix form: \n", " \\begin{align*}\n", " \\mathbf X_t &= (X_t, X_{t-1})^T, \\quad\\mathbf \\mu = (\\phi_0, 0)^T, \\quad\\mathbf\\epsilon_t = (\\epsilon_t, 0)^T, \\\\\n", " \\mathbf X_t &= \\mathbf \\mu + \\mathbf M \\mathbf X_{t-1} + \\mathbf\\epsilon_t, \\\\\n", " \\mathbf M &= \\begin{pmatrix}\n", " \\phi_1 & \\phi_2\\\\\n", " 1 & 0\n", " \\end{pmatrix}\n", " \\end{align*}\n", " * Weak Stationarity of AR$(p)$\n", " * assuming constant mean, \n", " \\begin{align*}\n", " X_t &= \\phi_0 + \\phi_1 X_{t-1} + \\cdots + \\phi_p X_{t-p} + \\epsilon_t, \\\\\n", " \\mu &= \\frac{\\phi_0}{1 - \\sum_{i=1}^p\\phi_i}, \\quad\\sum_{i=1}^p\\phi_i < 1\\\\\n", " Y_t &= X_t - \\mu\n", " \\end{align*}\n", " * AR polynomial\n", " \\begin{align*}\n", " \\phi(z) = 1 - \\sum_{i=1}^p \\phi_i z^i\n", " \\end{align*}\n", " * matrix form\n", " \\begin{align*}\n", " \\mathbf M = \\begin{pmatrix}\n", " \\phi_1 & \\phi_2 & \\cdots & \\phi_{p-1} & \\phi_p\\\\\n", " 1 & 0 & \\cdots & 0 & 0 \\\\\n", " 0 & 1 & \\cdots & 0 & 0 \\\\\n", " \\vdots & \\vdots & \\vdots & \\vdots & \\vdots \\\\\n", " 0 & 0 & \\cdots & 1 & 0 \n", " \\end{pmatrix}\n", " \\end{align*}\n", " * Partial Correlation Coefficients $\\rho(X, Y|\\vec Z)$\n", " 1. regress $X$ on $\\vec Z$\n", " 2. regress $Y$ on $\\vec Z$\n", " 3. compute correlation coefficient of the residuals\n", " * PACF for AR$(p)$\n", " * estimate $\\hat \\phi_{k, k}$ in \n", " \\begin{align*}\n", " X_t = \\phi_{0, k} + \\phi_{1, k}X_{t-1} + \\cdots + \\phi_{k, k}X_{t-k} + \\epsilon_{k, t}\n", " \\end{align*}\n", " * $p+1$ is the smallest $k$ such that the test concludes $\\phi_{k, k} = 0$\n", "* MA Processes\n", " * MA$(q)$\n", " \\begin{align*}\n", " X_t = \\mu + \\sum_{i=1}^q \\theta_i\\epsilon_{t-i} + \\epsilon_t\n", " \\end{align*}\n", " * weakly stationary for all $\\{\\theta_i\\}$\n", " * $E(X_t) = \\mu$\n", " * $Var(X_t) = \\sigma_\\epsilon^2(1 + \\sum_{i=1}^q \\theta_i^2), \\quad\\forall t$\n", " \\begin{align*}\n", " \\gamma(h) &= \\begin{cases}\n", " \\sigma_\\epsilon^2 \\sum_{i=1}^{q-|h|} \\theta_i\\theta_{i+|h|} &\\mbox{ if }q\\le |h|\\\\\n", " 0 &\\mbox{ if }q>|h|\n", " \\end{cases},\\\\\n", " \\rho(h) &= \\gamma(h)/\\gamma(0)\n", " \\end{align*}\n", " * Invertibility of MA Processes\n", " * the two MA(1) processes have the same ACF: \n", " \\begin{align*}\n", " Y^{(1)}_t &= \\epsilon_t - \\theta_1 \\epsilon_{t-1}\\\\\n", " Y^{(2)}_t &= \\epsilon_t - \\frac{1}{\\theta_1} \\epsilon_{t-1}\n", " \\end{align*}\n", " * write residuals as an AR process\n", " \\begin{align*}\n", " \\epsilon_t &= Y_t^{(1)} + \\sum_{i=1}^\\infty \\theta_1^i Y_{t-i}^{1}\\\\\n", " \\epsilon_t &= Y_t^{(2)} + \\sum_{i=1}^\\infty \\frac{1}{\\theta_1^i} Y_{t-i}^{2}\n", " \\end{align*}\n", " * MA$(q)$ is invertible if the residuals can be represented by an AR process with convergent coefficients\n", " * MA polynomial \n", " \\begin{align*}\n", " \\theta(z) = 1 - \\theta_1 z - \\theta_2 z^2 - \\cdots - \\theta_q z^q\n", " \\end{align*}\n", " * An MA process is invertible iff all roots of $\\theta(z) = 0$ have modulus great than 1\n", " * Formal Notations\n", " * ARMA$(p, q): Y_t - \\phi_1 Y_{t-1} - \\cdots - \\phi_pY_{t-p} = \\epsilon_t + \\theta_1\\epsilon_{t-1} + \\cdots + \\theta_q \\epsilon_{t-q}$\n", " \\begin{align*}\n", " \\phi(B)Y_t = \\theta(B)\\epsilon_t\n", " \\end{align*}\n", " * ARIMA$(p, d, q)$\n", " \\begin{align*}\n", " \\phi(B)(1-B)^dY_t = \\theta(B)\\epsilon_t\n", " \\end{align*}\n", "* ARMA Models\n", " * AMRA Models\n", " \\begin{align*}\n", " &X_t - \\phi_0 - \\phi_1 X_{t-1} - \\cdots - \\phi_pX_{t-p} = \\epsilon_t + \\theta_1\\epsilon_{t-1} + \\cdots + \\theta_q \\epsilon_{t-q}\\\\\n", " &E(X_t) = \\mu = \\frac{\\phi_0}{1-(\\phi_1+\\cdots+\\phi_p)}\n", " \\end{align*}\n", " * ARMA(1, 1)\n", " \\begin{align*}\n", " X_t = \\phi_0 + \\phi_1X_{t-1} + \\theta_1\\epsilon_{t-1} + \\epsilon_t\n", " \\end{align*}\n", " * assuming stationary, \n", " \\begin{align*}\n", " E(X_t) &= \\phi_0/(1-\\phi_1)\\\\\n", " Var(X_t) &= \\gamma(0) = \\frac{(1 + \\theta_1^2 + 2\\phi_1\\theta_1)\\theta_\\epsilon^2}{1-\\phi_1^2}\\\\\n", " \\rho(1) &= \\frac{\\gamma(1)}{\\gamma(0)} = \\frac{(1 + \\phi_1\\theta_1)(\\phi_1 + \\theta_1)}{1 + \\theta_1^2 + 2\\phi_1\\theta_1}\\\\\n", " \\rho(h) &= \\phi_1^{h-1}\\rho(1), \\quad h\\ge 2\n", " \\end{align*}\n", " * ARMA$(p, q)$ Analysis\n", " * ARIMA, Differencing to Obtain Stationarity\n", " * $X_t$ is $\\mathcal I(k)$ if $\\nabla^{k-1}X_t$ is non-stationary but $\\nabla^{k}X_t$ is stationary, where $\\nabla = (1-B)$\n", " * $\\mathbf X_t$ is $\\mathcal I(k)$ if at least one of its coordinates is $\\mathcal I(k)$ and all the others are $\\mathcal I(j)$ for some $j\\le k$\n", " * Dickey-Fuller Test\n", " \\begin{align*}\n", " H_0 &: \\text{a unit root is present}\\\\\n", " H_1 &: \\text{no unit root}\n", " \\end{align*}\n", " * Parameter Estimation: OLS for AR$(p)$\n", " \\begin{align*}\n", " Y_t = \\phi_1Y_{t-1} + \\phi_2Y_{t-2} + \\cdots + \\phi_pY_{t-p} + \\epsilon_t\n", " \\end{align*}\n", " * assuming the errors are white noise, the least square estimate $\\hat \\phi$ is asymptotically normal \n", " \\begin{align*}\n", " &\\sqrt{n}(\\hat\\phi - \\phi) \\implies N_p(\\mathbf 0, \\sigma_\\epsilon^2\\mathbf \\Gamma_p^{-1}), \\\\\n", " &\\mathbf\\Gamma_p = E(\\mathbf Y^T\\mathbf Y), \\\\\n", " &\\mathbf Y = (Y_1, Y_2, \\ldots, Y_p)\n", " \\end{align*}\n", " * the $(i, j)$ element of the matrix is $E(Y_iY_j) = \\gamma(i-j)$\n", " * Yule Walker Equations\n", " * $\\rho(k) = \\phi_1\\rho(k-1) + \\phi_2\\rho(k-2) + \\cdots + \\phi_p\\rho(k-p), \\quad\\forall 1\\le k\\le p$\n", " * solve the $p\\times p$ linear system to obtain an estimate $\\hat \\phi$: \n", " \\begin{align*}\n", " \\mathbf \\rho &= \\mathbf R\\mathbf \\phi, \\\\\n", " \\mathbf \\rho &= (\\rho(1), \\rho(2), \\ldots, \\rho(p))^T, \\\\\n", " \\mathbf R_{i, j} &= \\rho(i-j) \n", " \\end{align*}\n", " * can be used as the initial guess for numerical root finding in MLE \n", " * Likelihood Methods\n", " * statsmodels\n", " * Forecasting\n", "* Non-Stationary to Stationary\n", " * Box-Cox Transformation\n", " * Box-Cox Transformation\n", " \\begin{align*}\n", " X^{(\\lambda)} = \\begin{cases}\n", " (X^\\lambda - 1)/\\lambda &\\mbox{ if }\\lambda\\ne 0\\\\\n", " \\log(X) &\\mbox{ if } \\lambda = 0\n", " \\end{cases}\n", " \\end{align*}\n", " * Box-Cox only fixes the variance, not the mean, for example larger variance for higher values\n", " * Trend and Seasonal Components\n", " * $X_t = m_t + Y_t$\n", " * linear trend: $\\mu_t = \\beta_0 + \\beta_1 t$\n", " * quadratic trend: $\\mu_t = \\beta_0 + \\beta_1 t + \\beta_2 t^2$\n", " * moving average smoother\n", " \\begin{align*}\n", " \\hat m_t &= \\frac{1}{2q+1}\\sum_{j=-q}^q X_{t+j}\\\\\n", " &= \\frac{1}{2q+1}\\sum_{j=-q}^q m_{t+j} + \\frac{1}{2q+1}\\sum_{j=-q}^q Y_{t+j}\\\\\n", " &\\approx m_t + \\text{small error}\n", " \\end{align*}\n", " * seasonal component with period $d$\n", " \\begin{align*}\n", " \\hat X_t &= \\beta_0 + \\beta_1 t + \\sum_{j=2}^d\\beta_j l_j(t)\\\\\n", " l_j(t) &= \\begin{cases}\n", " 1 &\\mbox{if $t$ mod $d$ is $j$}\\\\\n", " 0 &\\mbox{otherwise}\n", " \\end{cases}\\quad \\forall 1\\le j\\le d\n", " \\end{align*}\n", " * there are January indicator function, February indicator function, and so on\n", " * one of the indicators is omitted as the sum of all indicators must be 0\n", " * Differencing \n", " * $\\nabla = (1-B)$ can remove polynomial trends; for example $\\nabla^2$ can remove quadratic trends\n", " * $\\nabla_d = (1-B^d)$ can remove seasonal trend: if $X_t = \\beta_0 + \\beta_1 t + s_t + \\epsilon_t$ where $s_t$ is the seasonal term such that $s_t = s_{t-d}$, then $\\nabla_d X_t$ is weakly stationary\n", " * $\\nabla_d \\ne \\nabla^d = (1-B)^d$\n", "* ARCH/GARCH Modeling\n", " * Motivation\n", " * ARIMA has non-constant $E(X_t|\\mathcal F_{t-1})$ but constant $Var(X_t|\\mathcal F_{t-1})$, GARCH is the opposite\n", " * deterministic models: $Var(X_t|\\mathcal F_{t-1})$ is deterministic\n", " * stochastic volatility models: $Var(X_t|\\mathcal F_{t-1})$ is a stochastic process\n", " * GARCH by itself does not explain the JPM GS situation\n", " * ARCH(1)\n", " \\begin{align*}\n", " a_t &= \\sigma_t\\epsilon_t\\\\\n", " \\sigma_t &= \\sqrt{\\omega + \\alpha a_{t-1}^2}, \\quad\\omega > 0, 0\\le \\alpha < 1\n", " \\end{align*}\n", " * $\\epsilon_t$ is iid with mean 0 and variance 1\n", " * $E(X_t|\\mathcal F_{t-1}) = 0$\n", " * $Var(X_t|\\mathcal F_{t-1}) = \\sigma_t^2 = \\omega + \\alpha a_{t-1}^2$\n", " * assuming weak stationarity, ARCH(1) is a white noise: $\\gamma_a(0) = E(\\sigma_t^2) = E(\\omega + \\alpha a_{t-1}^2) = \\omega + \\alpha\\gamma_a(0)$, so \n", " \\begin{align*}\n", " \\gamma_a(0) &= \\frac{\\omega}{1-\\alpha}\\\\\n", " \\gamma_a(h) &= 0\n", " \\end{align*}\n", " * $\\alpha$ controls the mean reversion of $\\sigma^2_t$\n", " * AR(1)/ARCH(1)\n", " \\begin{align*}\n", " X_t = \\mu + \\beta(X_{t-1} - \\mu) + a_t, \\quad |\\beta| < 1\n", " \\end{align*}\n", " * $\\rho_X(h) = \\beta^{|h|}, \\rho_{a^2} = \\alpha^{|h|}$\n", " * non-constant conditional mean and variance\n", " * ARCH(p)\n", " * ARCH(1) Properties\n", " * $a_t^2$ is an AR(1) if $E(\\epsilon_t^4) < \\infty$: \n", " \\begin{align*}\n", " a_t^2 = \\omega + \\alpha a_{t-1}^2 + \\sigma_t^2(\\epsilon_t^2 - 1), \n", " \\end{align*}\n", " * $\\nu_t = \\sigma_t^2(\\epsilon_t^2 - 1)$ can be shown to be a white noise\n", " * when $\\epsilon_t$ is iid $N(0, 1)$, the unconditional kurtosis > 3: Following AR(1) properties, we have \n", " \\begin{align*}\n", " E(a_t^2) &= \\frac{\\omega}{1-\\alpha}, \\\\\n", " Var(a_t^2) &= \\frac{2E(\\sigma_t^4)}{1-\\alpha^2}, \\\\\n", " E(\\sigma_t^4) &= E((\\omega + \\alpha a_{t-1}^2)^2) \\\\\n", " &= \\frac{\\omega^2(1+\\alpha)}{(1-3\\alpha^2)(1-\\alpha)}\\\\\n", " &= 3(E(a_t^2))^2\\frac{1-\\alpha^2}{1-3\\alpha^2} > 3(E(a_t^2))^2\n", " \\end{align*}\n", " * ARCH Effect: $a_t^2$ and $a_{t+h}^2$ are positively correlated\n", " * ARCH and Stylized Facts\n", " * ARCH does not support asymmetry or the leverage effect\n", " * Weaknesses of ARCH Model\n", " * From ARCH to GARCH\n", " \\begin{align*}\n", " a_t &= \\sigma_t\\epsilon_t, \\\\\n", " \\sigma^2_t &= \\omega + \\sum_{i=1}^p \\alpha_i a_{t-i}^2 + \\sum_{j=1}^q \\beta_j\\sigma_{t-j}^2, \\quad\\omega \\ge 0, \\alpha_i \\ge 0, \\beta_j > 0\n", " \\end{align*}\n", " * $\\epsilon_t$ is iid $N(0, 1)$\n", " * GARCH(1, 1) squared is ARMA(1, 1)\n", " \\begin{align*}\n", " a_t^2 - c &= (\\alpha + \\beta)(a_{t-1}^2 - c) - \\beta\\eta_{t-1} + \\eta_t, \n", " \\end{align*}\n", " * $c = \\omega/(1-\\alpha-\\beta), \\eta_t = a_t^2 - \\sigma_t^2$\n", " * ARMA(1, 1) with mean $c$ and coefficients $\\phi_1 = \\alpha + \\beta, \\theta_1 = -\\beta$\n", " * Fitting ARCH to S&P 500 Data\n", " * GARCH$(p, q)$ squared is ARMA$(p, q)$\n", " \\begin{align*}\n", " a_t^2 - c &= \\sum_{i=1}^{\\max(p, q)}(\\alpha_i + \\beta_i)(a_{t-i}^2 - c) - \\sum_{i=1}^{\\max(p, q)}\\beta_i\\eta_{t-i} + \\eta_t, \n", " \\end{align*}\n", " * $c = \\omega/(1-\\sum_{j=1}^{\\max(p, q)}(\\alpha_i + \\beta_i)), \\eta_t = a_t^2 - \\sigma_t^2$\n", " * given $\\alpha_i > 0, \\beta_i \\ge 0$, $a_t^2$ is weakly stationary if $\\sum_{i=1}^p\\alpha_i + \\sum_{j=1}^q\\beta_j < 1$\n", " * GARCH Forecasting\n", " * 1-step ahead forecast of the conditional variance $\\sigma_{t+1}^2$ is already given by the model\n", " * for GARCH(1, 1), let $\\lambda = \\alpha + \\beta < 1$, the $k$-step ahead forecast is \n", " \\begin{align*}\n", " \\hat \\sigma_{t+k}^2 &= \\omega + \\lambda \\hat \\sigma_{t+k-1}^2\\\\\n", " &= \\omega(1 + \\lambda + \\cdots + \\lambda^{k-2}) + \\lambda^{k-1} \\hat \\sigma_{t+1}^2 \\\\\n", " &\\rightarrow \\frac{\\omega}{1-\\lambda}\\quad \\text{ as }k\\rightarrow \\infty\n", " \\end{align*}\n", " * half-life of the volatility difference is approximately $\\lambda^T = 1/2$, so $T\\approx -\\frac{\\log 2}{\\log\\lambda}$\n", " * Engle Test for ARCH Effects\n", " * GARCH Forecasting Example in Risk Management\n", " * Other Volatility Models\n", " * GARCHM\n", " \\begin{align*}\n", " X_t &= \\mu + c\\sigma_t^2 + a_t\\\\\n", " a_t &= \\epsilon_t\\sigma_t\\\\\n", " \\sigma_t^2 &= \\omega + \\alpha a_{t-1}^2 + \\beta \\sigma_{t-1}^2\n", " \\end{align*}\n", " * EGARCH\n", " \\begin{align*}\n", " g(\\epsilon_t) &= \\theta\\epsilon_t + \\gamma(|\\epsilon_t| - E(|\\epsilon_t|))\\\\\n", " &= \\begin{cases}\n", " (\\theta + \\gamma)\\epsilon_t - \\gamma(|\\epsilon_t|) \\mbox{ if } \\epsilon_t\\ge 0\\\\\n", " (\\theta - \\gamma)\\epsilon_t - \\gamma(|\\epsilon_t|) \\mbox{ if } \\epsilon_t < 0\n", " \\end{cases},\\\\\n", " a_t &= \\sigma_t\\epsilon_t\\\\\n", " \\log(\\sigma_t^2) &= \\omega + \\sum_{i=1}^p\\beta_i \\log(\\sigma_{t-i}^2) + \\sum_{j=1}^q g_j(\\epsilon_{t-j})\n", " \\end{align*} \n", "* Multivariate Time Series\n", " * Multivariate Time Series\n", " * weak stationary: mean vector and autocovariance function (now a matrix) are independent of $t$\n", " \\begin{align*}\n", " \\mathbf X_t &= (X_{1,t}, X_{2,t}, \\ldots, X_{m,t})\\\\\n", " \\mathbf \\Gamma(t+h, t) &= E((\\mathbf X_{t+h}-\\mathbf \\mu_{t+h})(\\mathbf X_{t}-\\mathbf \\mu_{t})^T)\\\\\n", " \\rho_{i, j}(h) &= \\frac{\\gamma_{i, j}(h)}{\\sqrt{\\gamma_{i, i}(0)\\gamma_{j, j}(0)}}\n", " \\end{align*}\n", " * the diagonal elements are the ACovF of the individual component time series\n", " * white noise: weak stationary + zero mean + zero ACF $\\forall h\\ne 0$\n", " * $\\rho_{i, j}(h) = \\rho(X_{i,(t+h)}, X_{j, t}) = \\rho_{i, j}(-h)$\n", " * the sample mean of a weakly stationary process converges and is asymptotically normal\n", " * Vector Autoregressioive Processes \n", " \\begin{align*}\n", " \\mathbf X_t = \\mathbf a_0 + \\sum_{i=1}^p\\mathbf A_i\\mathbf X_{t-i} + \\epsilon_t\n", " \\end{align*}\n", " * stationarity condition: roots of \n", " \\begin{align*}\n", " \\det\\left(I - \\sum_{i=1}^p \\mathbf A_i x^i\\right) = 0\n", " \\end{align*}\n", " have modulus strictly larger than 1\n", "* Cointegration\n", " * Cointegration \n", " * the components of a multivariate time series $X_t$ is CI$(d, b)$ if \n", " 1. all components are $\\mathcal I(d)$\n", " 2. there exists a nonzero $\\vec\\alpha$ (the cointegrating vector) such that $\\vec\\alpha X_t$ is $\\mathcal I(d-b)$ with $b>0$\n", " * two cointegrated time series $X_t, Y_t$ can have small correlation: \n", " \\begin{align*}\n", " W_t &= W_{t-1} + \\epsilon_t\\\\\n", " X_t &= W_t + \\epsilon_{X, t}\\\\\n", " Y_t &= W_t + \\epsilon_{Y, t}\n", " \\end{align*}\n", " * both $\\mathcal I(1)$ but $X_t - Y_t$ is stationary\n", " \\begin{align*}\n", " Cov(X_t, Y_t) = \\frac{t\\sigma^2}{\\sqrt{(t\\sigma^2 + \\sigma_X^2)(t\\sigma^2 + \\sigma_Y^2)}}\n", " \\end{align*}\n", " * if $m=2$, $\\vec\\alpha$ is unique up to scale\n", " * cointegration does not imply high correlation: \n", " \\begin{align*}\n", " X_t &= X_{t-1} + \\epsilon_{X, t}\\\\\n", " Y_t &= Y_{t-1} + \\epsilon_{Y, t}\\\\\n", " Z_t &= X_t + Y_t\n", " \\end{align*}\n", " * $X_t$ and $Z_t$ are not cointegrated but $\\rho_{X, Z} = 1/\\sqrt{1+\\sigma_Y^2/\\sigma_X^2}$ which will be large if $\\sigma_Y/\\sigma_X$ is small\n", " * Johansen Test\n", " * difference the time series until it's $\\mathcal I(1)$\n", " * $\\mathbf X_t$ is VAR$(p)$\n", " \\begin{align*}\n", " \\nabla\\mathbf X_t &= \\mathbf a + (\\mathbf A_1 - I) \\nabla\\mathbf X_{t-1} + (\\mathbf A_1 + \\mathbf A_2 - I)\\mathbf X_{t-2} + \\sum_{i=3}^p\\mathbf A_i\\mathbf X_{t-i} + \\mathbf \\epsilon_t = \\cdots\\\\\n", " \\mathbf B_i &= (\\mathbf A_1 + \\cdots + \\mathbf A_i -I)\n", " \\end{align*}\n", " * $\\mathbf B_i\\mathbf X_{t-i}$ is stationary iff the rows of $B_i$ are cointegrating vectors or 0\n", " * $\\mathbf B$ can not be full rank or otherwise taking inverse we find $X_{t-i}$ to be stationary\n", " * if $rank(\\mathbf B) = 0$, no cointegrating vector\n", " \\begin{align*}\n", " H_0 &: rank(B)=0\\\\\n", " H_1 &: rank(B)>0\n", " \\end{align*}\n", " * Cryptocurrency Example\n", "* State Space Modeling\n", " * State Space Models\n", " \\begin{align*}\n", " \\mathbf X_{t+1} &= \\mathbf F_t \\mathbf X_t + \\mathbf V_t\\\\\n", " \\mathbf Y_t &= \\mathbf G_t \\mathbf X_t + \\mathbf W_t\n", " \\end{align*}\n", " * $\\mathbf V_t$ and $\\mathbf W_t$ are uncorrelated WN\n", " * Kalman Recursions: Kalman Prediction & Filtering\n", " * Prediction: Estimate $\\mathbf X_{t+1}$ or $\\mathbf X_{t+k}$ using $\\mathbf Y_0, \\mathbf Y_1, \\ldots, \\mathbf Y_t$; denoted $\\hat{\\mathbf X}_{t+k|t}$\n", " * Filtering: Estimate $\\mathbf X_t$ using $\\mathbf Y_0, \\mathbf Y_1, \\ldots, \\mathbf Y_t$; denoted $\\hat{\\mathbf X}_{t|t}$\n", " * Smoothing: Estimate $\\{\\mathbf X_t\\}_{t=1}^{T-1}$ using $\\mathbf Y_0, \\mathbf Y_1, \\ldots, \\mathbf Y_T$; denoted $\\hat{\\mathbf X}_{t|T}$\n", " * Example (Linear Regression)\n", " * AR, MA in State Space Form\n", " * AR(2) with zero mean: $X_t = \\phi_1X_{t-1} + \\phi_2X_{t-2} + \\epsilon_t$\n", " \\begin{align*}\n", " &\\mathbf X_t = (X_t, X_{t-1})^T, \\quad\\mathbf \\epsilon_t = (\\epsilon_t, 0)^T, \\\\\n", " &\\mathbf X_t = \\mathbf F\\mathbf X_{t-1} + \\epsilon_t, \\\\\n", " &Y_t = (1, 0)\\mathbf X_t, \\\\\n", " &\\mathbf F = \\begin{pmatrix}\n", " \\phi_1 & \\phi_2\\\\\n", " 1 & 0\n", " \\end{pmatrix}\n", " \\end{align*}\n", " * MA(2) with zero mean: $X_t = \\theta_1\\epsilon_{t-1} + \\theta_2\\epsilon_{t-2} + \\epsilon_t$\n", " \\begin{align*}\n", " &\\mathbf X_t = (\\epsilon_{t}, \\epsilon_{t-1}, \\epsilon_{t-2})^T, \\quad\\mathbf \\epsilon_t = (\\epsilon_t, 0, 0)^T, \\\\\n", " &\\mathbf X_t = \\mathbf F\\mathbf X_{t-1} + \\epsilon_t, \\\\\n", " &Y_t = (1, \\theta_1, \\theta_2)\\mathbf X_t, \\\\\n", " &\\mathbf F = \\begin{pmatrix}\n", " 0 & 0 & 0\\\\\n", " 1 & 0 & 0\\\\\n", " 0 & 1 & 0\n", " \\end{pmatrix}\n", " \\end{align*}\n", " * Bayesian Background to Kalman Methods\n", " * Stochastic Volatility " ] } ], "metadata": { "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.7.10" } }, "nbformat": 4, "nbformat_minor": 5 }