UA MATH566 統計理論 概念與定理總結

Part 1 Exponential Family
Tip 1: Form of Exponential Family
$$f(x|\eta )=h(x)\exp \left(\sum _{i=1}^n{\eta }_i{T}_i(x)?A(\eta )\right)$$

Tip 2: $\{T_i(X)\}$ are complete minimal sufficient statistics of Exponential family
Tip 3:
$$E{T}_i(X)=\frac{?A(\eta )}{?{\eta }_i},Var(T(X))=A^{\prime \prime }(\eta )$$

Tip 4:
$$I(\eta )=Var(T(X)),I(\theta )=I(\eta (\theta ))(\frac{?\eta }{?\theta }{)}^2$$

Part 2 Sufficient Statistics and Complete Statistics
Tip 1: Neyman-Fisher Theorem, $T(X)$ is sufficient $?$ $f(x,\theta )=h(x)g(T(x),\theta )$
Tip 2: $T(X)$ is minimal sufficient statistics $?$ $\frac{L(\theta |\text{X})}{L(\theta |\text{Y})}$ independent of $\theta$ holds iff $T(X)=T(Y)$
Tip 3: $f(x,\theta )$ is complete iff $E[g(X)]=0?g(x)=0a.s.$. If $f(x,\theta )$ is complete, any statistics from $f(x,\theta )$ are complete. A statistics $T(X)$ is complete iff $E[h(T)]=0?h(t)=0a.s.$.
Tip 4: Basu’s Theorem, complete minimal sufficient statistics are independent of ancillary statistics.

Part 3 C-R Inequality and Fisher Information
Tip 1: C-R inequality, $\hat{g}(X)$ and $\hat{\theta }(X)$are unbiased estimator of $g(\theta )$ and $\theta$ where $g(\theta )$ is differentiable, and then
$$Var(\hat{\theta })\ge I^{?1}(\theta ),Var(\hat{g}(X))\ge [g^{\prime }(\theta ){]}^2{I}^{?1}(\theta )$$

Tip 2: C-R inequality under multivariate case, $\hat{g}(X)$ and $\hat{\theta }(X)$ are unbiased estimator of $g(\theta )$ and $\theta$的, where $g(\theta )$ is differentiable and its Jacobian determinant is $Dg(\theta )$, and then
$$Var(\hat{g}(X))\ge Dg(\theta )I^{?1}(\theta )[Dg(\theta ){]}^T$$

Tip 3: Properties of Fisher information

$I_n(\theta )=E{\left[\frac{?\log f(x,\theta )}{?\theta }\right]}^2=E\left[\frac{{?}^2\log f(f,\theta )}{?{\theta }^2}\right]$

$I_n(\theta )=n{I}_1(\theta )$

For arbitrary statistic $T(X)$, $0\le I_T(X)\le I_X(\theta )$

Tip 4: Fisher information of transformation of random variable, if want to transform the parameter $\theta$ to $\xi$,
$$I(\xi )=[D_{\xi }\theta (\xi ){]}^{T}I(\theta )D_{\xi }\theta (\xi )$$

Part 4 Point Estimation
Tip 1: Moment estimation equation
$$\overline{X^m}=\frac{1}{n}\sum _{i=1}^n{X}_i\approx {\mu }_m(\theta )=E_{\theta }[X^m],m=1,2,?,d$$

By forcing sample moments and theoretical moments to be equal and solving for $\theta$, we get moment estimators which are naturally functions of sample moments. So always remember Delta method to estimate their mean and variance.

Tip 2: Maximum likelihood estimation
$$\hat{\theta }=\underset{\theta \in \Theta }{\mathrm{argmax}}\ln f(x_1,x_2,?,x_n,\theta )$$

If $g$ is bijection, $?=g(\theta )$, then $g(\hat{\theta })$ is MLE of $?$

$\sqrt{n}(\hat{\theta }?\theta ){\to }_{d}N(0,1/I(\hat{\theta }))$ holds under some assumption

Tip 3: Lehmann-Scheffe Theorem

$T(X)$ is a sufficient statistic, for arbitrary unbiased estimator $\delta (X)$, $E[\delta (X)|T(X)]$ is UMVUE; if $T(X)$ is also complete, then $E[\delta (X)|T(X)]$ is unique;

If $g(\hat{\theta })$is an unbiased estimator of $g(\theta )$and a function of sufficient statistic $T(X)$, then $g(\hat{\theta })$ is a UMVUE of $g(\theta )$; if $T(X)$ is also complete, then $g(\hat{\theta })$ is unique;

UMVUE must be a function of sufficient statistics.

Part 5 Hypothesis Testing
Tip 1: Likelihood ratio test, define likelihood ratio
$$\lambda (X)=\frac{{\sup }_{\{\theta ={\theta }_0\}}L(\theta |X)}{{\sup }_{\{\theta ={\theta }_0,{\theta }_1\}}L(\theta |X)}$$

and construct reject region with this statistic
$$C^{?}=\{X:\lambda (X)\le c_{\alpha }\}$$

where $c_{\alpha }$ satisfied $P(\lambda (X)\le c_{\alpha })=\alpha$.

Tip 2: Neyman-Pearson Lemma, likelihood ratio test is UMP for two point test.

Tip 3: Karlin-Rubin theorem, if likelihood ratio is monoton on some statistic, likelihood ratio test is UMP for arbitrary test.

Part 6 Confidential Interval
Tip 1: $\gamma$-level confidential interval, $$P\{\theta \in \hat{C}(X)\}=\gamma$$,

Double-sided: $\hat{C}(X)=\{\theta :{\hat{\theta }}_l\le \theta \le {\hat{\theta }}_u\}$

with upper confidential bound: $\hat{C}(X)=\{\theta :?\infty \le \theta \le {\hat{\theta }}_u\}$

with lower confidential bound: $\hat{C}(X)=\{\theta :{\hat{\theta }}_l\le \theta \le \infty \}$

Tip 2: Pivot method, if $Q(X,\theta )$ is a pivot, solve this equation to get a confidential interval
$$P(l\le Q(X,\theta )\le u)=\gamma$$

Tip 3: Optimal confidential interval
$$\begin{gathered} \min E[{\hat{\theta }}_u?{\hat{\theta }}_l] \\ s.t.P[{\hat{\theta }}_l\le \theta \le {\hat{\theta }}_u]\ge \gamma \end{gathered}$$

Sometimes it’s hard to calculate the difference of two quantiles, but if the quantile is from chi-sq distribution, we can use normal approximation$${\chi }_n^2\approx n(1+\sqrt{2/n})Z=n+\sqrt{2n}Z$$

For unimodal density, optimal confidential interval must be constructed by symmetric quantiles.

Tip 4: For $\Theta ?\mathbb{R}$, $\gamma$-level confidential interval is exactly the accepted region of an $1?\gamma$ hypothesis testing

總結

以上是生活随笔為你收集整理的UA MATH566 统计理论 概念与定理总结的全部內容，希望文章能夠幫你解決所遇到的問題。

如果覺得生活随笔網站內容還不錯，歡迎將生活随笔推薦給好友。