Conditional Expectation and Prediction

經常會產生一種情形, 即在觀測一隨機變數 X 之值後, 我們要根據此觀測值去預測另一隨機變數 Y 的值. 設 g(X) 表 預測量 (predictor), 也就是說, 當 X 的觀測值為 x 時, g(x) 為 Y 的預測值. 很明顯地, 我們希望選擇一函數g 使得 g(X) 和 Y 的值很接近. 而是否 "接近" 的一種可能的判定方式就是選取使得 E[(Y-g(X))2]為最小的函數 g. 下面我們將證明在此準則下, Y 的 最佳可能預測量 (best possible predictor) 為 g(X) = E[Y|X].

Proposition

$ E[(Y-g(X) )^2] \geq E[(Y-E[Y\vert X])^2]$
Proof:
$\begin{array}{rcl} E[(Y-g(X))^2\vert X] & = & E[(Y-E[Y\vert X] + E[Y\vert X] - ... ...X))^2[X] \\ \\ && + 2E[(Y-E[Y\vert X])(E[Y\vert X] - g(X))\vert X] \end{array}$
但,在給予 X 後, E[Y|X] - g(X) 為 X 的函數, 可看作是一常數. 因此
$\begin{array}{l} E[(Y-E[Y\vert X])(E[Y\vert X] - g(X))\vert X] = (E[Y\vert X] -... ...X][X]\\ \\ = (E[Y\vert X]-g(X))(E[Y\vert X] - E[Y\vert X]) \\ = 0 \end{array}$
因此, 由上面的式子得
$E[(Y-g(X))^2\vert X] \geq E[(Y-E[Y\vert X])^2\vert X]$
兩邊取期望值, 即得所要證的結果.          $\rule[0.02em]{1.0mm}{1.5mm}$

Example
Suppose that the son of a man of height x (in inches) attains a height that is normally distributed with mean x+1 and variance 4. What is the best prediction of the height at full growth of the son of a man who is 6 feet tall?
Solution:
Formally, this model can be written as Y=X+1+e where e is a normal random variable, independent of X, having mean 0 and variance 4. The X and Y, of course, represent the heights of the man and his son, respectively. The best prediction E[Y|X=72] is thus equal to
$\begin{array}{rcl} E[Y\vert X=72]&=&E[X+1+e\vert X=72] \\ \\ &=&73+E[e\vert X=... ...ox{ by independence } \\ \\ &=&73\qquad\rule[0.02em]{1.0mm}{1.5mm} \end{array}$