A ej : the j-th column of A
(eiT A : the i-th row of A )
[Def] , is called isometric
i.e. the column vectors are O.N.
, then where
and is isometric .
<Proof> A = QR , Q , where , R = = =
Exercise : P168 Ex3.4.1 Ex3.4.3 Gram-Schmidt process is a basis
such procsee is called Gram-Schmidt orthonomalization .
Let rjk = < vk , gj > , j < k
vi are the column vextors of V
1. Subspace : is subspace of
2. Linear Independent 3. Span =
is the orthogonal complement of .
Note : 1. is also a subspace of . 2. are the O.N. vectors .
[Thm] subspace of , then
( i.e. if , then ,where
moreever , )
N(A) = null space of
R(A) = range of [Thm] then < nx , y > = < x , AT y >
yT A x = (AT y)T x
[Thm] If A is symmetric ,
If ( i.e. )
Range : R(A) =
Null space : N(A) =
<Proof of >
If for all x
Note : nullity (T) + range (T) = dim S
( If T : S S linear transformation )
Exercise : 3.5.7
( because column space A = row space of AT )
The Discrete Least-Square Problem
[Thm] ( Pythagorean Thm. )
if ( u ,v ) = 0
<Proof> = < u , u > + < u , v > + < v , u > + < v , v > =
Note : The Least-Square Error
[Thm] If , then
where y is the unique element in
( i.e. y is the orthogonal projection of b into )
[Cor] Let . Then =
b = y + z , where
b = y + z uniquelly , with and
consider b - s = ( b - y ) + ( y - s ) , where
AT ( b - Ax ) = 0
AT A x = AT b ( normal equation )
Note : 1. Solving L.S. Problems Ax = b , is the same to solving normal equation AT A x = AT b . 2. AT A i. symmetry . ii. nonneqative defined ( Positive semidefined ). 3. If A is full rank ( i.e. rank A = m ) , then AT A is positive define Chelosky Decomposition ( on AT A ) to solve the ( AT A ) x = AT b
The way of solving Least Square Solution :
1. QR 2. Cholesky 3, A b
1. (AT A)T = AT ATT = AT A2. nonnegative defined
3. <Proof> A is full rank , AT A is nonsingular . If for all
The Continuous Least Square Problem
Inner Product :
If are basis of .
( m equations )
cx = d
Note : c is 1. Symmetry . 2. Positive defined .