Chapter 3.4 O.N. vectors and the Gram-Schmidt Process

, and

then

A ej : the j-th column of A

(eiT A : the i-th row of A )

[Def] , is called isometric

( )

i.e. the column vectors are O.N.

, then where

and is isometric .

<Proof> A = QR , Q , where , R = = =

Gram-Schmidt

: O.N.vectors

Exercise : P168 Ex3.4.1 Ex3.4.3 Gram-Schmidt process is a basis

orthonormal basis

such procsee is called Gram-Schmidt orthonomalization .

In general

,

Let

Let rjk = < vk , gj > , j < k

=

vi are the column vextors of V

1. Subspace : is subspace of

if and

2. Linear Independent 3. Span =

If =

=

subspace of

is the orthogonal complement of .

Note : 1. is also a subspace of . 2. are the O.N. vectors .

[Thm] subspace of , then

( i.e. if , then ,where

moreever , )

<Proof>

= =

and =

= =

N(A) = null space of

R(A) = range of [Thm] then < nx , y > = < x , AT y >

yT A x = (AT y)T x

[Thm]

[Thm]

[Thm] If A is symmetric ,

If ( i.e. )

Range : R(A) =

Null space : N(A) =

<Proof of >

If for all x

In particular

i.e.

If

i.e.

[Lemma]

[Thm]

<Proof> Because

Note : nullity (T) + range (T) = dim S

( If T : S S linear transformation )

Exercise : 3.5.7

( because column space A = row space of AT )

The Discrete Least-Square Problem

If

[Thm] ( Pythagorean Thm. )

if ( u ,v ) = 0

<Proof> = < u , u > + < u , v > + < v , u > + < v , v > =

Note : The Least-Square Error

[Thm] If , then

such that

where y is the unique element in

such that

( i.e. y is the orthogonal projection of b into )

[Cor] Let . Then =

<Proof>

b = y + z , where

b = y + z uniquelly , with and

If s.t.

consider b - s = ( b - y ) + ( y - s ) , where

=

is

i.e.

AT ( b - Ax ) = 0

AT A x = AT b ( normal equation )

Note :

Note : 1. Solving L.S. Problems Ax = b , is the same to solving normal equation AT A x = AT b . 2. AT A i. symmetry . ii. nonneqative defined ( Positive semidefined ). 3. If A is full rank ( i.e. rank A = m ) , then AT A is positive define Chelosky Decomposition ( on AT A ) to solve the ( AT A ) x = AT b

The way of solving Least Square Solution :

1. QR 2. Cholesky 3, A b

1. (AT A)T = AT ATT = AT A2. nonnegative defined

3. <Proof> A is full rank , AT A is nonsingular . If for all

The Continuous Least Square Problem

Discrete

Continuous

Inner Product :

2-norm

If are basis of .

,

,

( m equations )

,

,

,

cx = d

c =

Note : c is 1. Symmetry . 2. Positive defined .