Chapter 3.4 O.N. vectors and the Gram-Schmidt Process

, and

then

*A e*_{j} : the j-th column of A

(*e*_{i}^{T} *A* : the i-th row of A )

[`Def`]
,
is called isometric

( )

i.e. the column vectors are O.N.

, then where

and is isometric .

<**Proof**>
*A* = *QR* , *Q*
,
where
,
*R* =
=
=

Gram-Schmidt

: O.N.vectors

**Exercise : ** P168 Ex3.4.1 Ex3.4.3
Gram-Schmidt process
is a basis

orthonormal basis

such procsee is called Gram-Schmidt orthonomalization .

In general

,

Let

Let
*r*_{jk} = < *v*_{k} , *g*_{j} > , *j* < *k*

=

*v*_{i} are the column vextors of V

1. Subspace : is subspace of

if and

2. Linear Independent 3. Span =

If =

=

subspace of

is the orthogonal complement of .

Note : 1. is also a subspace of . 2. are the O.N. vectors .

[`Thm`]
subspace of
,
then

( i.e. if , then ,where

moreever , )

<**Proof**>

= =

and =

= =

N(A) = null space of

R(A) = range of
[`Thm`]
then
< *nx* , *y* > = < *x* , *A*^{T} *y* >

*y*^{T} *A x* = (*A*^{T} *y*)^{T} *x*

[`Thm`]

[`Thm`]

[`Thm`] If A is symmetric ,

If ( i.e. )

Range : R(A) =

Null space : N(A) =

<**Proof of **
>

If for all x

In particular

i.e.

If

i.e.

[*Lemma*]

[`Thm`]

<**Proof**> Because

Note : nullity (T) + range (T) = dim S

( If T : S S linear transformation )

**Exercise :** 3.5.7

( because column space A = row space of *A*^{T} )

The Discrete Least-Square Problem

If

[`Thm`] ( Pythagorean Thm. )

if
( *u* ,*v* ) = 0

<**Proof**>
=
< *u* , *u* > + < *u* , *v* > + < *v* , *u* > + < *v* , *v* >
=

Note : The Least-Square Error

[`Thm`] If
,
then

such that

where y is the unique element in

such that

( i.e. y is the orthogonal projection of b into )

[*Cor*] Let
.
Then
=

<**Proof**>

*b* = *y* + *z* , where

*b* = *y* + *z* uniquelly , with
and

If s.t.

consider
*b* - *s* = ( *b* - *y* ) + ( *y* - *s* ) ,
where

=

is

i.e.

*A*^{T} ( *b* - *Ax* ) = 0

*A*^{T} *A x* = *A*^{T} *b* ( normal equation )

Note :

Note : 1. Solving L.S. Problems *Ax* = *b* ,
is the same to solving normal equation
*A*^{T} *A x* = *A*^{T} *b* .
2. *A*^{T} *A*
i. symmetry .
ii. nonneqative defined ( Positive semidefined
).
3. If A is full rank ( i.e. rank A = m ) ,
then *A*^{T} *A* is positive define
Chelosky Decomposition ( on *A*^{T} *A* )
to solve the
( *A*^{T} *A* ) *x* = *A*^{T} *b*

The way of solving Least Square Solution :

1. QR 2. Cholesky 3, A b

1.
(*A*^{T} *A*)^{T} = *A*^{T} *A*^{TT} = *A*^{T} *A*2. nonnegative defined

3. <**Proof**> A is full rank ,
*A*^{T} *A* is nonsingular . If
for all

The Continuous Least Square Problem

Discrete

Continuous

Inner Product :

2-norm

If are basis of .

,

,

( m equations )

,

,

,

*cx* = *d*

*c* =

Note : c is 1. Symmetry . 2. Positive defined .