Linear Algebra II

Table of Contents
CHAPTER ONE ........................................................................................................................................... 1
THE CHARACTERISTIC EQUATION OF A MATRIX ........................................................................... 1
1.1 Eigenvalues and Eigenvectors ............................................................................................................ 1
1.2. The Characteristic Polynomial ........................................................................................................... 5
1.3. Similarity of Matrices and Characteristic Polynomial ..................................................................... 10
1.4. The Spectral radius of a matrix ........................................................................................................ 12
1.5. Diagonalization ................................................................................................................................ 13
1.6. Decomposable Matrices ................................................................................................................... 15
1.7. Minimal Polynomial and Cayley-Hamilton Theorem...................................................................... 17
CHAPTER TWO ........................................................................................................................................ 22
ORTHOGONALITY .................................................................................................................................. 22
2.1. The Inner Product ....................................................................................................................... 22
2.2. Inner Product space ..................................................................................................................... 22
2.3. Orthonormal Sets ........................................................................................................................ 25
2.4. The Gram-Schmidt Orthogonalization process ........................................................................... 26
2.5. Cauchy-Schwarz and Triangular Inequalities. ............................................................................ 28
2.6. The Dual Space ........................................................................................................................... 30
2.7. The Adjoint of Linear Operator .................................................................................................. 32
2.8. Self- adjoint Linear Operators..................................................................................................... 33
2.9. Isometry ...................................................................................................................................... 37
2.10. Normal Operators and the Spectral Theorem.......................................................................... 38
2.11. Factorization of a matrix / LU, Cholesky, QR/ ....................................................................... 39
2.11.1 LU- factorization ................................................................................................................. 40
2.11.2 Cholesky factorization ............................................................................................................ 42
2.11.2 QR- factorization................................................................................................................. 44
2.11.4 Singular Value Decomposition ........................................................................................... 48
CHAPTER THREE .................................................................................................................................... 51
CANONICAL FORMS............................................................................................................................... 51
3.1. Elementary row and column operation on Matrices ................................................................... 51

I
 Linear Algebra II

3.2. Equivalence of Matrices of polynomials..................................................................................... 51

3.3. The Smith Canonical Form and Invariant Factors ...................................................................... 53
3.4. Similarity of Matrices and Invariant Factors .............................................................................. 58
3.5. The Rational Canonical Forms ................................................................................................... 60
3.6. Elementary Divisors.................................................................................................................... 64
3.7. The Normal and Jordan Canonical Forms .................................................................................. 65
CHAPTER FOUR ....................................................................................................................................... 70
BILINEAR AND QUADRATIC FORMS ................................................................................................. 70
4.1. Bilinear forms and matrices ........................................................................................................ 70
4.2. Alternating bilinear forms ........................................................................................................... 73
4.3. Symmetric bilinear forms and quadratic forms ........................................................................... 74
4.4. Real Symmetric Bilinear Forms and Positive Definite Forms .................................................... 78
CHAPTER FIVE ........................................................................................................................................ 83
DIRECT SUM DECOMPOSITION OF VECTOR SPACES .................................................................... 83
5.1. Definition of a direct sum of vector spaces................................................................................. 83
5.2. Projection and Invariant subspaces of a linear operator .............................................................. 86
5.3. Primary Decomposition Theorem ............................................................................................... 89

II
 Linear Algebra II

CHAPTER ONE

THE CHARACTERISTIC EQUATION OF A MATRIX

Introduction
✓ Let V,V’ be vector spaces over the field K.

A linear mapping F:V → V’ is a mapping which satisfies the following two properties.

LM1: For any elements u,v in V we have

F(u+v)= F(u)+F(v)

LM2: For all C in K and v in V we have

F(cv) =cF(v).

If we wish to specify the field K, we also say that F is K-linear. Since we usually deal with a
fixed field K, we omit the prefix K, and say simply Linear.

Example-1:

Let V be any vector space. The mapping which associates to any element u of V this element
itself is obviously a linear mapping, which is called the identity mapping .We denote it by id or
simple I. Thus id(u)=u.

✓ /Kernel of a linear map/

Let V, W be vector spaces over K, and let F:V → W be a linear map. The set of elements

v  V such that F(v)=0 is called the Kernel of F.

Example-2:

Let P: R3 → R2 be the projection, such that

P(x,y,z)=(x,y).

Then the Kernel of P is a linear map whose first two coordinates are equal to 0, i.e. all vectors
(0,0,z) with arbitrary component z.

1.1 Eigenvalues and Eigenvectors

Goals:
✓ Master the definition and fundamental properties of eigenvalues and eigenvectors.

1
 Linear Algebra II

✓ Be able to compute eigenspaces.

✓ Know examples and be able to prove simple properties.

Example-3:

3 0
a) Consider M= [ ]  Mat2*2 (R) . We have :
0 2
3 0 1 1
[ ] [ ] = 3 [ ] and
0 2 0 0
3 0 0 0
[ ] [ ] = 2 [ ].
0 2 1 1
3 1
b) Consider M= [ ]  Mat2*2 (R) . We have :
0 2
3 1 a a
[ ] [ ] = 3 [ ] for all a  R.
0 2 0 0
3 1 a 3a + b a
[ ][ ] = [ ] = 2 [ ]  a=-[Link] for all a  R. We have
0 2 b 2b b
3 1 a a
[ ] [ ] = 2 [ ].
0 2 −a −a
5 1
c) Consider M= [ ]  Mat2*2 (R) . We have :
−4 10
5 1 1 1
[ ] [ ] = 9 [ ] and
−4 10 4 4
5 1 1 1
[ ] [ ] = 6 [ ].
−4 10 1 1
0 1
d) Consider M= [ ]  Mat2*2 (R) . We look at
−1 0
0 1 a b a
[ ] [ ] = [ ]. This vector is equal to  [ ] iff b=  a and a =-  b. This gives
−1 0 b −a b
a 0
a= -  a. Thus there is no   R with this property if [ ] ≠ [ ].
2
b 0
We will study these phenomena in general .Given a linear operator T:V → V, it is important in
many problems in science and mathematics to determine those scalars  for which T(v)=  v
has non-zero solutions.

Definition 1.1:

Let T:V → V be a linear operator on a vector space V over a field K. Then an element v  V is
called an eigenvector of T if there exists   K such that T(v)=  v.  is called an eigenvalue of T
belonging to the eigenvector v. we also say that v is an eigenvector with eigen value  .

2
 Linear Algebra II

Examples:

1. Let V=R2 and consider f(x,y) =(y,x). To determine the eigenvectors and eidenvalues by the
above definition
f(x,y) =(y,x)
=  (x,y)=(  x,  y)
y =  x and x =  y
 y-  x =0.
 Y-  (  y) =0.
 y-  2 y =0.
 y(1-  2 )=0.
 1-  2 =0 or y=0.
  =1,-1.
Then 1 and -1 are eigenvalues of f, and
E1(f) = {(x,x):x  R},
E-1(f) = {(x,-x):x  R}.The eigenvectors (1,1) and (1,-1) form a basis of V.
2. Let V be the vectorspace over R generated by all infinitely differentiable functions. Let
t
  R. Then the function f such that f(t) = e is an eigenvector of the derivative d/dt
because df/dt = e .And its eigenvalue  .
t

3. Let V = C∞ (R) be the space of infinitely differentiable functions on R.

Consider the linear operator D: f → f’’ . Then every   R is an eigenvalue, and
all eigenspaces are dimension two:
 L(1, x)if = 0.


(D)=  L(e , e if =   0.
x − x 2
E

 L(sin x, cos x)if = −   0.
2

Note :

If A is a square n*n matrix, with coefficients in K, then an eigenvector of A is by definition an

eigenvectorn K, then an eigenvector of A is by definition an eigenvector of the linear map of Kn
into itself represented by this matrix, relative to the usual basis. Thus an eigenvector X of A is a
9column) vector of Kn for which there exists   R such that AX=  X.

Example:

3
 Linear Algebra II

a11 ⋯ 0
A = [ ⋮ ⋱ ⋮ ] is a diagonal matrix in K. Then every unit vector (i=1,…,n) is an
0 ⋯ ann
eigenvector of A. In fact, we have: Aei = aiei.

Note that:

1. One of the meanings of the word “eigen“ in German is “proper.” Thus eigenvalues are also
called proper values or characteristics values or latent roots.
2. If v≠0, then  is uniquely determined, because:
 1v=  2v.
  1=  2.
3. If v and w are eigenvectors with eigenvalue  ,then:
i) V+w is also an eigenvector with eigenvalue  , because :
T(v+w) =T(v) + T(w) =λv + λw = λ(v+w).
ii) Each scalar multiple kv is also an eigenvector with eigenvalue λ, because:
T(kv) =kT(v)=k(λv) =  (kv).
4. Let T:V → V be a linear operator with ker T≠{0}.Let v be any non-zero vector in ker T. Then
T(v) =0=0.v.
So, v is an eigenvector of T with eigenvalue 0.

Examples:

1. Let id : V → V be the identity operator.

Every non-zero vector in V is an eigenvector of id with an eigenvalue 1,
since:
Id (v) = v=1v.
2→ 2
2. Let T: R R be a linear operator which rotates each v  R2 by an angle of 900.
Notice that   R , T(v) ≠  v.
Therefore, T has no eigenvectors and hence no eigenvalues.
3. Let D: V → V be the differential operator on the vector space of differentiable functions.
We have
5t 5t
D( e ) = 5 e .
5t
Hence, e is an eigenvector of D with eigenvalue 5.

Eigenspace of a linear operator

Let T: V → V a linear operator over a field K. Let   K.

4
 Linear Algebra II

Let Vλ= The set of all eigenvectors of T with eigenvalues λ.

Or Eλ(T)={v  V:T(v)=  v}=  -eigenspace of T.

Theorem -1.1:

Vλ is a subspace of V.

Proof:

i. T(0) = 0=  0,showing that 0  Vλ.

ii. V1,v2  Vλ  T(v1+v2)=T(v1)+T(v2)
=λv1+λv2=λ(v1+v2)
 v1+v2  Vλ
iii. v  Vλ,   K  T( v ) =  T(v)=  (v) =  (v)
  v  Vλ.
Definition 1.2:

Vλ is called the eigenspace or spectrum of T belonging to λ.

1.2. The Characteristic Polynomial

Eigenvalues and Eigenvectors of a Matrix
Definition -1.3:

Let A = an n×n matrix over a field K.

An element X  K n is said to be an eigenvector of A if there exist scalar λ such that

AX = λ X.

Note:

1. For X≠ 0, λ is uniquely determined, for λ1, λ2 satisfy equation then

λ1X = λ2X= AX  λ1 = λ2.
2. λ is called the eigenvalue corresponding to the eigenvector X and that the sets of all
eigenvalues is called the spectrum of A.
3. An eigenvector of A is a vector that remains unchanged or reversed direction when A
operates on the vector space.

How to determine the Eigenvalues and the corresponding Eigenvectors of a Matrix

5
 Linear Algebra II

X is an eigenvector with eigenvalue λ  AX = λ X.

 (A - λ I)X= 0. …………..(*)

a11 − λ ⋯ a1n x1 0
[ ⋮ ⋱ ⋮ ] [ ⋮ ]=[ ⋮ ].
an1 ⋯ ann − λ xn 0
X=0 is the trivial solution of (*).
Further solutions will exist iff |A − λIn |=0. Hence, solving the equation |A − λIn |=0
gives the eigenvalue of A.

Note:

1. |A − λIn | is called the characteristic polynomial of A. Or written out in full,

a11 − λ ⋯ a1n
PA (λ) = [ ⋮ ⋱ ⋮ ].
an1 ⋯ ann − λ
For an arbitrary matrix A = (aij ), the characteristic polynomial can be found by
expanding according to the first column, and will always consist of a sum:
(a11 − λ) (a22 − λ)…( ann − λ) + ….
 PA (λ)= (−1)n (λ)n + terms of lower degree.

2. The equation |A − λIn | = 0 is called the characteristic equation.

3. For any eigenvalue λ , the corresponding eigenvector is found by substituting λ back into
the equation (A − λIn )X= 0.

Example-1: Determine the characteristic polynomial of the matrix

1 −1 3
A= [−2 1 1 ].
0 1 −1
Solution:

λ−1 1 −3
PA (λ)= | 2 λ − 1 −1 |=|λIn − A| which we can expand according to the first
0 −1 λ + 1
column, to find
PA (λ)= λ3 - λ2 - 4 λ +6.

1 6
Example-2: A= [ ] . Find the eigenvectors and eigenvalues of A.
5 2
Solution:

PA (λ)= λ2 - tra(A) λ + det(A)= λ2 - 3 λ -28.

6
 Linear Algebra II

Or |A − λIn | = 0
1 6 1 0
|( ) − λ( )|= 0.
5 2 0 1
1−λ 6
| |= 0.
5 2−λ
 λ2 - 3 λ -28=0.

 λ= 7 or λ=-4.

The corresponding eigenvectors are now found.

a) Let λ=7.
Then (A − 7I2 )X =0.
1 6 1 0 x 0
 [( ) − 7( )] [ ] = [ ].
5 2 0 1 y 0
−6 6 x 0
( ) [ ] = [ ].
5 −5 y 0
 y=x.

Hence, any vector of the type  (1,1), where  isa non-zero scalar, is an eigenvector
corresponding to the eigenvalue 7.

b) Let λ= - 4.
Then (A + 4I2 )X =0.
1 6 1 0 x 0
 [( ) + 4( )] [y] = [ ].
5 2 0 1 0
5 6 x 0
( ) [ ] = [ ].
5 6 y 0
 5x + 6y= 0.
−5
 y= 6
x.

−5
Hence, any vector of the type  (1, ), where  is a non-zero scalar, is an eigenvector
6
corresponding to the eigenvalue - 4.

Note:

If X is an eigenvector with eigenvalue λ,  X is also an eigenvector with the same eigenvalue,

where  is non-zero scalar.

Justification

AX=λX…/given from the definition/

7
 Linear Algebra II

And A(  X) =  (AX) =  ( λ X)= λ(  X).

3 −2 0
Example-3: Let A= [−2 3 0].Find bases for eigenspace of A.
0 0 5
Solution:

Characteristic equation of A:

3 − λ −2 0
| −2 3 − λ 0 |=0.
0 0 5−λ
 (3 − λ) (3 − λ) (5 − λ) - 4(5 − λ) =0.
 [(3 − λ)2 – 4] (5−λ)=0.
 (λ2- 6 λ +5)( λ-5) = 0.
 ( λ − 1)( λ - 5) 2= 0.
So the eigenvalue of A:
λ = 1 and λ= 5.
x
From the definition, X =[y]is an eigenvector corresponding to λ iff X is non-trivial solution of
z
(A − λI3 )X =[Link] is,

3 − λ −2 0 x 0
[ −2 3 − λ y
0 ] [ ] = [0]…………………….(*)
0 0 5−λ z 0
a) Let λ=1. Then (*) becomes
3 − 1 −2 0 x 0
[ −2 3 − 1 0 ] [y] = [0]
0 0 5−1 z 0

2 −2 0 x 0
 [−2 2 0] [y] = [0].
0 0 4 z 0
 2x-2y = 0 and 4z = 0.
 x= y and z=0.

Thus, the eigenvectors corresponding to eigenvalue 1 are non-zero vectors of the form:
x 1
X =[ x]. Therefore, {[1]} is a base for V1 = The eigenspace of A corresponding to
0 0
eigenvalue 1.
b) Let λ=5. Then (*) becomes

8
 Linear Algebra II

3 − 5 −2 0 x 0
[ −2 3 − 5 0 ] [y ] = [ 0]
0 0 5−5 z 0

−2 −2 0 x 0
 [−2 −2 0] [y] = [0].
0 0 0 z 0
 -2x-2y = 0 .
 x= - y .

Thus, the eigenvectors corresponding to eigenvalue 5 are non-zero vectors of the form:
x x 0 1 0
−x −x
X =[ ]= [ ] + [0]= x [−1] + z [0].
z 0 z 0 1
1 0
Therefore, {[−1] , [0]} is a base for V5 = The eigenspace of A corresponding to eigenvalue 5.
0 1
Note:

If v1, v2 are eigenvectors of a linear operator T:V → V, with eigenvalues λ1≠ λ2 , then v1 + v2
isnot an eigenvector of T.

Theorem 1.1.3: Let v1, v2,…, vn be eigenvectors of a linear operator T:V → V, with eigenvalues
λ1, λ2,…, λn respectively. If λi≠ λj , then { v1, v2,…, vn } is Linearly independent.
Proof: By induction on n.
If n=1, an element v1 ∈V, v1 ≠ 0, is linearly independent.
Assume n >1.
Suppose α1 v1+α2 v2+…,+αn vn =0, with αi ∈ K. ----------------------- (1)
WTS: αi =0, for all i.
Multiplying equation (1) by λ1 , we obtain:
i. λ1 α1 v1+ α2 v2 λ1 +…,+αn vn λ1 =0
We apply T to equation (1) , we obtain:
ii. λ1 α1 v1+ α2 v2 λ2 +…+αn vn λn =0.
Subtracting (i) from (ii), we obtain:
α2 v2 (λ2- λ1) + α3 v3 (λ3- λ1) +…,+αn vn( λn- λ1) =0.
Since λj- λ1 ≠ for j = 2,3,…,n. We conclude by induction that
α2 = α3 =…=αn = 0.

Going back to the original relation, we see that α1 v1 =0. Hence α1 =0, and the theorem is
proved.

Exercise - 1.1:

9
 Linear Algebra II

1. Find the eigenvalues and associated eigenspaces of each of the following matrices.

0 1 0 1 1 1
1 1
a) A=[ ] b) A = [0 0 1] c) A = [0 2 1]
−2 3
0 0 0 0 0 1

2. Let A be an n*n matrix. Prove that A is singular if and only if λ=0 is an eigenvalue of A.
1
3. Let A be a non-singular matrix and let λ be nonzero eigenvalue of A. Show that is an
λ
eigenvalue of A−1.
4. Let A be n*n matrix. Show that A and AT have the same eigenvalues. Do they also share
eigenvectors?
5. Show that the eigenvalues of a triangular matrix are the diagonal elements of the matrix.

1.3. Similarity of Matrices and Characteristic Polynomial

Definition 1.3.1:

Let A and B be n*n matrices over a field K. We say that B is similar to A over K if there is an
invertible n*n matrix P over K such that B =P-1AP./PB=AP/

Notation: B∼A.

1 2 1 0
Example: Let A= [ ] and B = [ ], show that B∼A.
0 −1 −2 −1
1 2 1 −1 3 1 1 −1 1 0
AP=PB → [ ][ ]=[ ]=[ ][ ].
0 −1 1 1 −1 −1 1 1 −2 −1
Properties of similar matrices

For any n*n matrices A, B and C.

1. A is similar to itself.

Proof: A= I-1AI using P= In.

2. If B∼A, then A∼ B.

Proof: If B= P-1AP, then PBP-1 =A → (P-1)-1BP-1 =A.

3. If B∼A and C∼B, then C∼A.

Proof: B∼A→ B= P-1AP and C∼B → C= Q-1AQ implies C= Q-1P-1APQ = (PQ)-1A(PQ)

→ C∼A.

4. The only matrix similar to the identity matrix In is In itself.

10
 Linear Algebra II

Proof: In= P-1AP. → Pn = AP. → P=AP. → A= In .

5. The only matrix similar to the zero matrix is the zero matrix itself.
6. If A and B are similar, so are Ak and Bk for any k=1,2,….
Proof: If B= P-1AP , then
Bk = (P-1AP)k.
= (P-1AP) (P-1AP)… (P-1AP) / k times /
=P A(PP )A(PP )… (PP )AP.
-1 -1 -1 -1

= P-1AIAI… IAP = P-1Ak P.

Then Ak and Bk are similar.
7. If A and B are similar , so are AT and BT.
Proof: If B= P-1AP, then BT = (P-1AP)T = PTAT(P-1)T = [(P-1)T]-1AT(P-1)T.
So for Q = (P-1)T , and BT=Q-1ATQ. That means that AT and BT are similar.

Similar Matrices share many similarity invariants

Similar matrices have the same

a) Determinant and invertibility.

b) Characteristics equation and eigenvalues
c) Trace
d) Rank.

Proof: / of the first two only/. Suppose that A and B are similar,i.e. that B= P-1AP for some
matrix P .

a) Det(B) = det(P-1AP)
= det(P-1)det(A)det(P).
= [det(P)]-1det(A)det(P) = det(A).
So the matrices have the same determinant, and one is invertible if the other is.
b) Det(B-λI)=det(P-1AP- λP-1P)

= det[P-1(A- λI)P]

= det(P-1)det(A- λI)det(P).

= det(A- λI).

So the matrices have the same characteristic polynomial and hence the same eigenvalues.

Example: Consider the pairs of matrices:

1 2 2 1
i) Let A= [ ] and B = [ ] are not similar since det(A)=-3 but det(B)=3.
2 1 1 2

11
 Linear Algebra II

1 3 1 1
ii) Let A= [ ] and B = [ ] are not similar since the characteristic polynomial of
2 2 3 −1
A is λ2-3λ-4 while B has λ2-4.

Activity

1 0 1 1
Consider A= [ ] and B = [ ] both have determinant 1 and rank 2, are invertible and have
0 1 0 1
characteristic polynomial (1- λ)2 and eigenvalues λ1= λ2=[Link] these two matrices similar?

Theorem 1.3.1:

A matrix A and its transpose AT have the same characteristics polynomial.

Proof: Notice (λI – A) T = λI – A T .
Therefore, |λI – A| = |(λI – A) T |=|λI – A T |.
Hence, A and AT have the same characteristic polynomial.
Exercise-1.2:
1. If A and B are similar and invertible, then A-1 and B-1 are similar for A and B are n*n
matrices.
2. If A and B are similar and q is any polynomial , then q(A) and q(B) are similar.

1.4. The Spectral radius of a matrix

The spectrum of a square matrix A, denoted by σ(A) is the set of all eigenvalues of A.
The spectral radius of A, defined as:
ρ(A)= max{|λ| : λ ∈ σ(A)}

An eigenvalue of A that is larger in absolute value than any other eigenvalue is called a
dominant eigenvalue.

9 −1 2
Example: Consider the matrix A = [−2 8 4] whose eigenvalues are 5,10,10.
1 1 8
σ(A)= {5,10}.

ρ(A)= max{5,10}.

Exercise-1.3:

Determine the spectrum of matrix and spectral radius of A.

5 1 2 1 0 1
i) A=[ ] ii) A=[ ] iii) A=[ ]
−4 10 0 2 −1 0

12
 Linear Algebra II

1.5. Diagonalization
The best we can hope for when we are given a matrix is when it is similar to a diagonal matrix.
In fact there is a close relationship between when a matrix is diaonalizable and the eigenvalues
and eigenvectors of a matrix.

Definition: A n*n matrix A is diagonalizable if there is a diagonal matrix D such that A is

similar to D, i.e. there is some n*n invertible matrix P such that P-1AP=D.

1 3 1 3
Example: The matrix A =[ ] is diagonalizable since the matrix P= [ ] and
2 2 1 −2
4 0
D=[ ] produce the identity P-1AP=D or equivalently AP=PD.
0 −1
This is wonderful, but we have no idea where P and D arose from .To answer this question we
note that the diagonal entries 4 and -1 of D are the eigenvalues of A . How P is found is a more
interesting question, as in the case of D, the entries of P are related to the eigenvectors of A.

Theorem: Let A be an n*n matrix, then A is diagonalizable if and only if A has n linearly
independent eigenvectors.
To be precise , there exists an invertible matrix P with a diagonal matrix D such that P-1AP=D if
and only if the columns of P are linearly independent eigenvectors of A and the diagonal entries
of D are the eigenvalues of A corresponding to the eigenvectors in P in the same order.
Corrollary:

Let A be 2*2 matrix having distinct eigenvalues λ1, λ2 and corresponding eigenvectors X1 and
X2 .Let P be the matrix whose columns are X1 and X2 respectively. Then P is non-singular and P-
λ 0
1
AP = [ 1 ].
0 λ2

2 1 −1 1
Example-1: Let A= [ ]. Then X1 = ( ) and X2 = ( ) are eigenvectors assoiciated to the
1 2 1 1
−1 1 1 0
eigenvalues 1 and 3 respectively. Hence if P=[ ],we have P-1AP = [ ].
1 1 0 3
0 1 0
Example-2: Determine whether a matrix P exists to diagonalizable A = [0 0 1].
2 −5 4
Solution: We have seen that is matrix has eigenvalues λ1 = λ2 =1 and λ3 =2 with the
corresponding eigenspaces

1 1
E1 = span{[1]}, E2 = span{[2]}.
1 4

13
 Linear Algebra II

As all other eigenvectors are just multiples of one of these two basis vectors there cannot be
three linearly dependent eigenvectors. Thus it cannot be diagonalized.

Example -3: Find a P that will diagonalize

−1 0 1
A = [ 3 0 −3].
1 0 −1
Solution: The eigenvalues λ1 = λ2 =0 and λ3 = -2 with the basis for the eigenspaces.

0 1 −1
E0 = span {[1] , [0]}, E-2 = span{[ 3 ]}.
0 1 1
It is easy to check that the three vectors are linearly independent, and so we from P.

0 1 −1
P = [1 0 3 ] this matrix will be invertible and furthermore
0 1 1
0 0 0
P-1AP = [0 0 0 ]=D.
0 0 −2
Properties of Diagonalize a matrix

1. If A is a n*n matrix with n distinct eigenvalues, then A is diagonalize.

Proof: Let v1, v2,…, vn be eigenvectors corresponding to the n distinct eigenvalues of A. These
vectors linearly independent by previous theorem. A is diagonalizable.

2 −3 7
Example: The matrix A = [0 5 1 ] has eigenvalues λ1 = 2, λ2 =5 and λ3 = -1, as these
0 0 −1
eigenvalues are distinct for the 3*3 matrix A, A is diagonalizable.

2. To calculate An ./A2*2 /
λ 0 λ 0 -1
If P-1AP = [ 1 ], then A = P [ 1 ] P and
0 λ2 0 λ2
n λ1 0 -1 n λ1 0 n -1 λ1 n 0
A = [P [ ]P ] = P[ ] P =P[ ] P-1
0 λ2 0 λ2 0 λ2 n

0 1
Example: Compute A10 if A= [ ].
2 1
Solution: Here

14
 Linear Algebra II

−1
λ1 = -1 → v1 = ( )
1
1
λ2 =2 → v2 = ( ).
2
1 1 −1 0
→P = [ ] and D = [ ].
−1 2 0 2
2 −1
10
λ 0 -1 10 −1 0 10 -1 1 1 (−1) 0
A = [P [ 1
10
]P ] = P[ ] P =[ ] [ ] [3 3
].
0 λ2 0 2 −1 2 0 (2)10 1 1
3 3
342 341
=[ ].
682 683

1.6. Decomposable Matrices

Block Matrices

Let A be a matrix over a field K. Partition A in to parts means of a system of horizontal and
vertical lines.

1 2 3 4 1 2 3 4 1 2 3 4
Example: A= [5 6 7 8] = [5 6 7 8 ] = [5 6 7 8].
9 0 1 3 9 0 1 3 9 0 1 3
The parts are smaller matrices and are called blocks of the matrix A.

A matrix A partitioned into blocks is called a block matrix.

The blocks may be considered as elements of A.

Diagonal Blocks

Given a square matrix A, it is often necessary to partition in to blocks so that the diagonal blocks
are square matrices.

A1 0 … 0
0 A2 … 0
A block matrix of the form: A = [ ]. Where A1 , A2 ,…, Ar are square matrices
⋮ ⋮ ⋱ ⋮
0 0 ⋯ Ar
and each zero is a zero matrix of proper dimensions, is called a diagonal block matrix.

Note:

1. We say that :
i. A is decomposed into blocks A1 , A2 ,…, Ar .

15
 Linear Algebra II

Or

ii. A is a direct sum of matrices A1 , A2 ,…, Ar .

2. By analogy with diagonal matrices , one writes symbolically: A = diag(A1 , A2 ,…, Ar ).
Prosperities of Decomposable Matrices

Let A be a square matrix and suppose A = diag(A1 , A2 ,…, Ar ).

Then

i. |A|= |A1 ||A2 | …|Ar |.

ii. If A is non- singular, A-1 = diag(A1 −1 , A2 −1 , … Ar −1 ).
iii. If B= diag(B1 , B2 ,…, Br ) also with Ai the same size as Bi for i = 1,2,…,r, then

AB= diag(A1 B1 , A2 B2 ,…, Ar Br ).

2 −3 7 2 −3 7
Example-1: Let A = [0 5 1 ] , B = [0 5 1 ]. Then it is not difficult to see that
0 0 −1 0 0 −1
1 2 6 7 0 22 25 0
{[ ][ ]} [ ]
AB= diag {[ 3 4 8 9 0 ]} = [ 50 57 0].
0 0 5∗1 0 0 5
1 3 0 0
Example-2: Find |A|. Let A= [0 2 0 0 ].
0 0 4 3
0 0 −1 1
1 3 4 3
Solution: A1 = | |=2. A2 = | |=4+3=7.
0 2 −1 1
|A|= |A1 ||A2 | = 2*7=14.

1 3 0
Example-3: Find A-1 . Let A = [2 2 0] .
0 0 1
1 3
Solution : A1 = [ ] ; A2 = [1].
2 2
−1 3
−1 2 4
A1 = [ 1 −1]; A2 −1 = [1].
2 4

−1 3
0
2 4
-1
A =[ 1 −1
0].
2 4
0 0 1

16
 Linear Algebra II

Theorem 1.6.1: The characteristic polynomial of a decomposable matrix is the product of

characteristic polynomial of its diagonal blocks.

Proof: Let A = diag(A1 , A2 ,…, Ar ), where Ai is a square matrix of order ni for i= 1,2,…,r.

Then, A − λIn = diag (A1 -λIn1 , A2 -λIn2 , …, Ar − λInr ), where Ini is an identity matrix of order
ni .

|A − λIn |= |A1 − λIn1 ||A2 − λIn2 |…|Ar − λInr |.

I.e. 𝒳A (λ) = 𝒳A1 (λ) 𝒳A2 (λ)… 𝒳Ar (λ).

1 3 0
Example-4: Determine the characteristics polynomial of A = [2 2 0].
0 0 1
1−λ 3
Solution: 𝒳A1 (λ) = | |=(1 − λ) (2 − λ)-6= λ 2-3λ-4.
2 2−λ
𝒳A2 (λ) =|1 − λ|= (1 − λ)

𝒳A (λ) = 𝒳A1 (λ) 𝒳A2 (λ)= ( λ 2-3λ-4) (1 − λ).

Exercise-1.4:

A1 B
1. Show that , suppose M= [ ] , where A1 and A2 are square matrices. Then
0 A2

𝒳M (λ) = 𝒳A1 (λ) 𝒳A2 (λ).

A1 B … C
0 A2 … D
2. Suppose M = [ ], Where A1 , A2 ,…, Ar are square matrices.
⋮ ⋮ ⋱ ⋮
0 0 ⋯ Ar
Then, show that:
𝒳M (λ) = 𝒳A1 (λ) 𝒳A2 (λ)… 𝒳Ar (λ).

1.7. Minimal Polynomial and Cayley-Hamilton Theorem

Let A= an n*n matrix over a field K.

Let f(x) ϵ k[x].

Suppose f(x) = at x t +at−1 x t−1 + ….+ a1 x + a0 .

By f(A) we mean : f(A)= at At +at−1 At−1 + ….+ a1 A + a0 In .

We say f annihilates A if f(A)=0.

17
 Linear Algebra II

Claim: There exists a non-zero polynomial f annihilating A.

Proof: All n*n matrices form a vectorspace of dimension n2. Hence, the matrices
2
I,A,A2 , …,An are linearly independent. Therefore, there exists scalars c0 , c1 , c2 ,…, cn2 , not all
2
zero, such that : c0 I+c1 A+c2 A2 + ⋯+cn2 An =0.
2
Let f(x) = c0 +c1 x+c2 x 2 + ⋯+cn2 x n .

f(x) ≠ 0,deg ≤ n2 and f(A)=0.

Among all the non-zero polynomials annihilating A, consider those of least degree. By
multiplying by suitable non-zero constants, we can ensure that they are monic.

Claim: There is only one monic polynomial of least degree annihilating A.

Proof: Suppose f1 and f2 are two such polynomials.

Let f1 ≠ f2. Let g = f1 - f2 .

Then g≠ 0 and deg(g) < deg (f1).

Let g(A)= f1 (A)- f2(A)=0.→← / This contradict the definition of f1 ,f2 as non-zero polynomials of
least degree annihilating A./

Thus, f1 = f2.

Definition 1.7.1:

Let A be an n*n matrix over a field K. The unique monic polynomial of least degree, which
annihilates A is called the minimial polynomial of A and is denoted by mA (x).

Theorem 1.7.1: Let A be an n*n matrix over a field K. A is non-singular if and only if the
constant term of mA (x) is non-zero.

Proof: Let mA (x) = x t +at−1 x t−1 + ….+ a1 x + a0 .

(→): Suppose A is non-singular.

0=mA (A)= At +at−1 At−1 + ….+ a1 A + a0 In .

If a0 = 0, then A(At−1 +at−1 At−2 + ….+ a2 A + a1 In ) = 0.

Hence, At−1+at−1 At−2 + ….+ a2 A + a1 In =0.

→←/ contradicts the minimality of mA (x) /.

Therefore, a0 ≠ 0.

18
 Linear Algebra II

(←): Suppose a0 ≠ 0.

0=mA (A)= At +at−1 At−1 + ….+ a1 A + a0 In .

−1
Therefore, I= (At−1 +at−1 At−2 + ….+ a2 A + a1 In )A.
a0

Hence, A is non-singular.

Theorem 1.7.2:/Cayley- Hamilton/

Every square matrix A is a root of its own characteristic polynomial, i.e.

𝒳A (A)=0.

Proof: Let B= adj(xI-A).

P11 (x) P12 (x) … P1n (x)

=[ ⋮ ⋮ ⋱ ⋮ ].
Pn1 (x) Pn2 (x) … Pnn (x)

Where the Pij (x) are polynomials in x. B can be written in polynomial in x having matrix
coefficients. Let B= Bk x k +Bk−1 x k−1 + ….+ B1 x + B0 = f(x).

Where B0 , B1 ,… , Bk are matrices whose entries are scalars.

Let 𝒳A (x)= |xI − A| = an x n + an−1 x n−1 + ….+ a1 x + a0 .

By property of adjoint , we have:

B(xI-A) = 𝒳A (x)I.

→(Bk x k +Bk−1 x k−1 + ….+ B1 x + B0 )(xI-A) = (an x n + an−1 x n−1 + ….+ a1 x + a0 )I.

→f(x) (xI-A) = q(x), where q(x)= (an I)x n + (an−1 I) x n−1 + ….+ (a1 I) x + a0 I.

→q(A)= 0,

from g(x)= f(x) (xI-A)= (Bk x k +Bk−1 x k−1 + ….+ B1 x + B0 )(xI-A)

= - B0 A+ (- B1 A + B0 )x + (- B2 A + B1 ) x 2 + … +(- Bk A + Bk−1) x k + Bk x k+1 .

g(A) = - B0 A+ (- B1 A + B0 )A + (- B2A + B1 ) A2 + … +(- Bk A + Bk−1) Ak + Bk Ak+1

=0.

But q(A)= 𝒳A (A)=0.

Theorem 1.7.3:

19
 Linear Algebra II

Let A be a square matrix. mA (x) is a divisor of every polynomial which annihilates A.

Proof: Let f(x) ∈ K[x] such that f(A) =0. By division Algorithm, there exist q(x), r(x) ∈ K[x]
such that : f(x) = q(x) mA (x) + r(x), where either r(x) = 0or else deg r(x) < deg mA (x) .

Therefore, 0= f(A) = q(A) mA (A) + r(A) = r(A).

By minimality of mA (x), it follows that r(x) = 0.

Therefore, f(x) = q(x) mA (x) .i.e. mA (x)/ f(x).

Corollary: Let A be a square matrix. Then mA (x)/ 𝒳A (x).

Example-1: Here are key examples to understand the difference between minimal and
characteristic polynomial. The following three matrices have the same characteristic
polynomials,(x-1)2 :

1 0 1 1 1 169
M1= [ ], M2 = [ ], M3 = [ ].
0 1 0 1 0 1
0 1 0 169
The minimal polynomial of M1 is [Link] M2 − I = [ ] ≠ 02 and M3 − I = [ ] ≠ 02 ,
0 0 0 0
the minimal polynomial has to be them.

1 4
Example-2: Verified Cayley Hamilton Theorem for A= [ ]. Find A−1.
2 3
Solution: We know that |A − λI2 | =0.

1−λ 4
| |= 0.
2 3−λ
λ2 -4λ -5= 0.

The characteristic equation of A is:

P(λ) = λ2 -4λ -5=0.

Replace λ with A: P(A) = A2 -4A -5I=0………………………………..(eqn. 1)

1 42 1 4 1 0
So, [ ] − 4[ ]-5[ ]
2 3 2 3 0 1
9 16 −4 −16 −5 0 0 0
[ ]+[ ]+[ ]= [ ].
8 17 −8 −12 0 −5 0 0
Cayley- Hamilton theorem is verified.

Multiplying (eqn. 1) by A−1 , we get,

20
 Linear Algebra II

A−1(A2 -4A -5I) = 0.

A-4I-5A−1=0.
1
A−1 = 5 [A-4I].

1 1 4 1 0
= 5 [[ ]-4 [ ]].
2 3 0 1
1 −3 4
=5[ ].
2 −1
Exercise-1.5:

1. Verified Cayley Hamilton Theorem for A and find A−1.

1 1
a) A = [ ].
2 1
2 1 1
b) A = [0 1 0]
1 1 2
1 2 0
c) A= [−1 1 2]
1 2 1
2. Let f(x), g(x) be polynomials in x with matrix coefficient; and suppose that they are related
by the equation g(x) = f(x) (xI-A), where A is a given matrix. Then g(A)=0.

21
 Linear Algebra II

CHAPTER TWO

ORTHOGONALITY
2.1. The Inner Product
Let V be a vector space over K where K=R or C.

If K=R, V is called a Real vector space.

If K=C, V is called a Complex vector space.

Definition -2.1.1:

Let V be a real or complex vector space. An inner product on V is a function , :V×V → K

satisfying the following properties for all v,u,w ∈ V,α ∈ K.
______
1) v, w = w,v
2) v, v ≥ 0 and v, v =0 iff v=0.
3) v, u + w = v, u + v, w .
4) v, w =  v, w .

2.2. Inner Product space

Definition -2.2.1: A real or complex vector space V with an inner product is called an Inner
Product space .

Note:

i. A real inner product space is called a Euclidian Space.

ii. A complex inner product space is called a Unitary Space.

Example-1: Let V= Rn be the real vectorspace.

Let u= (u1 , u2 , … , un ) ∈ Rn .

v= (v1 , v2 , … , vn ) ∈ Rn .

Define u, v = ∑ni=1 ui vi = u1 v1 + u2 v2 + …. + un vn . Show that it is an inner product called

dot product /standard inner product/

Solution

i) u, u = ∑ni=1 ui ui = u1 u1 + u2 u2 + …. + un un .

22
 Linear Algebra II

= u1 2 + u2 2 + … + un 2 ≥ 0.
and u, u = 0 iff u1 2 + u2 2 + … + un 2 = 0.
↔ u1 2 = u2 2 = … = un 2 = 0.
↔ ui =0,for all i=1,2,….
↔u = 0.
ii) u, v = ∑ni=1 ui vi = ∑ni=1 vi ui = v, u = v, u .
iii) Let w= (w1 , w2 , … , wn ) ∈ Rn .
v, u + w = 〈(v1 , v2 , … , vn ), (u1 + w1 , u2 + w2 , … , un + wn )〉.
= v1 (u1 + w1 ) + v2 (u2 + w2 ) + …+ vn (un + wn ).
= ( v1 u1 ++ v2 u2 + …+ vn wn ) + ( v1 w1 ++ v2 w2 + …+ vn wn )
= v, u + u, v .
iv) u, v = 〈(αu1 , αu2 , … , αun ), (v1 , v2 , … , vn )〉
= αu1 v1 + αu2 v2 + …+ αun vn .
=α[u1 v1 + u2 v2 +…+ un vn ].
= α u, v .

Therefore, , is an inner product.

Example-2: Let f,g ∈ C[a,b] →C./ The space of continuous complex valued functions on [a,b].
b
̅̅̅̅̅ dt. Show that f , g inner product.
Define f , g = ∫ f(t)g(t)
a

b
i) ̅̅̅̅̅ =∫b|f|2dt ≥ 0.
f , f =∫a f(t)f(t) a
b
and ∫a |f|2 dt = 0 iff f(t)= 0.
b
ii) f ,g ̅̅̅̅̅ dt
=∫a f(t)g(t)
b
g, f =∫a g(t)f(t)̅̅̅̅̅ dt
b ̅̅̅̅̅̅̅̅̅̅
g, f =∫a g(t)f(t) ̅̅̅̅̅ dt.
b
̅̅̅̅̅ f(t) dt.
=∫a g(t)
b
̅̅̅̅̅ dt.
=∫a f(t)g(t)
Therefore, f , g = g, f
.
iii) Let α be a scalar
b
̅̅̅̅̅ dt =α ∫b f(t)g(t)
f , g =∫a αf(t)g(t) ̅̅̅̅̅ dt = α f , g
a .
iv) Let h ∈ C[a,b].
b ̅̅̅̅̅̅̅̅̅̅̅̅̅̅
f , g + h =∫a f(t)g(t) + h(t) dt

23
 Linear Algebra II

b ̅̅̅̅̅ } dt
̅̅̅̅̅ + f(t)h(t)
=∫a {f(t)g(t)
b
̅̅̅̅̅ dt + ∫b f(t)h(t)
=∫a f(t)g(t) ̅̅̅̅̅ dt
a
= f , g + f ,h
.

Therefore, , is an inner product.

Exercise-2.1:

1) V= Rn and v, w = v  w. This inner product space is usually referred to as a Euclidean

n- space.
2) V= The space of continuous real-valued functions on [0,1].
1
f , g = ∫0 f(t)g(t)dt.
3) V= The vector space of m×n matrices over R.
A, B = Trace (BtA).

Definition 2.2.2

Let V be an inner product space. Let v, w ∈ V. We say v and w are orthogonal, and denote v⊥w,
if v, w = 0.

Definition 2.2.3

Let V be an inner product space. Let S ⊆ V , S ≠ ∅. We define the orthogonal complement of S,

denoted by S ⊥ , by :

S ⊥ = { v ∈ V/ v, w = 0 for all w ∈ S}.

= The set of all elements of V which are orthogonal to all elements of S.

Theorem 2.2.1

S ⊥ is a subspace of V.

Proof:

i) 0 ∈ S ⊥ , since 0, v = 0 for each v ∈ S.

ii) v1 , v2 ∈ S ⊥ .
→ 〈v1 , w〉=0=〈v2 , w〉, for all w∈ S.
→ 〈v1 + v2 , w〉 = 〈v1 , w〉 + 〈v2 , w〉=0, all w∈ S.
→ v1 + v2 ∈ S ⊥ .
iii) v ∈ S ⊥ and λ ∈ K.

24
 Linear Algebra II

→ v, w = 0 for all w ∈ S.
→ v, w = λ v, w = λ×0=0 for all w ∈ S.
→ λv ∈ S ⊥ .

𝐃𝐞𝐟𝐢𝐧𝐢𝐭𝐢𝐨𝐧 𝟐.2.4

v, w
Let V be a Euclidean space. Let v, w ∈ V, w ≠0. The number is called the Fourier
w, w
coefficient of v with respect to w.

Theorem 2.2.2

v, w
Let V be a Euclidean space. Let v, w ∈ V, w ≠0. Let λ = be the Fourier coefficient of v
w, w
with respect to w. Then, (v- λw ) ⊥ w.

Proof: v − w, w = v, w - w, w .

= v, w - λ w, w

v, w
= v, w - w, w
w, w

=0.

Definition 2.2.5

v, w
Let V be a Euclidean space. Let v, w ∈ V, w ≠0. Let λ = be the Fourier coefficient of v
w, w
with respect to w. The vector λw is called the projection of v along w and is denoted by Projvw .

2.3. Orthonormal Sets

Definition 2.3.1

Let V be an inner product space. Let v ∈ V. We define the norm(length ) of v by:

‖v‖ = √ v, v .

Note:

25
 Linear Algebra II

1) If ‖v‖ = 1, then v is called a unit vector / normalized vector/.

v
2) Every non-zero vector v ∈ V can be normalized by setting μ = ‖v‖.

Definition 2.3.2

Let V be an inner product space of dimension n. A basis {v1 , v2 , … , vn } of V is called an

orthogonal basis if 〈vi , vj 〉 = 0, for all i ≠ j.

If in addition, each vi is a unit vector, the basis {v1 , v2 , … , vn } is called an orthonormal basis.

Theorem 2.3.1

Let V be an inner product space. Let {v1 , v2 , … , vn } is called an orthogonal basis of V. Suppose

{x1 , x2 , … , xn } is the coordinate vector of v ∈ V with respect to the given orthogonal basis.
〈v,v 〉
Then, xi = 〈v ,vi 〉 = The Fourier coefficient of v with respect to vi .
i i

Proof:

V = ∑nj=1 xj vj .

Therefore, 〈v, vi 〉 = 〈∑nj=1 xj vj , vi 〉.

= ∑nj=1 xj 〈vj , vi 〉

=xi ∑nj=1〈vj , vi 〉.

=xi 〈vi , vi 〉, since 〈vj , vi 〉 = 0 for all j≠i.

〈v,v 〉
Therefore, xi = 〈v ,vi 〉 .
i i

2.4. The Gram-Schmidt Orthogonalization process

Lemma: Let V be an inner product space. Let {u1 , u2 , … , ur } be an orthonormal set in V.

Then
(i) {u1 , u2 , … , ur } is linearly independent.
(ii) For any v ∈ V , the vector
w = v- 〈v, u1 〉u1 -〈v, u2 〉u2 - …-〈v, ur 〉ur is orthogonal to each of the ui .

Proof:

i) Suppose α1 u1 + α2 u2 , … + αr ur = 0.
Taking the inner product of both sides with respect to ui :

26
 Linear Algebra II

0= 〈0, ui 〉= 〈α1 u1 + α2 u2 , … + αr ur , ui 〉=αi .

Since i is arbitrary, α1 =α2 = …=αr =0.
ii) Taking the inner product of w with each ui :
〈w, ui 〉= 〈v, ui 〉- 〈v, u1 〉〈u1 , ui 〉- …-〈v, ur 〉〈ur , ui 〉.
= 〈v, ui 〉-1〈v, ui 〉=0.

Theorem 2.4.1: Let V be a Euclidean space. Let{v1 , v2 , … , vn } be an arbitrary basis of V. Then,

there exists an orthonormal basis {u1 , u2 , … , un }of V such that the translation matrix from
{vi }to { ui } is triangular, that is, for i=1,2,…,n,

ui = ai1 v1 + ai2 v2 +…+ aii vi .

Proof:
v
i) Set u1 = ‖v1‖ . Then { ui } is orthonormal.
1
w2
ii) Next, set w2 = v2 - 〈v2 , u1 〉u1 and u2 = ‖w2 ‖
.By Lemma , w2 /and hence u2 / is
orthogonal to u1 . Then {u1 , u2 } is orthonormal.
w
iii) Next, set w3 = v3 - 〈v3 , u1 〉u1 −〈v3 , u2 〉u2 and u3 = ‖w3‖ .By Lemma , w3 /and hence
3
u3 / is orthogonal to u1 and u2 .Then {u1 , u2 , u3 } is orthonormal.
iv) In general , after obtaining {u1 , u2 , … , ui }, set
wi+1 = vi+1 - 〈vi+1 , u1 〉u1 −〈vi+1 , u2 〉u2 - …- 〈vi+1 , ui 〉ui ; and
w
ui+1 = ‖wi+1 ‖. Then, {u1 , u2 , … , ui , ui+1 } is [Link] induction, we obtain an
i+1
orthonormal set {u1 , u2 , … , un }which is linearly independent and hence a basis for V.

Note:

The proof of the above theorem is also a method for finding an orthogonal basis given an
arbitrary basis. It is called the Gram-Schmidt orthonogonalization process.

Example-1

Consider R3 with the inner product is the dot product. Consider the basis {v1 , v2 , v3 }. Where

v1 = (1,1,1), v2 = (0,1,1)and v3 = (0,0,1). We use the Gram-Schmidt orthonogonalization

process to obtain a normal basis of R3 .

Solution
v (1,1,1) (1,1,1) 1 1 1
i) Set u1 = ‖v1‖ = ‖(1,1,1)‖ = =( , , ).
1 √3 √3 √3 √3
ii) Next, set w2 = v2 - 〈v2 , u1 〉u1
1 1 1 1 1 1
= (0,1,1) - 〈(0,1,1), ( , , )〉 ( , , ).
√3 √3 √3 √3 √3 √3

27
 Linear Algebra II

2 1 1 1
= (0,1,1) – ( , , )
√3 √3 √3 √3
2 2 2 −2 1 1
=(0,1,1) – (3 , 3 , 3) = ( 3 , 3 , 3).

−2 1 1 −2 1 1
w (, , ) ( , , ) −2 1 1
and u2 = ‖w2‖ = 3 33
−2 1 1 = 3 33
=( , , ).
2 ‖( , , )‖ √
2 √6 √6 √6
3 33
3

iii) Next, set w3 = v3 - 〈v3 , u1 〉u1 −〈v3 , u2 〉u2

1 1 1 1 1 1 −2 1 1 −2 1 1
=(0,0,1)- 〈(0,0,1), ( , , )〉 ( , , )-〈(0,0,1), ( , , )〉 ( , , )
√3 √3 √3 √3 √3 √3 √6 √6 √6 √6 √6 √6
1 1 1 1 1 −2 1 1
=(0,0,1)- ( , , )- ( , , ).
√3 √3 √3 √3 √6 √6 √6 √6
1 1 1 −2 1 1
=(0,0,1)- (3 , 3 , 3)-( 6 , 6 , 6).
−1 1
= (0, , 2).
2

−1 1 −1 1
w (0,
, ) (0, , ) −1 1
and u3 = ‖w3‖ = 2 2
−1 1 = 2 2
= (0, , ).
3 ‖(0, , )‖ √
1 √2 √2
2 2
2

By Lemma , w3 /and hence u3 / is orthogonal to u1 and u2 .Then {u1 , u2 , u3 } is orthonormal. The

1 1 1 −2 1 1 −1 1
required normal basis of R3 is:{ ( , , ),( , , ) , (0, , )}
√3 √3 √3 √6 √6 √6 √2 √2

Exercise-2.2:

Consider {(1,-1,1,1),(1,0,1,0),(0,1,0,1)} in R4 with the standard inner product/dot product/. Find

orthonormal set of vectors in R4 .

2.5. Cauchy-Schwarz and Triangular Inequalities.

Lemma 1: Let V be an Euclidean space. Let v ∈ V, λ ∈ R. Then

‖λv‖ = |λ|‖v‖.

Proof: ‖λv‖= √〈λv, λv〉 = √λ2 〈v, v〉 =√λ2 √〈v, v〉 = |λ|‖v‖.

Lemma 2/Pythagoras/: Let V be an Euclidean space. Let v ∈ V such that v⊥ w. Then

‖v + w‖2 = ‖v‖2 + ‖w‖2 .

Proof: ‖v + w‖2 = 〈v + w, v + w〉

=〈v, v〉 + 2〈v, w〉 + 〈w, w〉.

=〈v, v〉 + 〈w, w〉, since 〈v, w〉=0.

28
 Linear Algebra II

=‖v‖2 + ‖w‖2 .

Theorem 2.5.1/ Cauchy-Schwarz and Inequalities/

Let V be an Euclidean space. Let v, w ∈ V. Then |〈v, w〉| ≤ ‖v‖‖w‖.

Proof:

Case 1: w=0.

Then both sides are equal to 0.

Case 2: ‖w‖=1.

v, w
Let λ = = 〈v, w〉.
w, w

We know by theorem 2.2.2 that (v-λw) ⊥ w.

Hence, (v-λw) ⊥ λw.

Thus, ‖v‖2 = ‖v − λw‖2 + ‖λw‖2

=‖v − λw‖2 + λ2 ; ‖w‖2 = 1.

Therefore, λ2 ≤ ‖v‖2

→ |λ| ≤ ‖v‖.

Hence, |〈v, w〉| ≤ ‖v‖‖w‖.

Case 3: w ≠0.
1
Let u= ‖w‖ w.

By case 2, |〈v, u〉| ≤ ‖v‖.

1
→ |〈v, ‖w‖ w〉| ≤ ‖v‖.

1
→ ‖w‖ |〈v, w〉| ≤ ‖v‖.

Hence, |〈v, w〉| ≤ ‖v‖‖w‖.

Theorem 2.5.2/Triangular Inequality/

Let V be an Euclidean space. Let v, w ∈ [Link] ‖v + w‖ ≤ ‖v‖ + ‖w‖.

29
 Linear Algebra II

Proof: ‖v + w‖2 = 〈v + w, v + w〉

=〈v, v〉 + 2〈v, w〉 + 〈w, w〉.

≤ ‖v‖2 + 2‖v‖‖w‖ + ‖w‖2

= (‖v‖ + ‖w‖)2 .

→ ‖v + w‖ ≤ ‖v‖ + ‖w‖.

2.6. The Dual Space

Let V be a vector space over a field K.

Let V* = L(V,K) be the set of all linear maps from V to K.

V* is a vector space over K.

Definition 2.6.1:

The vector space V* is called the dual space of V.

Elements of V* are called linear functional on V.

Notation: Let   V* and v  [Link] shall use the notation 〈  , v〉=  (v).

Note:

1) 〈  +  , v〉=〈  , v〉 + 〈  , v〉.
1 2 1 2
2) 〈  , v1 + v2 〉=〈  , v1 〉 + 〈  , v2 〉.
3) 〈λ  , v〉=λ〈  , v〉.
4) 〈  , λv〉=λ〈  , v〉.

Here, in the symbole 〈  , v〉 the two components donot belong to the same space.

Example-1:

1. If V = Rn and  : Rn → Rn , the projection on the ith component, i.e., we define  (x1

i
, x2 ,…, xn ) = xi . Then  ∈ V*.
2. Let V be an inner product space. Let v0 ∈ V. Then the linear mapping  :V→K, define by
 (v) = 〈v, v0 〉 is a linear functional. Hence,  ∈ V*.

Lemma 1:

30
 Linear Algebra II

Assume that V and W are vector spaces over a field K. Let {v1 , v2 , … , vn } be a basis of V. Let
w1 , w2 , … , wn be arbitrary elements of W. Then , there exists a unique linear map T:V→W such
that T(vi ) = wi for i=1,2,…,n.

Proof:

i) Existence

Let v ∈ V. Define T(v) = ∑ni=1 xi wi , where v = ∑ni=1 xi vi . Clearly, T is a linear map. T(vi ) = wi
for i=1,2,…,n.

ii) Uniqueness

Suppose T’: V→W is also a linear map such that T’(vi ) = wi for i=1,2,…,n.

Let v ∈ V. Then v = ∑ni=1 xi vi uniquely.

T’(v)= ∑ni=1 xi T′(vi ) = ∑ni=1 xi wi = T(v).

Therefore, T’=T.

Theorem 2.6.1

Let V be finite dimensional vector space over a field K. Then dim V* = dim V.

Proof

Let {v1 , v2 , … , vn }be a basis of V. By Lemma 1, for each i=1,2,…,n, there exists a linear
functional vi ∗ , such that

1, ifi = j
〈vi ∗ , vj 〉 =  / is called the Kronecker delta function/
0, i  j

𝐂𝐥𝐚𝐢𝐦: {v1 ∗ , v2 ∗ ,…, vn ∗ } is a basis of V*.

i) Let  ∈ V* and αi = 〈  , vi 〉.Then 〈∑nj=1 αj vj ∗ , vi 〉.

=αi 〈vi ∗ , vi 〉= αi =〈  , vi 〉.

Therefore,  = ∑nj=1 αj vj ∗ .Hence, v1 ∗ , v2 ∗ ,…, vn ∗ generate V*.

ii) Suppose ∑nj=1 βj vj ∗ =0.

Then 0=〈∑nj=1 βj vj ∗ , vi 〉 = βi 〈vi ∗ , vi 〉 = βi .

Therefore, v1 ∗ , v2 ∗ ,…, vn ∗ are linear independent.

31
 Linear Algebra II

Hence,{ v1 ∗ , v2 ∗ ,…, vn ∗ } is a basis of V*.

Definition 2.6.2

{ v1 ∗ , v2 ∗ ,…, vn ∗ } is called the dual basis of {v1 , v2 , … , vn }.Let V be an inner product space.
Then to each v ∈ V, we can associate linear functional Lv ∈ V* given by:

Lv (w) = 〈w, v〉 for each w ∈V.

To check that

i) Lv (w1 + w2 )= 〈w1 + w2 , v〉= 〈w1 , v〉 + 〈w2 , v〉.

= Lv (w1 ) + Lv (w2 ).
ii) Lv (λw)= 〈λw, v〉= λ 〈w, v〉= λLv (w).

𝐓𝐡𝐞𝐨𝐫𝐞𝐦 𝟐. 𝟔. 𝟐

Let V be a finite dimensional inner product space. Let  ∈ V*. Then there exists a unique v ∈V
such that  = Lv .

2.7. The Adjoint of Linear Operator

Theorem 2.7.1: Let V be a finite dimensional inner product space. Let T be a linear operator on
V. Then there exists a unique linear operator T* on V such that:

〈T(v), w〉 = 〈v, T ∗ (w)〉 for all v, w ∈ V.

Proof:

i) Existence
Let w ∈ V. Define  :V→K by  (v)= 〈T(v), w〉. Clearly  is a linear functional on
[Link] theorem 2.6.2, there exist a unique v’∈ V such that  = Lv ’.
Define T* : V→V by T*(w)= v’.
〈T(v), w〉=  (v)= Lv ‘(v) = 〈v, v ′ 〉 = 〈v, T ∗ (w)〉.
Now, we need only show that T* is linear.
∗
a) 〈v, T (w1 + w2 )〉 = 〈T(v), w1 + w2 〉.
= 〈T(v), w1 〉 + 〈T(v), w2 〉
= 〈v, T ∗ (w1 )〉 + 〈v, T ∗ (w2 )〉.
=〈v, T ∗ (w1 ) + T ∗ (w2 ) 〉
Therefore, T ∗ (w1 + w2 ) = T ∗ (w1 ) + T ∗ (w2 ).
b) 〈v, T ∗ (λw)〉 = 〈T(v), λw)〉 =λ̅ 〈v, T ∗ (w)〉 = 〈v, λT ∗ (w)〉.

Therefore, T ∗ (λw) =λT ∗ (w).

Uniqueness: Exercise.

32
 Linear Algebra II

Definition 2.7.1: T* is called the adjoint of T.

Theorem 2.7.2:

Let V be a finite-dimensional inner product space. Let λϵK and let T,S be linear operators on V.

Then :

1. (T + S)∗ = T ∗ + S ∗ .
2. (λT)∗ = λ̅ T ∗ .
3. (T  S)∗ = s ∗  T ∗ .
4. (T ∗ )∗ = T.
5. I ∗ = I.
6. 0∗ =0.
7. If T is invertible, then (T −1 )∗ = (T ∗ )−1 .

Proof: Let w∈V. Then for all v∈V:

i) 〈v, (T + S)∗ (w)〉 = 〈(T + S)(v), w〉=〈T(v), w〉 + 〈S(v), w〉

=〈v, T ∗ (w)〉 + 〈v, S ∗ (w)〉.
=〈v, T ∗ (w) + S ∗ (w)〉.
=〈v, (T ∗ + S ∗ )(w)〉.
Therefore, (T + S)∗ (w) = (T ∗ + S ∗ )(w).
Hence, (T + S)∗ = T ∗ + S ∗.
ii) 〈v, (λT)∗ (w)〉= 〈(λT)(v), w〉= 〈λT(v), w〉=λ〈T(v), w〉
= λ〈v, T ∗ (w)〉=〈v, λ̅T ∗ (w)〉=〈v, (λ̅T ∗ )(w)〉.
Therefore, (λT)∗ (w) =, (λ̅T ∗ )(w).
Hence, (λT)∗ = λ̅ T ∗ .
iii) 〈v, (T°S)∗ (w)〉 = 〈(T°S)(v), w〉 =〈(T(S((v)), w〉 = 〈S(v), T ∗ (w)〉
=〈v, S ∗ (T ∗ (w))〉.
=〈v, (S ∗ °T ∗ )(w)〉.
Therefore, (T°S)∗ (w)= (S ∗ °T ∗ )(w).
Hence, (T°S)∗= S ∗ °T ∗ .
(iv),(v),(vi) and (vii) left as an exercise.

2.8. Self- adjoint Linear Operators

2.8.1 Preliminary Concepts
A. Conjugate of matrix
̅ , is defined to be the
Definition 2.8.1: Let A be a matrix over C. Conjugate of A, denoted A
matrix obtained from A by replacing each entry by its conjugate.

33
 Linear Algebra II

Theorem 2.8.1 : Let A,B matrices over C. Let λ ∈ C. Then:

i) A̿ = A.
ii) ̅λA
̅̅̅ = λ̅ A
̅.
iii) ̅̅̅̅̅̅̅
A+B=A ̅+B ̅/ provided A and B have the same sizes/
iv) ̅̅̅̅
AB = A ̅B̅
v) (A̅)t = A̅t ./ provided A is a square matrix /

𝐍𝐨𝐭𝐚𝐭𝐢𝐨𝐧: A* = A̅t .

B. Hermitian and Skew Hermitian Matrices

Definition 2.8.2: Let A be a square matrix over C.A is called

i)
ii) Skew- Hermitian if A*=-A.

Note:

1) The diagonal elements of a hermitian matrix are real numbers.

2) The diagonal elements of a skew-symmetric matrix are either zeros or pure imaginaries.
C. Orthogonal Matrices
Definition 2.8.3: Let A be a square matrix over R. A is called orthogonal if At =A-1.

Theorem 2.8.2: Let A,B be orthogonal matrices of the same [Link],

i) At ,A-1 and AB are also orthogonal.

ii) Det A= ± 1.
D. Unitary Matrices
Definition 2.8.4: Let A be a square matrix over C. A is called unitary if A* =A-1.

Theorem 2.8.3: Suppose A, B are unitary matrices of the same size. Then, At ,A-1 and AB are
also unitary.

2.8.2 Self-adjoint Linear Operators

Definition 2.8.5: Let V= a finite dimensional inner product space. A linear operator Ton V is
called self-adjoint if T*=T.

Note:

i) If V is a Euclidean space and T is a self-adjoint linear operator on V, then T is called

symmetric.

34
 Linear Algebra II

ii) If V is a Unitary space and T is a self-adjoint linear operator on V, then T is called

hermitian.

Lemma 1: Let V= a finite dimensional inner product space.

T = a linear operator on V.

Then, 〈T(v), w〉 =0, for all v,w ∈ V. → T=0.

Proof: Choose w= T(v).

Then, 〈T(v), T(v)〉 =0 f for all v ∈ V.

→ T(v)=0,for all v ∈ V.

→ T=0.

Lemma 2: Let V= a finite dimensional unitary space.

T = a linear operator on V.

Then, 〈T(v), v〉 =0, for all v ∈ V. → T=0.

Proof:

〈T(v + w), v + w〉 =0, for all v,w ∈ V.

→ 〈T(v), w〉 + 〈T(w), v〉 =0, for all v,w ∈ V. ………………(a)

→ 〈T(v), iw〉 + 〈T(iw), v〉 =0, for all v,w ∈ V./replacing w by iw/

→ i̅〈T(v), w〉 + i〈T(w), v〉 =0, for all v,w ∈ V.

→ −〈T(v), w〉 + 〈T(w), v〉 =0, for all v,w ∈ V. ………………(b)

Adding(a) and (b),we get:

〈T(w), v〉 =0, for all v ∈ V.

Therefore, T=0 by Lemma 1.

Theorem 2.8.4: Let V= a finite dimensional inner product space.

T = a self-adjoint linear operator on V.

Then,

i) Each eigenvalue of T is real.

ii) Eigenvectors of T associated with distinct eigenvalues are orthogonal.

35
 Linear Algebra II

Proof:

i) Suppose λ is eigenvalue of T, i.e., T(v) = λv for some v ∈ V,v ≠ 0. Then

λ〈v, v〉 =〈λv, v〉 = 〈T(v), v〉 =〈v, T(v)〉 =〈v, λv〉 =λ̅ 〈v, v〉 .
→ λ〈v, v〉 − λ̅ 〈v, v〉 = 0.
→ (λ − λ̅)〈v, v〉=0.
→ (λ − λ̅)=0.
→ λ = λ̅.
→ λ is real.
ii) Suppose λ1 , λ2 are distinct eigenvalues of T and let v1 , v2 ∈ V be eigenvectors of T
associate with λ1 , λ2 respectively.
Then, λ1 〈v1 , v2 〉 =〈λ1 v1 , v2 〉 =〈T(v1 ), v2 〉 =〈v1 , T(v2 )〉 =〈v1 , λ2 v2 〉
=λ2 〈v1 , v2 〉 .
→ (λ1 − λ2 )〈v1 , v2 〉=0.
→ 〈v1 , v2 〉=0, since λ1 ≠ λ2 .
→ v1 ⊥ v2 .

Theorem 2.8.5: Let V= a finite dimensional inner product space.

Let T be a linear operator on V. Let β= {v1 , v2 ,…, vn } be an orthonormal basis of V.

Let A=[T]β = The matrix of T with respect to β.

Then, [T ∗ ]β = A*.

Proof: Left as an exercise.

Example: Let T:R3 → R3 be defined by:

T(x,y,z)= (x+2y,3x-4z,y).

Clearly, T is a linear operator on R3 .

Find T*(x,y,z).

Solution:

T(1,0,0)=(1,3,0)

T(0,1,0)=(2,0,1)

T(0,0,1)=(0,-4,0).

1 2 0
Therefore, A=[T]= [3 0 −4].
0 1 0

36
 Linear Algebra II

1 3 0
Therefore, A*=[T*]= [2 0 1].
0 −4 0

Therefore, T*(x,y,z)= (x+3y,2x+z,-4y).

2.9. Isometry
Theorem 2.9.1: Let V= a finite dimensional inner product space.

Let T be a linear operator on V. Then, the following statements are equivalent.

i) T*=T-1./T*T=I/
ii) T preserves inner products, i.e., 〈T(v), T(w)〉 = 〈v, w〉 ;for all v,w ∈ V.
iii) T preserves length,i.e.,‖T(v)‖ = ‖v‖ ;for all v ∈ V.

Proof:

(i)→(ii): 〈T(v), T(w)〉 = 〈v, T ∗ T(w)〉 = 〈v, w〉 .

(ii)→(iii): ‖T(v)‖=√〈T(v), T(v)〉 = √〈v, v〉 = ‖v‖.

(iii)→(i):

Claim: T is invertible.

T(v)=0. → ‖T(v)‖=0.

→ ‖v‖=0.

→ v=0.

Therefore, T is one-to-one. Since V is finite dimensional, then T is invertible.

Now, 〈T ∗ T(v), v〉 = 〈T(v), T(v)〉 = 〈v, v〉 = 〈I(v), v〉.

→ 〈(T ∗ T − I)(v), v〉 = 0.

→ T ∗ T − I = 0.

→ T ∗ T = I.
→ T ∗ = T −1.

𝐃𝐞𝐟𝐢𝐧𝐢𝐭𝐢𝐨𝐧 2.9.1: Let V be a finite dimensional inner product space. Let T be a linear operator
on V. T is called an isometry if it satisfies any one / and hence all/ of the three equivalent
conditions stated in theorem 2.9.1.

Note:

37
 Linear Algebra II

i) An isometry on a Euclidean space is called an orthogonal operator.

ii) An isometry on a Unitary space is called an Unitary operator.
iii) Let T be an isometry and let {v1 , v2 ,…, vn } be an orthonormal basis of [Link]:
a) ‖T(vi )‖ =1.
b) 〈T(vi ), T(vi )〉 = δij .
Hence , {T(v1 ), T(v2 ),…, T(vn )} is also an orthonormal basis of V.
iv) Because an isometry measures the inner product, it measures orthogonality and all other
metric notions.

Theorem 2.9.2: Let V= a finite dimensional inner product space.

Let T be a linear operator on V. Let A be the matrix of T relative to an orthonormal basis.

Then, T is an isometry if and only if A* =A-1.

Proof: T is an isometry ↔ T* =T-1.

↔ [ T*] =[T-1].

↔ [ T*] =[T]-1.

↔ A*=A-1.

2.10. Normal Operators and the Spectral Theorem

Definition 2.10.1: Let V= a finite dimensional inner product space.

Let T be a linear operator on V.

T is called a normal operator if TT*=T*T.

Lemma 1: Let V= a finite dimensional inner product space.

Let T be a normal operator on V. Then,

‖T(v)‖=‖T ∗ (v)‖ for all v∈ V.

𝐏𝐫𝐨𝐨𝐟: 〈T(v), T(v)〉=〈v, T ∗ T(v)〉 =〈v, TT ∗ (v)〉 = 〈T ∗ (v), T ∗ (v)〉 .

Therefore, ‖T(v)‖=‖T ∗ (v)‖.

Lemma 2: Let V= a finite dimensional inner product space.

Let T be a normal operator on V. Then, for λ ∈ K, T- λI is also normal operator.

Proof: (T- λI)* =T*- λ̅I.

(T- λI) (T- λI)* = (T- λI)(T*- λ̅I).

38
 Linear Algebra II

=TT* - λ̅ T - λT* + λλ̅ I.

(T- λI)*(T- λI) = (T*- λ̅I) (T- λI).

=T*T - λ T* - λ̅T + λ̅λ I.

= TT* - λ̅ T - λT* + λλ̅ I.

Therefore, (T- λI) (T- λI)* = (T- λI)* (T- λI).

Hence, (T- λI) is normal.

Theorem 2.10.1: Let V= a finite dimensional inner product space.

Let T be a normal operator on V. Then v ∈ V/v≠0/ is an eigenvector of T with eigenvalue λ if

and only if v is an eigenvector of T* with eigenvalue λ̅ .

Proof: T(v)= λv ↔ (T- λI)(v) = 0.

↔ (T*- λ̅I)(v) = 0, by lemma 1, since T- λI is normal.

↔ T*(v) = λ̅v.

Lemma 3: Let V be a finite dimensional Euclidean space. Let T be a symmetric / self-adjoint/

operator on V. Then

a) 𝒳T is a product of linear polynomials over R.

b) T has non-zero eigenvector.

Proof:

a) By the fundamental Theorem of Arithmetic,

𝒳T (λ)= (x-λ1 ) (x-λ2 )… (x-λn ) over C. Since T is self-adjoint each eigenvalues of T is real.
Therefore, the λi ’s are all real.

→ 𝒳T (λ) is a product of linear polynomials over R.

b) By (a), T has at least one real eigenvalues.
Hence, T has non-zero eigenvector.

2.11. Factorization of a matrix / LU, Cholesky, QR/

Definition: Matrix factorization is the process of factorize matrix as a product of lower and
upper triangular matrices and upper triangular and transpose factor of matrix A by different
methods.

39
 Linear Algebra II

The main purpose of matrix factorization in this section is

✓ To obtain some form of lower ran (simplified) approximation to matrix A.

✓ For understanding the structure of the data matrix that is how the object relates to the
attributes.
✓ To solving linear systems of equation.
✓ To compute rank and estimation etc.

Note: Matrix factorization is not unique but it has many factors in different methods of
factorization.

2.11.1 LU- factorization

In this method the coefficient matrix A of the system is decomposed into product of a lower
triangular matrix L and an upper triangular matrix U.

That is : A=LU.

l11 0 ⋯ 0 u11 u12 ⋯ u1n

l l22 ⋯ 0 0 u22 ⋯ u2
Let L = [ 21 ] and U = [ ].
⋮ ⋮ ⋱ ⋮ ⋮ ⋮ ⋱ ⋮
ln1 l2n … lnn 0 0 … unn

l11 u11 l11 u12 ⋯ l11 u1n

l u l21 u12 + l22 u22 ⋯ l21 u1n
→ A =LU= [ 21 11 ].
⋮ ⋮ ⋱ ⋮
ln1 u11 ⋯ … ⋱

Definition: Let A be square matrix if there is a lower triangular matrix with all diagonal
elements are equal to 1/lii =1/ and upper triangular matrix such that A=LU then we say this is
LU- decomposition is Doolitteles Method.

Definition: Let A be square matrix if there is an upper triangular matrix with all diagonal
elements are equal to 1/uii =1/ and lower triangular matrix such that A=LU then we say this is
LU- decomposition is Crout’s Method.

Note:-LU-decomposition develops the breakdown of coefficient matrix into a product of two

matrices.A=LU where L=lower triangular matrix and U=upper triangular matrix.

Example-1: Compute the LU- decomposition of matrix A.

1 2 4
A=(3 8 14) where L=lower triangular matrix U33 and U=upper triangular matrix.
2 6 13

40
 Linear Algebra II

1 0 0 U11 U12 U13

Solution:-A=LU where L=(L21 1 0)and U=( 0 U22 U23 )
L31 L32 1 0 0 U33

1 2 4 1 0 0 U11 U12 U13

A=LU(3 8 14)=(L21 1 0) ( 0 U22 U23 )
2 6 13 L31 L32 1 0 0 U33

U11 =1,U12 =2,U13 =4.

Now take second raw that is

L21 U11= L21 × 1 = 3L21 = 3

L21 U12 + U22 =83 × 2 + U22 = 8U22 = 2

L21 U13 + U23 =143 × 4 + U23 = 14U23 = 2

L31 U11 = 2L31 × 1 = 2L31 = 2

L31 U12 + L32 U23 = 62 × 2 + L32 × 2 = 6L32 = 1

L31 U13+L32 U23 +U33 =13(2×4) + (1×2) + U33 =13U33 =3

1 2 4 1 0 0 1 2 4
Therefore A=LU=(3 8 14)=(3 1 0) (0 2 2) is the result of LU-decomposition.
2 6 13 2 1 1 0 0 3
Definition:-LU-decomposition is used when the matrix is invertible and all its leading sub
matrices have non-zero determinant. Otherwise it does not work.

1 2 4
Example-2: A=(3 8 14)
2 6 13
1 2 4
1 2
Solution: A1 = 1, A2 = | |,A |3 8 14| and the matrix is invertible anda11= |A1 | = 1
3 8 3=
2 6 13
,|A2 | =(1×2) − (2 × 3) = 2

8 14 3 14 3 8
And |A3 |=| | -2| | +4| | =20-(2×11) + (4×2) =6
6 13 2 13 2 6
Therefore all determinants are non-zero then a matrix A has LU- decomposition.

Example-3: show that matrix A has no LU- decomposition?

41
 Linear Algebra II

1 2 3
A=(2 4 5)
1 3 4
Solution:- The leading sub matrices are

1 2 3
1 2
A1 =1, A2 =( ) , A3 =(2 4 5)
2 4
1 3 4
|A1 |=1,|A2 |=(1×4) - (2×2)=0

2 4
|A3 |=|4 5 2
|-2|
5
|+3| |=1+6+6=13
3 4 1 4 1 3
Therefore the matrix A has no LU-decomposition because determinant of one sub matrices A2 is
0 by the fact we have above it has no LU- decomposition.

Note: By reordering row of invertible sub matrices we can make it has non-zero determinant and
also has LU-decomposition.

2.11.2 Cholesky factorization

Definition: Let matrix A is n×n symmetric and positive definite, then A can be decomposed as

A=LLT where L= ( lij ) , lij =0,i<j .

Or

A=UU T where U= ( uij ) , uij =0,i<j .

Is called Cholesky decomposition. Where L= lower triangular matrix and LT =transpose of matrix
L and U=upper triangular matrix and U T =transpose of matrix U.

Note: A matrix A must be positive definite to be has cholesky decomposition only symmetric is
not enough.

Example-1: Decompose the matrix by cholesky decomposition method.

1 2 3
A=(2 8 22)
3 22 82

Solution: From the fact that A=LLT

l11 0 0 l11 l21 l31

l
L= ( 21 l22 0 ) and LT = ( 0 l22 l32 )
l31 l32 l33 0 0 l33

42
 Linear Algebra II

l11 2 l11 l21 l11 l31

A=LLT = (l21 l11 l21 2 + l22 2 l21 l31 + l22 l32 ).
l31 l11 l31 l21 + l32 l22 l31 2 + l32 2 + l33 2

i) l11 = 1,
l21 l11 = 2 → l21 =2
l31 l11 = 2 → l31 =3.
ii) l21 2 + l22 2 = 8.

→ l22 = √8 − l21 2 = √8 − 22 = √8 − 4 = √4 = 2.
l21 l31 + l22 l32 =22.
2 ∗ 3 + 2*l32 = 22.
22−6
→ l32 = = 8.
2
iii) l31 2 + l32 2 + l33 2 = 82.

l33 = √82 − l32 2 − l33 2 = √82 − 82 − 32 = √82 − 64 − 9 = 3.

1 0 0 1 2 3
T
Therefore, L= (2 2 0) and L = (0 2 8).
3 8 3 0 0 3

Example-2: Decompose the matrix by cholesky decomposition method.

25 15 −5
A=( 15 18 0 )
−5 0 11

Solution: From the fact that A=LLT

• First column of R
25 15 −5 5 0 0 5 3 −1
A = RRT = ( 15 18 0 ) = ( 3 R 22 0 ) (0 R 22 R 32 )
−5 0 11 −1 R 32 R 33 0 0 R 33

• Second column of R

18 0 3 R 0 R R 32
=( ) − ( ) (3 −1) = ( 22 ) ( 22 )
0 11 −1 R 32 R 33 0 R 33

9 3 3 0 3 1
=( )=( )( )
3 10 1 R 33 0 R 33

• Third column of R

43
 Linear Algebra II

1+ R 33 2 =10

R 33 2 = 10 − 1=9.

R 33 = 3.
25 15 −5 5 0 0 5 3 −1
Then the conclusion is A = RRT ( 15 18 0 ) = ( 3 3 0 ) (0 3 1 ).
−5 0 11 −1 1 3 0 0 3

2.11.2 QR- factorization

The QR decomposition (also called the QR factorization) of a matrix is a decomposition

of the matrix into an orthogonal matrix and a triangular matrix. A QR decomposition of

a real square matrix A is a decomposition of A as:

A = QR,

where Q is an orthogonal matrix (i.e. QTQ = I) and R is an upper triangular matrix.

If A is nonsingular, then this factorization is unique.

There are several methods for actually computing the QR decomposition. One of such method is
the Gram-Schmidt process.

Note: Decompose matrix by QR- decomposition method with Gram- Schmidt process has
difficult if matrix is linearly dependent columns has.

Let’s consider a non-singular n*n matrix A/A= [ x1 x2 … xn ].

Processes:

To determine Q:

. Use Gram- Schmidt orthogonalization processes.

To determine R:

We can find scalar αij ,1≤j≤I such that :

xi = α1i u1 + α2i u2 +…+αii ui .

= [(α1i , α2i ,…αii ,0,0,…,0)t].

Coordinates of xi :

44
 Linear Algebra II

α1i
α11 α12
⋮
0 α22
αii
x1 = 0 , x2 = 0 ,…, xi = .
⋮ ⋮ 0
[ 0 ] [ 0 ] ⋮
[0]

α11 α12 ⋯ α1n

0 α22 ⋯ α2n
R= [ ].
⋮ ⋮ ⋱ ⋮
0 0 … αnn

Q= [u1 u2 … un ].

Example-1: Compute the QR-factorization of matrix A.

1 0 0
A = [1 1 0].
1 0 1
Solution: A= [ x1 x2 x3 ].This vectors are linear independent. And apply Gram-Schmidt
orthogonalization process:
x (1,1,1) (1,1,1) 1 1 1
i) Set u1 = ‖v1 ‖ = ‖(1,1,1)‖ = =( , , ).
1 √3 √3 √3 √3
ii) Next, set w2 = x2 - 〈x2 , u1 〉u1
1 1 1 1 1 1
= (0,1,1) - 〈(0,1,1), ( , , )〉 ( , , ).
√3 √3 √3 √3 √3 √3
2 1 1 1
= (0,1,1) – ( , , )
√3 √3 √3 √3
2 2 2 −2 1 1
=(0,1,1) – (3 , 3 , 3) = ( 3 , 3 , 3).

−2 1 1 −2 1 1
w (, , ) ( , , ) −2 1 1
and u2 = ‖w2‖ = 3 33
−2 1 1 = 3 33
=( , , ).
2 ‖( , , )‖ √
2 √6 √6 √6
3 33
3

iii) Next, set w3 = x3 - 〈x3 , u1 〉u1 −〈x3 , u2 〉u2

1 1 1 1 1 1 −2 1 1 −2 1 1
=(0,0,1)- 〈(0,0,1), ( , , )〉 ( , , )-〈(0,0,1), ( , , )〉 ( , , )
√3 √3 √3 √3 √3 √3 √6 √6 √6 √6 √6 √6
1 1 1 1 1 −2 1 1
=(0,0,1)- ( , , )- ( , , ).
√3 √3 √3 √3 √6 √6 √6 √6
1 1 1 −2 1 1
=(0,0,1)- (3 , 3 , 3)-( 6 , 6 , 6).
−1 1
= (0, , 2).
2

−1 1 −1 1
w3 (0,
, ) (0, , ) −1 1
2 2 2 2
and u3 = ‖w3 ‖
= −1 1 = = (0, , ).
‖(0, , )‖ √
1 √2 √2
2 2
2

45
 Linear Algebra II

By Lemma , w3 /and hence u3 / is orthogonal to u1 and u2 .Then {u1 , u2 , u3 } is orthonormal. The

1 1 1 −2 1 1 −1 1
required normal basis of R3 is:{ ( , , ),( , , ) , (0, , )}
√3 √3 √3 √6 √6 √6 √2 √2

1 −2
0
√3 √6 α11 α12 α13
1 1 −1
Q=[u1 u2 u3 ] = √3 √6 √2
. And R= [ 0 α22 α23 ].
1 1 1 0 0 α33
[√3 √6 √2]

From x1 = α11 u1
1 1 1
→(1,1,1) =α11 ( , , ).
√3 √3 √3

→ α11 = √3 .

x2 = α12 u1 + α22 u2 .
1 1 1 −2 1 1
→(0,1,1)= α12 ( , , ) + α22 ( , , ).
√3 √3 √3 √6 √6 √6

2√3 √6
α12 = and α22 = .
3 3

And

x3 = α13 u1 + α23 u2 + α33 u3 .

1 1 1 −2 1 1 −1 1
→(0,0,1)= α13 ( , , ) + α23 ( , , ) + α33 (0, , ).
√3 √3 √3 √6 √6 √6 √2 √2

√3 √6 √2
α13 = , α23 = and α33 = .
3 6 2

2√3 √3
√3 3 3
√6 √6
Therefore, R = 0 .
3 6
√2
[ 0 0 2]

→A= QR.

Example-2: Compute the QR-factorization of matrix A

−1 3
A=( )
1 5
Solution: Let as use Gram-Schmidt process with columns

46
 Linear Algebra II

u1= a1=[−1]
1

u1= u1 1 −1
= [ ]
‖u1 ‖ √2 1

u2= a2−proju1 a2= a2− 〈u1 ,a2 〉 u1

3 1 1 −1 3 −1 4
=[ ] − ( (−3 + 5)) [ ] = [ ] − [ ] = [ ]
5 √2 √2 1 5 1 4

u2 = u2
=
1 4 1 1
[ ]= [ ]
‖u2 ‖ 4√2 4 √2 1

−1 1
1 −1 1
Therefore the matrix Q is Q=[√2
1
√2
1 ]=√2 [ 1 ] this produce by Gram schmdit process then
1
√2 √2
multiplies by elementary matrices on right as

−1 3
A=[ ]
1 5
−1
1
0 3
A[√2 ] = [√2
1 ]
0 1 5
√2

−1
1
0 1 4
A[√2 ][ −√2] = [√2 ]
1
0 1 0 1
√2
4

−1 1
1 1 0
0 1 −√2] [
A[√2 ][ 0
1 ] = [√2
1
√2
1] =Q
0 1 0 1 4√2
√2 √2

From this we obtain

1 −1
1 0 0
A=Q [0 4 ] [1 √2] [√2 ]
√2 0 1 0 1
1 0
=Q[0 ] [1 √2] [√2 0]
4
√2 0 1 0 1

1 1
=Q[√2 √2 ]=QR where R =[√2 √2 ]=2[ ].
0 4√2 0 4√2 0 4

−1 3 1 −1 1 1 1
Now QR-factorization is : A= [ ]= [ ] √2 [ ].
5 5 √2 1 1 0 4

47
 Linear Algebra II

2.11.4 Singular Value Decomposition

Definition: Let A be an m × n matrix with m ≥ n. There are orthogonal matrices U /m*m/ and
V/n*n /such that where Σ= diag( σ 1, …,σ n), σ 1 ≥σ 2 ≥ … ≥σ n ≥ [Link] decomposition is
called the singular value decomposition of A. If U A consists of the first n columns of U, we can
write

A = UA Σ VT; / UUT = I, VVT=I/

which is sometimes called the singular value factorization of A. The diagonal elements of σ are
called the singular values of A. The corresponding columns of U and V are called left and right
singular vectors of A.

From the above equation it follows that:

ATA = VΣ2VT.

Note:

. Σ is an m*n matrix where the (i,j)th entry σ i ,0,…,i= 1… min (m,n) and other entries are zero.

. The singular value of A/ σ/ is the square roots of the eigenvalues of ATA.

. The columns of U are the normalized eigenvectors of AAT.

. The columns of V are the normalized eigenvectors of ATA.

Example-1: Find the SVD for

1 0
A= [ ].
0 4
Solution:

A = U Σ VT, where the columns of V are the eigenvectors of ATA and the columns of U
are the eigenvectors of AAT.

1 0
First we form: A TA = [ ], and identify the eigenvalues as 𝛌𝐢 =16,[Link]
0 16
corresponding normalized eigenvectors are the columns of V:

0 1
V=[ ].
1 0

The singular values of A are σ1 = √16 =4, σ2 = √1 =1.

4 0
Thus, Σ=[ ].
0 1

48
 Linear Algebra II

1 0
Next we form: AAT = [ ], and as this matrix identical to ATA, we can construct U=V
0 16
without much work at all.

The SVD of A is complete.

0 1 4 0 0 1
A = U Σ VT = [ ][ ][ ].
1 0 0 1 1 0
3 2 2
Example-2: Find the SVD of A, UΣV T , where A = [ ] .
2 3 −2

First we compute the singular values σi by finding the eigenvalues of AAT .

17 8
AAT = [ ]
8 17

The characteristic polynomial is det (AAT − λI) = λ2 − 34λ + 225 = (λ − 25)(λ − 9), so the
singular values are σ1 = √25 = 5 and σ2 = √9 = 3. Now we find the right singular vectors (the
columns of V) by finding an orthonormal set of eigenvectors of ATA. It is also possible to
proceed by finding the left singular vectors (columns of U) instead. The eigenvalues of are 25, 9,
and 0, and since AT A is symmetric we know that the eigenvectors will be orthogonal.

−12 12 2
T
For λ = 25, we have: A A−25I= [ 12 −12 −2 ]
2 −2 −17
1 −1 0
Which row -reduces to [0 0 1] . A unit-length vector in the kernel of that matrix is
0 0 0
1/√2
u1 = (1/√2).
0
1
4 12 2 1 0 4
For λ = 9 we have AT A−9I = [12 4 −2] which row reduces to [1 0 4
1 ].
2 −2 −1
0 0 0

1⁄√18
A unit-length vector in the kernel is u2 = (−1⁄√18).
4⁄√18

For the last eigen vector, we could compute the kernel of AT A or find a unit vector perpendicular
a
to u1 and u2 . To be perpendicular to u1 = ( b ) we need −a = b.
c

49
 Linear Algebra II

Then the condition that uT2 u3 = 0 becomes 2a⁄√18 + 4a⁄√18 = 0 or − a = 2c. So

a 2⁄3
u3 = ( −a ) and for it to be unit-length we need a = 2⁄3 so u3 = (−2⁄3).
−a⁄2 −1⁄3

So at this point we know that

1 1
0
√2 √2
5 0 0 1 −1 4
A = UΣV T = U ( ) .
0 3 0 √18 √18 √18
2 −2 −1
[ √3 3 3 ]
1
Finally, we can compute U by the formula σui = Aui , or ui = σ Aui . This gives

1⁄√2 1⁄√2
U=( ).
1⁄√2 1⁄√2

So in its full glory the SVD is:

1 1
0
√2 √2
1⁄√2 1⁄√2 5 0 0 1 −1 4
A = UΣV T = ( )( ) .
1⁄√2 − 1⁄√2 0 3 0 √18 √18 √18
2 −2 −1
[ √3 3 3 ]

50
 Linear Algebra II

CHAPTER THREE

CANONICAL FORMS

3.1. Elementary row and column operation on Matrices

Let F be a field.

F[x] = the polynomial domain/field.

We wish to study matrices over F[x]. Consider elementary operations in such matrices, that is

i) Interchanging two rows(or columns).

ii) Multiplying a row(or column) by non-zero constant.
iii) Adding the ith row(or ith column) the product of the jth row(jth column) by f(x), where j≠
i and f(x)∈ F[x].
Each of the operation is called F[x]-elementary operation.

Definition 3.1.1: A matrix obtained from the identity matrix I by applying an F[x]-elementary
operation is called F[x]-elementary matrix.

Note:

Every F[x]- elementary matrix is non-singular and its inverse is also F[x]- elementary matrix.

1 0
Example: A matrix A= [ 2 ] is an F[x]- elementary matrix. Then A is non-singular.
x 1
1 0
i.e.,|A|=1≠0 and its inverse is A−1 = [ 2 ] and also A−1 is an F[x]- elementary matrix. We
−x 1
can obtain by multiply identity

matrix I of row one by −x 2 and adding on row two.

3.2. Equivalence of Matrices of polynomials

Definition 3.2.1: Let A and B be matrices over F[x]. If B is obtained from A by performing any
succession of F[x]-elementary operations, A is said to be F[x]-equivalent to B.

Notation: A F[x] B.

Note: A F[x] B if and only if B= PAQ, where P, Q are products of F[x]- elementary matrices.

Example-1: Show that the following matrices are F[x]-equivalent.

x x+1 1 0
a) A[x]= [ ] and B[x] = [ ].
x2 − x x2 − 1 0 0

51
 Linear Algebra II

2
b) A[x] = [ x x + 1] , B[x] = [1 0
].
x−1 x 2 0 x − x2 + 1
4

0 1 x 1 0 0
c) A[x]= [ x x 1 ] ,B[x]= [0 1 0]
x2 − x x2 − 1 x2 − 1 0 0 0
Solution:

a)
i) Add –(x-1) times the first row to the second row to obtain :

1 0 x x+1
[ ]A[x] = [ ].
1−x 1 0 0
ii) Add -1 times the first column to the second column to obtain :
x x + 1 1 −1 x 1
[ ][ ]= [ ].
0 0 0 1 0 0
iii) Add –x times the second column to the first column to obtain :
x 1 1 0 0 1
[ ][ ]=[ ].
0 0 −x 1 0 0
iv) Interchange columns to obtain :
0 1 0 1 1 0
[ ][ ]=[ ] =B[x].
0 0 1 0 0 0
1 0
Thus, P(x) = [ ].
1−x 1
1 −1 1 0 0 1 −1 x + 1
Q(x)= [ ][ ][ ]=[ ].
0 1 −x 1 1 0 1 −x
(b) and (c) are exercises.

The Kth- Order Minor

Let A be an m*n matrix over F[x]. Let k≤ min{m,n}.Choose arbitrary k rows and k columns of
A. The elements at the intersections of these rows and columns constitute a square matrix of
order k. The determinant of this matrix is called a the Kth- Order Minor of A.

There are (m
k
)×(nk) k-order minors of A.

1 2x 3
Example-2: Let A= [0 4 5x]. Find all the 2nd- order minors of A.
x 2 1

Solution:

First- order minor

52
 Linear Algebra II

|1|,|2x|,|3|,|0|,|4|,|5x|,|x|,|2|,|1| = 1,2x,3,0,4,5x,x,2,1.

Second-order minor
Rows Columns
1,2 1,3 2,3
1 2x 1 3 2x 3
| |=4 | | = 5x | | = 10x 2 − 12
1,2 0 4 0 5x 4 5x
1 2x 1 3 2x 3
| | = 2 − 2x 2 | | = 1 − 3x | | = 2x − 6
1,3 x 2 x 1 2 1
0 4 0 5x 4 5x
| |= -4x | | = −5x 2 | |= 4-10x
2,3 x 2 x 1 2 1

Third-order minor
1 2x 3
4 5x 2x 3
|0 4 5x|=1 | |+x| |
2 1 4 5x
x 2 1
= 4-10x + x[10x 2 − 12]= 10x 3 -22x +4.

And dk(x):

d1(x)= gcd{1,2x,3,4,5x,x,2,1}=1.

d2(x)=gcd{4,5x,10x 2 − 12,2-2x 2 , 1 − 3x, 2x − 6, −4x, −5x, 4 − 10x}=1.

d3(x) = 10x 3 -22x +4.

Therefore Rank of matrix A:d3(x)≠0.

Rank=3.

Definition 3.2.3: Let A be a non-zero m*n matrix over F[x]. A is said to be of rank k if k is the
largest integer such that not all kth- order minors of A are identically zero.

3.3. The Smith Canonical Form and Invariant Factors

Let A and B be m*n matrices over F[x].

Lemma-1: Suppose B=PA, where P is a product of F[x]- elementary matrices. Then, every Kth –
order minor of B is a linear combination, over F[x], of Kth –order minor of A.

Proof: Left as an exercise.

To help you understand the Lemma-1, consider the matrix

1 2x 3
A= [0 4 5x] from the above example-2.
x 2 1

53
 Linear Algebra II

Case-1: Interchanging row 2 with row [Link]

1 0 0 1 2x 3
P= A= [0 0 1] and PA = A= [ x 2 1 ]=B.
0 1 0 0 4 5x
All the second order minors of B are

Columns
Rows 1,2 1,3 2,3
1 2x 1 3 2x 3
| |= 2 − 2x 2 | | = 1 − 3x | | = 2x − 6
1,2 x 2 x 1 2 1
1 2x 1 3 2x 3
| |=4 | | = 5x | | = 10x 2 − 12
1,3 0 4 0 5x 4 5x
x 2 x 1 2 1
| |= 4x | | = 5x 2 | |= 10x-4
2,3 0 4 0 5x 4 5x

Thus, the Kth –order minor of A=N= the Kth –order minor of B=M. Or N=-M or N= another
second order minor except for sign.

Case-2: Row 2 multiplied by α.Then

1 0 0 1 2x 3
P= [0 α 0] and PA = [0 4α 5αx] = B.
0 0 1 x 2 1
All the second minors of B are:

Columns
Rows 1,2 1,3 2,3
1 2x 1 3 2x 3
| |=4 α | | = 5αx | | = 10αx 2 − 12α
1,2 0 4α 0 5xα 4α 5αx
1 2x 1 3 2x 3
| | = 2 − 2x 2 | | = 1 − 3x | | = 2x − 6
1,3 x 2 x 1 2 1
0 4α 0 5αx 4α 5αx
| |= -4 α x | | = −5αx 2 | |= 4 α -10 α x
2,3 x 2 x 1 2 1

Thus, N=M or N= α M

Case-3: x times row1 added to row [Link]

1 0 0 1 2x 3
P= [0 1 0] and PA = [ 0 4 5x ] = B.
x 0 1 2x 2x 2 + 2 3x + 1

54
 Linear Algebra II

Columns
Rows 1,2 1,3 2,3
1 2x 1 3 2x3
| |=4 | | = 5x | | = 10x 2 − 12
1,2 0 4 0 5x 45x
1 2x 1 3 2x 3
| | = 2 − 2x 2 | | = 1 − 3x | | = 2x − 6
1,3 x 2 x 1 2 1
0 4 0 5x 4 5x
| 2 |= -8x | | = −10x 2 | 2 |= -10x 3 + 2x + 4
2,3 2x 2x + 2 2x 3x + 1 2x + 2 3x + 1
Thus, N=M or N=M-XL where L=(1,2),(2,3).

Theorem 3.3.1: Let A,B be m*n matrices over F[x].

A F[x] B → Every Kth –order minor of B is a linear combination, over F[x], of Kth –order minor

of A.

Proof: Done in Lemma-1.

Corollary -1: Let A,B be m*n matrices over F[x]. Then A F[x] B→ rank A=rank B.

Proof: Let r = rank B.

Let N be a non-zero rth –order minor of B.

By Theorem 3.3.1, N is a linear combination, over F[x], rth –order minors of A.

→ There exists a non-zero rth –order minors M of A.

→ rank B ≤ rank A.

Since B F[x] A , we also conclude that rank A ≤ rank B.

Therefore, rank A=rank B.

Corollary -2: Let A F[x] B. Let dk (x) be the gcd of all kth –order minors of A. Then, dk (x) is

also the gcd of all kth –order minors of B.

Proof: Let d′ k (x) be the gcd of all kth –order minors of B. Theorem 3.3.1, dk (x) ∖ d′ k (x).

Since also, B F[x] A , again by Theorem 3.3.1, d′ k (x) ∖ dk (x) .So, both dk (x) and d′ k (x) are

monic, it follows that d′ k (x) = dk (x) .

Lemma-2: Let A be a non-zero m*n matrix over F[x].Then, A is F[x]-equivalent to the m*n
matrix, over F[x], of the form:

55
 Linear Algebra II

f (x) 0
[1 ] where f1 (x) is monic polynomial of minimal degree among all non-zero elements of
0 A1
all matrices F[x]- equivalent to A.

Proof: Left as an exercise.

Theorem-3.3.2: Let A= a non-zero m*n matrix of rank r over F[x].Then, A is F[x]- equivalent to
a matrix B of one of the following types:

D D 0
D, [D 0],[ ],[ ] ,where
0 0 0
i) D= diag[ f1 (x), f2 (x),…, fr (x)].
ii) fi (x)’s are monic polynomials such that fi (x)/ fi+1 (x) for i= 1,2,…,r-1.

Proof: Left as an exercise.

Theorem 3.3.4: Let A, B be as in theorem 3.3.2. The polynomials f1 (x), f2 (x),…, fr (x) are
uniquely determine by the matrix A.

Proof: Suppose g1 (x), g 2 (x),…, g r (x) also satisfy the conditions of theorem 3.3.2.

By Theorem 3.3.3, dk (x) =f1 (x) f2 (x)… fk (x)= g1 (x) g 2 (x)… g k (x). In particular ,

d1 (x) =f1 (x) = g1 (x).

dk (x)
Then, d = fk (x)= g k (x) for k=2,3,….
k−1 (x)

Therefore, f1 (x), f2 (x),…, fr (x) are uniquely determined.

Definition 3.3.1: Let A,B be as in Theorem 3.3.2.

1. The polynomials f1 (x), f2 (x),…, fr (x) are called the invariant factors of A.
2. The matrix B is called the Smith Canonical form of A.

Corollary-3: Let A1 , A2 be m*n matrices over F[x]. A1 F[x] A2 , if and only if A1 , A2 have

the same invariant factors.

Proof: (→): Suppose A1 F[x] A2 .

Let B1 , B2 be the smith canonical forms of A1 , A2 respectively. By transitivity of F[x] ,

A1 F[x] B2 .Uniqueness of the smith canonical form, B1 = B2 .

56
 Linear Algebra II

Therefore, A1 and A2 have the same invariant factors.

(←): Suppose A1 , A2 have the same invariant [Link] they are F[x]-equivalent to the same

smith canonical form B, hence F[x]-equivalent to each other.

x 1 0
Example-1: Let A= [0 x 1 ]. Find the Smith canonical form of A.
2 3 x−1
Solution:

d1(x)= gcd{x,1,2,3,x-1}

d2(x)=gcd{
x 1 x 0 1 0 x 1 x 0 1 0 0 x 0 1 x 1
| |,| |,| |,| |,| |,| |,| |,| |,| |}
0 x 0 1 x 1 2 3 2 x−1 3 x − 1 2 3 2 x−1 3 x − 1

=gcd{x2,x,1,3x-2,x2-x,x-1,-2x,-2,x2-x-3}=1.

x 1 1 0
d3(x)= x| | + 2| | =x(x2-x-3) +2= x3-x2-3x +2.
3 x−1 x 1

Therefore, the invariant factors:

f1(x)= d1(x)=1.
d2(x) 1
f2(x)= d1(x) = 1 =1.
d3(x) x3 −x2 −3x +2
f3(x)= d2(x) = =x3-x2-3x +2.
1

1 0 0
Therefore, the SCF of A=A=[0 1 0 ].
0 0 x 3 − x 2 − 3x + 2
0 1 x
Example-2: Let A= [ x x 1 ] . Find the Smith canonical form of A.
x2 − x x2 − 1 x2 − 1
Solution:

d1(x)= gcd{1,x, x 2 − x , x 2 −1}=1.

d2(x)=gcd{
x 1 x 1 1 0 x x 1 x
| 2 2 |,| 2 2 |,| |,| 2 2 |,| 2 2 |,
x −1 x −1 x −x x −1 x 1 x −x x −1 x −1 x −1
0 x 0 1 0 x 0 1 1 x
| 2 |,| |,| |,| |,| |}
x − x x2 − 1 x2 − x x2 − 1 x 1 x x x 1

57
 Linear Algebra II

=gcd{ x3-x2 –x+1, x3-x2 , x3+x, x3-x2 –x+1, x3-x2 , x2 –x,x-1,x2,x}=1.

x 1 x x
d3(x)= -1| 2 | + x| 2 |
x −x x2 − 1 x −x x2 − 1

= -1(x3-x- x2 +x ) +x(x+x2)

= -1(x3-x2 ) + x(x2-x)

= x2- x3+x3-x2=0.

Therefore, the invariant factors:

f1(x)= d1(x)=1.
d2(x) 1
f2(x)= d1(x) = 1 =1.
d3(x) 0
f3(x)= d2(x) = 1 = 0.

1 0 0
Therefore, the SCF of A=A=[0 1 0].
0 0 0
Exercise-3.1: Find the Smith canonical form of A if:

x x x2 ] x 2 ].
a) A[x] = [ 2 ] b) B[x] = [ x3 c)A[x]= [ x3
x +x x x x5 x x4

3.4. Similarity of Matrices and Invariant Factors

Theorem 3.4.1: Let A be a matrix over F[x].

A is a product of F[x]- elementary matrices if and only if A is square and det(A) is a non-zero
constant.

Proof: (→) : Suppose A is a product of F[x]-elementary matrices. Let A= Ek Ek−1…E1, where Ei

is F[x]-elementary matrix. Since each Ei is a square matrix, A is a square matrix.

Observe:

−1, if Ei obtained by exchange of two rows

Det(Ei ) ={ 1, if Ei obtained by adding f(x)
α, ifEi obtained by multiplying a row by a
non − zero constant.
Therefore, det(A)= a non-zero constant.

(←): Suppose A is a square matrix of order n and det(A) = a constant α≠ 0.

58
 Linear Algebra II

Let B= the Smith canonical form of A.

Then, B=PAQ= diag[f1(x), f2(x),…, fn(x)].

p= det(P).

q=det(Q).

Then, det(B) = f1(x) f2(x)…fn(x)=pqα.

Therefore, each invariant factor fi(x) is a constant.

Therefore, fi(x) = 1, for all i= 1,2,…,n /by monicness/.

Therefore, B=I.

Therefore, I= PAQ→ A= P −1 Q−1 .

= a product of elementary matrices.

Theorem 3.4.2: Let A be a square matrix over F[x].

A has an inverse over F[x] if and only if A is a product of F[x]- elementary matrices.

Proof: (→) : Suppose A is a square matrix and A−1 exist in F[x].

AA−1 = I.

[det(A)][det(A−1)]=1.

Det(A) and det(A−1) are polynomials.

Since their product is 1, they must be non-zero constants.

Therefore, by Theorem 3.4.1, A is a product of elementary matrices.

(←): Suppose Ais a product of elementary matrices. By Theorem 3.4.1, A is a square matrix and
det(A) is a non-zero constant. Hence, A−1 exists in F[x].

Examples: Determine whether or not the following matrices over R[x] are non-singular over
R[x].

x x+1 1 x 1 x
a) A[x] = [ ] b) B[x] = [ 2 ] c) C [x]= [ ].
x+2 x+3 x x +1 x+1 x+3
Solution:

✓ Matrix A is square and det(A) = x 2 + 3x- x 2 -3x-2=-[Link], A is invertible.

✓ Matrix B is square and det(B) = x 2 + 1- x 2 =[Link], B is invertible.

59
 Linear Algebra II

✓ Matrix C is square and det(C) = (x+3)- (x 2 + x)=3- x 2 .Hence, C is not invertible.

Theorem 3.4.3: Let A = an n*n matrix over a field F.

Let 𝒳A (x) = The characteristic polynomial of A.

Then, 𝒳A (x) is the product of the invariant factors of (xI-A).

Proof: Since det(xI-A) = 𝒳A (x) ≠ 0, rank of (xI-A) is n. Hence,(xI-A) has n invariant factors
f1(x), f2(x),…, fn(x).

Let P,Q be products of elementary matrices over F[x] such that P(xI-A)Q is the Smith canonical
form of xI-A.

Then, P(xI-A)Q= diag[f1(x), f2(x),…, fn(x)].

→ |P|𝒳A (x)|Q| = f1(x) f2(x)…fn(x).

→ pq𝒳A (x)= f1(x) f2(x)…fn(x), where p= |P|,q=|Q|. Since 𝒳A (x) and all fi(x) are monic, pq=1.

Therefore, 𝒳A (x)= f1(x) f2(x)…fn(x).

Definition 3.4.1: Let A = an n*n matrix over a field F.

The invariant factors of (xI-A) are called Similarity invariants of A.

Theorem 3.4.4: Let A = an n*n matrix over a field F.

Let mA (x) = The minimal polynomial of A. Then mA (x) = fn(x), where fn(x) is similarity
invariant of A of highest degree.

3.5. The Rational Canonical Forms

Consider the monic polynomial of degree n:

f(x) = x n + an−1 x n−1 + an−2 x n−2 + …+ a1 x + a0 .

Associated with f(x) is the n*n matrix.

0 1 0 … 0 0
0 0 1 … 0 0
C(f) = ⋮ ⋮ ⋮ ⋱ ⋮ ⋮ .
0 0 0 … 0 1
[−a0 −a1 − a2 … −an−2 −an−1 ]

C(f) is called the companion matrix of f(x).

Note:

60
 Linear Algebra II

i) If n=1 and f(x) = x −a0 , then C(f) =(a0 ).

ii) If n=2 and f(x) = x 2 –(a1 x+ a0 ), then
0 1
C(f) = [ ].
a 0 a1
iii) If f(x)= x 4 + 2x 3 -3x 2 - 5x+2, then

f(x)= x 4 -[ −2x 3 +3x 2 + 5x-2] and hence

0 1 0 0
C4 = ( 0 0 1 0 ).
0 0 0 1
−2 5 3 −2
Theorem 3.5.1: Let f(x) = x n + an−1 x n−1 + an−2 x n−2 + …+ a1 x + a0 .Then, the characteristic
and minimal polynomials of C(f) are both equal to f(x).

Proof: Left as an exercise.

Theorem 3.5.2: Let f(x) = x n + an−1 x n−1 + an−2 x n−2 + …+ a1 x + a0 be monic polynomial of
degree n. Then

[xI-C(f)] F[x] diag[1,1,…,1,f(x)].

Proof: Let C= C(f).

1 0 0 … 0 0
x 1 0 … 0 0
x 2 x 1 … 0 0
Let Q(x)= .
⋮ ⋮ ⋮ ⋱ ⋮ ⋮
x n−2 x n−3 x n−4 … 1 0
[x n−1 x n−2 x n−3 … x 1]
0 −In−1
(xI-C)Q(x) = [ ], where B= 1*n-1 matrix over F[x].
f(x) B

I 0n−1∗1 0 1
Let P(x) = [ n−1 ], R[x]= [ 1∗n−1 ].
B 1 −In−1 0n−1∗1

In−1 01∗n−1
Then, P(x)(xI-C)Q(x)R(x)= [ ].
01∗n−1 f(x)

Therefore, [xI-C(f)] F[x] diag[1,1,…,1,f(x)].

Theorem 3.5.3: Let f1(x), f2(x),…,fr(x) be non-constant monic polynomials over F such that fi(x)
divides fi+1(x) for i= 1,2,…,r-1.

61
 Linear Algebra II

Let Ci = C(fi ) for i= 1,2,…,r.

Then, the matrix B= diag[C1 , C2 ,…, Cr ] has f1(x), f2(x),…,fr(x) as its non-trivial similarity
invariants.

Proof: Let deg fi = ni , so that Ci is ni ∗ ni matrix.

Now, (xI-B)= diag[xIn1 −C1 , xIn2 − C2 ,…, xInr − Cr ].

Therefore, 𝒳B (x) = |xI − B|.

= |xIn1 −C1 |. |xIn2 − C2 |…|xInr − Cr |.

= f1(x) f2(x)…fr(x).

Moreover, xI-B= diag[xIn1 −C1, xIn2 − C2 ,…, xInr − Cr ].

F[x] diag[In1−1,f1 (x), In2−1 ,f2 (x), …,Inr −1 ,fr (x) ].

F[x] diag[1,1,…,1, f1(x), f2(x),…,fr(x)].

Therefore , f1(x), f2(x),…,fr(x) are the non-trivial similarity invariants of B.

Theorem 3.5.4: Every square matrix A over a field F is similar to a diagonal block matrix,
where each diagonal block is the companion matrix of one of the non-trivial similarity invariants
of A.

Proof: Let A be an n*n matrix over a field F. Let f1(x), f2(x),…,fr(x) be non-trivial similarity
invariants of A.

Let Ci = C(fi ) for i= 1,2,…,r.

Let B= diag[C1 , C2 ,…, Cr ].

B is an n*n matrix, since 𝒳B (x)= f1(x) f2(x)…fr(x) which is of degree n. By Theorem 3.5.3, the
non-trivial similarity invariants of B are also f1(x) ,f2(x),…,fr(x).Hence, A is similar to B.

Definition 3.5.1: A diagonal block matrix diag[C1 , C2 ,…, Cr ] of theorem 3.5.4 is called the
Rational Canonical form for the matrices similar to A.

6 2 −2
Example -1: Let A = [−2 2 2 ]. Determine the Rational Canonical form of matrices
2 2 2
similar to A.

Solution:

62
 Linear Algebra II

a) The dk (x) - minors

x − 6 −2 2
xI- A =[ 2 x−2 −2 ].
−2 −2 x−2
d1(x)=gcd{x-6,-2,2,x-2}=1.

x − 6 −2 x − 6 2 −2 2
d2(x)=gcd{ | |,| |, | |,
2 x−2 2 −2 x−2 −2
2 x−2 2 −2 x − 2 −2 x − 6 −2 x − 6 2 −2 2
| |,| |,| |,| |,| |,| |}
−2 −2 −2 x − 2 −2 x − 2 −2 −2 −2 x − 2 −2 x − 2
=gcd{(x − 6)(x − 2) + 4,-2(x-6)-4,4-2(x-2),-4+2(x-2),-4+2(x-2),(x − 2)2-4,

-2 (x − 6) − 4, (x − 6)(x − 2) + 4,-2(x-2)+4}.

=gcd{x 2 -8x+16,-2x+8,-2x+8,2x-8,2x-8,x 2 -4x,-2x+8, x 2 -8x+16}.

=gcd{(x − 4)2 , -2(x-4),2(x-4),x(x-4)}=x-4.

x − 2 −2 2 −2 2 x−2
d3(x)=(x-6)| |+2| |+2 | |.
−2 x − 2 −2 x − 2 −2 −2
=(x-6) (x 2 − 4x)+ 2(2x-8)+2(2x-8).

=(x-6)x(x-4) + 4(x-4) +4(x-4).

=(x-4)( x 2 -6x+8).

= (x-4)(x-4)(x-2)= (x − 4)2 (x-2).

b) The similarity invariants of A.

f1(x)=d1(x)=1.
d2(x)
f2(x)=d1(x) =x-4. →C(f2(x))=(4).

d3(x) 0 1
f3(x)= =(x-4)(x-2)=x 2 − 6x + 8. →C(f3(x))=( )
d2(x) −8 6
c) Characteristic polynomial of A:

XA (x) = f1(x)f2(x)f3(x)= (x − 4)2(x-2).

d) Minimal polynomial of A:

mA (x) = f3(x) =(x-4)(x-2)=x 2 − 6x + 8.

63
 Linear Algebra II

d) The Rational Canonical Form of A.

4 0 0
[0 0 1].
0 −8 6

3.6. Elementary Divisors

Let A= n*n matrix over a field F.

XA (x) = The characteristic polynomial of A.

f1 , f2 , …,fr = Similarity invariants of A.

Suppose XA (x) = p1 e1 p2 e2 …pi ei , where p1 , p2 , …,pi are distinct monic polynomials which
are irreducible over F, and each ei is a positive integer.

Then, fi = p1 ei1 p2 ei2 …pi eii (i=1,2,…,r).

Since fi /fi+1, ei+1,j ≥ eij .

Notice, eij may be 0. But if eij is positive, then ei+1,j is also positive.

Definition 3.6.1: The polynomials pj eij which appear in the similarity invariants of A with non-
zero exponents eij are called the elementary divisors of A over F.

Note: The list of elementary divisors may include duplications.

Example-1: Let f1 =(x-1)(x+1)

f2 =(x − 1)2 (x+1)(x 2 + 2)

f3 = (x − 1)2 (x + 1)2(x 2 + 2) be similarity invariants of A.

Then, the elementary divisors of A are:

(x-1), (x − 1)2 , (x − 1)2 ; (x+1), (x + 1), (x + 1)2 ; (x 2 + 2), (x 2 + 2).

Example-2: Suppose x,x, x 2 , (x + 1), (x + 1)2 , (x-1), (x − 1)3 are the list of elementary
divisors of A. Then the non-trivial similarity invariants of A are:

f3 = x 2 (x + 1)2 (x − 1)3

f2 = x (x+1)(x-1)

f1 = x.

64
 Linear Algebra II

Theorem 3.6.1: Let A, B= n*n matrices over a field F. Then, A∼B if and only if A and B have
the same elementary divisors.

Proof: (→): Suppose A∼B.

Then A and B have the same similarity invariants and hence the same elementary divisors.

(←): Suppose A and B have the same elementary divisors. Then A and B have the same
similarity invariants.

Hence, A∼B.

3.7. The Normal and Jordan Canonical Forms

3.7.1 The Normal Canonical Forms
Let f(x) = x n + an−1 x n−1 + an−2 x n−2 + …+ a1 x + a0 ∈ F[x].

We can write f(x) = p1 e1 p2 e2 …pi ei .Where the pi ‘s are distinct, monic, irreducible polynomials
over F; and the ei ‘s are positive integers.

Lemma-1: Let C=C(f). Then

C ∼ diag[C1 , C2 ,…, Ct ], where C=C(pi ei ).

Proof: Let A= diag[C1 , C2 ,…, Ct ].Then

xI- A = diag[xI − C1 , xI − C2 ,…, xI − Ct ].

F[x] diag[1,1,…,1, p1 e1 , 1, … ,1, p2 e2 , 1, … . ,1 ,pi ei ].

Since the pi ’s are irreducible and distinct, the gcd of the (n-1)th order minors of the last matrix is
1.

Therefore, xI-A F[x] diag[1,1,…,1,f(x)].

F[x] xI- C(f).

Hence, A ∼C.

Theorem 3.7.1: Let A be an n*n matrix over a field F.

Let g1(x), g2(x),…,gr(x) be the elementary divisors of A.

65
 Linear Algebra II

Let Ci = C(g i ) for i= 1,2,…,r.

Then A ∼ diag[C1 , C2 ,…, Cr ].

Proof: Let f1(x), f2(x),…,ft(x) be non-trivial similarity invariants of A. Since we know that

A ∼ diag[C(f1), C(f2),…,C(ft)], the theorem follows from Lemma-1.

Definition 3.7.1: Let A, C1 , C2 ,…, Cr be as in theorem [Link] matrix diag[C1 , C2 ,…, Cr ] is

called the Normal Canonical Form for the matrices similar to A.

6 2 2
Example-1: Let A = [−2 2 0]. Determine the Normal Canonical form of matrices similar to
0 0 2
A.

Solution:

a) The dk (x) - minors

x − 6 −2 −2
xI- A =[ 2 x−2 0 ].
0 0 x−2
d1(x)=gcd{x-6,-2,2,x-2}=1.

x − 6 −2 x − 6 −2 −2 −2
d2(x)=gcd{ | |,| |, | |,
2 x−2 2 0 x−2 0
2 x−2 2 0 x−2 0 x − 6 −2 x − 6 −2 −2 −2
| |,| |,| |,| |,| |,| |}
0 0 0 x−2 0 x−2 0 0 0 x−2 0 x−2
=gcd{(x − 6)(x − 2) + 4,4,2(x-2),0,2(x-2),(x − 2)2,0,(x-6)(x-2),-2(x-2)].

= 1.

x − 6 −2
d3(x)=(x-2)| |= (x-2)[(x-6)(x-2)+4]= (x-2) (x 2 -8x+12+4]
2 x−2
=(x-2)( x 2 -8x+16).

= (x − 4)2(x-2).

b) The similarity invariants of A.

f1(x) = d1(x)=1.
d2(x)
f2(x) = d1(x) = 1.

66
 Linear Algebra II

d3(x)
f3(x) = d2(x) =(x − 4)2 (x-2)

c) Characteristic polynomial of A:

XA (x) = f1(x)f2(x)f3(x)= (x − 4)2(x-2).

d) Minimal polynomial of A:

mA (x) = f3(x) = (x − 4)2 (x-2).

e) The Normal Canonical Form of A.

Therefore, the elementary divisors of A are : (x-2) and (x − 4)2 . Hence, the normal
canonical form of matrices similar to A is:
0 1
C[x-2]= (2) and C[(x − 4)2 ]=C[x 2 -8x+16]=[ ].
−16 8

2 0 0
A~ [0 0 1].
0 −16 8

3.7.2. The Jordan Canonical Form

Let A= n*n matrix over a field F. Suppose the elementary divisors pj eij of A are of the form
(x − λj )eij , for all j. This is case if F is algebraically closed, for example.

We define the Jordan block corresponding to the elementary divisor (x − λj )eij to be the

eij × eij matrix Jj gevin by:

λj 1 0 … 0
0 λj 1 … 0
Jj = ⋮ ⋮ ⋱ ⋮ ⋮ .
0 0 0 λj 1
[0 0 0 … λj ]

𝐓𝐡𝐞𝐨𝐫𝐞𝐦 3.7.2: Let F be algebraically closed field. If the n*n matrix A over F has r elementary
divisors with associated Jordan blocks J1 , J2 ,…, Jr then

A ∼ diag [J1 , J2 ,…, Jr ].

𝐏𝐫𝐨𝐨𝐟: We need only show that the elementary divisors in Jj and C( pj eij ) coincide for i=1,2,…,r.

We already know that the elementary divisor of C( pj eij ) is pj eij .

67
 Linear Algebra II

Examination of xI- Jj shows that there are minors of orders 1,2,…, eij − 1 having values
±[Link],

|xI − Jj |= (x − λj )eij = pj eij .

It follows that pj eij is the only elementary divisors of Jj .Definition -3.7.2: Let A, J1 , J2 ,…, Jr be
as in Theorem 3.7.2. The matrix diag [J1 , J2 ,…, Jr ] is called the Jordan Canonical Form of the
matrices similar to A.

6 2 2
Example-2: Let A = [−2 2 0]. Determine the Jordan Canonical form of matrices similar to
0 0 2
A.

Solution:

a) The dk (x) - minors

x − 6 −2 −2
xI- A =[ 2 x−2 0 ].
0 0 x−2
d1(x)=gcd{x-6,-2,2,x-2}=1.

x − 6 −2 x − 6 −2 −2 −2
d2(x)=gcd{ | |,| |, | |,
2 x−2 2 0 x−2 0
2 x−2 2 0 x−2 0 x − 6 −2 x − 6 −2 −2 −2
| |,| |,| |,| |,| |,| |}
0 0 0 x−2 0 x−2 0 0 0 x−2 0 x−2
=gcd{(x − 6)(x − 2) + 4,4,2(x-2),0,2(x-2),(x − 2)2,0,(x-6)(x-2),-2(x-2)].

= 1.

x − 6 −2
d3(x)=(x-2)| |= (x-2)[(x-6)(x-2)+4]= (x-2) (x 2 -8x+12+4]
2 x−2
=(x-2)( x 2 -8x+16).

= (x − 4)2(x-2).

The similarity invariants of A.

f1(x)=d1(x)=1.
d2(x)
f2(x)=d1(x) = 1.

68
 Linear Algebra II

d3(x)
f3(x)=d2(x) =(x − 4)2 (x-2)

b) The Jordan Canonical Form of A.

Therefore, the elementary divisors of A are : (x-2) and (x − 4)2 . Hence, the Jordan
canonical form of matrices similar to A is:
4 1
J[x-2]= (2) and J[(x − 4)2 ]=[ ].
0 4
2 0 0
A~ [0 4 1].
0 0 4

𝐄𝐱𝐞𝐫𝐜𝐢𝐬𝐞 − 𝟑. 𝟑:
x 1 0
1. Let A[x] = [0 x 1 ].
2 3 x−1
a) Find the invariant and smith canonical forms of A .
b) Determine whether or not A is non-singular over R[x].
2. Determine the Similarity invariants of A:
2 2 −1 0 −1 2
a) A= [−1 −1 1 ] b) A=[3 −4 6].
−1 −2 −2 2 −2 3
In each case, write the Rational Canonical form of A.
0 0 1
3. Let A= [1 0 −1] .Find the Normal and Jordan canonical Forms/over C/ of matrices
0 1 1
similar to A.
4. Suppose x+1,x(x+1),x(x+1)(x 2 + 2) are the non-trivial similarity invariants of a matrix A.
a) Find the elementary divisors of A over R.
b) Find the elementary divisors of A over C.
c) Write the Jordan Canonical form of matrices similar to A over C.

69
 Linear Algebra II

CHAPTER FOUR

BILINEAR AND QUADRATIC FORMS

4.1. Bilinear forms and matrices

4.1.1. Bilinear Maps

Definition 4.1.1: Let V, W be vector spaces over the field K. A mapping:

φ : V×W → K is called bilinear if

i) φ(αv1 + βv2 , w) = αφ(v1 , w) + βφ(v2 , w), for all v1 ,v2 ∈ V;w ∈ W; α, β ∈ K.

ii) φ(v, αw1 + βw2 ) = αφ(v, w1 ) + βφ(v, w2 ), for all w1 ,w2 ∈ W;v ∈ V; α, β ∈ K.

Note: Condition (i) states that φ is a linear map with respect to the first variable and condition
(ii) states that φ is a linear map with respect to the second variable.

Example 4.1.1:

1. Let V= a vector over a field K with an inner product ⟨ , ⟩.Then the inner product
⟨ , ⟩: V×V→ K is a bilinear map.
2. Let V be a ε vector space over the field K. Let V* be the dual space of V. Then the map

φ : V*×V → K given by φ (ϕ,v) = ⟨ϕ ,v⟩ is also a bilinear map.

4.1.2. Matrix Representation Of Bilinear Maps

Suppose : V×W → K is called bilinear map.

Let β = {v1 ,v2 ,…, vm } be a basis of V.

β′ = {w1 ,w2 ,…, wn } be a basis of W.

Then for any w ∈ W , v ∈ V ,we have v = ∑m n

i=1 xi vi and w = ∑j=1 yj wj . Then,

φ(v,w) = φ (∑m n
i=1 xi vi , ∑j=1 yj wj ).

= ∑m n
i=1[xi φ(vi ), ∑j=1 yj wj ].

= ∑m n
i=1 xi [ ∑j=1 yj φ(vi, wj )].

= ∑m n
i=1 ∑j=1 xi yj φ(vi, wj ).

Thus, the mn scalars φ(vi, wj ) completely determine the value of the map φ. Let aij = φ(vi, wj )
and consider the m*n matrix : A = (aij ).

70
 Linear Algebra II

x1 y1
x2 y2
Then, φ(v,w) = X t A Y where X = [ ⋮ ] , Y= [ ⋮ ].
xm yn

Definition 4.1.2: A is called the matrix representation of the bilinear map : V×W → K with
respect to the basis β, β, of V and W respectively.

Example-1: Let u=(x1 , x2 ) and v =(y1 , y2 ) .Determine f(u,v)= x1 y2 + x2 y1 is bilinear map on R2 .

Solution: Take w= (z1 , z2 ).

a) f(αu + βw, v) = f(α(x1 , x2 ) + β(z1 , z2 ), (y1 , y2 ))

=f[(αx1 + βz1 , αx2 + βz2 ), (y1 , y2 )]

= [αx1 + βz1 ]y2 + [αx2 + βz2 ]y1

= α[x1 y2 + x2 y1 ] +β[z1 y2 + z2 y1 ].
=αf(u, v) + βf(w, v).
b) f(v, αu + βw) = f((y1 , y2 ), (αx1 + β z1 , αx2 + βz2 ))
=[αx2 + βz2 ]y1 + [αx1 + βz1 ]y2
= α[y1 x2 + x1 y2 ] +β[z1 y1 + z1 y2 ].
=αf(v, u) + βf(v, w).
Therefore, f is a bilinear form on R2 .
Example -2: Let f be the bilinear map on R2 defined by
F((x1 , x2 ), (y1 , y2 ))= x1 + y1 − x2 −y2 .

Find the matrix A of F in the basis {u1 = (1,1), u2 = (1,2)}.

Solution: A = (aij ) = F(ui , wj ) = F(ui , uj ) .

a11 =F(u1 , u1 )=F((x1 , x2 ), (x1 , x2 )) = x1 + x1 − x2 −x2 .

= 2x1 -2x2 =2*1-2*1=0.
a12 = F(u1 , u2 )= F((x1 , x2 ), (y1 , y2 ))= x1 + y1 − x2 −y2 .
= 1+ 1-1-2=-1.
a21 =F(u2 , u1 )= F((y1 , y2 ), (x1 , x2 )) = y1 + x1 − y2 −x2
= 1+1-2-1=-1.
a22 =F(u2 , u2 )= F((y1 , y2 ), (y1 , y2 )) = y1 + y1 − y2 −y2
=2y1 - 2y2 = 2*1-2*2=2-4=-2.

0 −1
Therefore, A= [ ].
−1 −2

71
 Linear Algebra II

4.1.3. Bilinear Forms

Definition 4.1.3: If in the definition of a bilinear map, the vector spaces V and W are the same,
then the bilinear map φ : V×V → K is called a bilinear form on V.

Example 4.1.2: V= an inner product space with inner product ⟨ , ⟩. The inner product

⟨ , ⟩: V×V → K is a bilinear form.

Note: Let B(V) = the set of all bilinear forms on V. If f,g ∈B(V) and λ ∈ K, we can define f+g
and λf by:

(f+g) (v,w) = f(v,w) + g(v,w).

(λf) (v,w) =λ f(v,w).

Thus B(V) is a vector space over K. In fact,

Theorem 4.1.1: Let V be a vector space of dimension n over K. Let {φ1 , φ2 , …,φn } be a basis
of the dual space V*.

Then {fij : i,j =1,2,…,n}, where fij is defined by fij (v,w) = φi (v)φj (w) , is a basis of B(V).
Thus, in particular, dim B(V) = n2 .

Proof: Let {e1 , e2 , …,en } be a basis of V dual to {φ1 , φ2 , …,φn }.Thus, φi (ej ) = δij .

i) We show {fij } generates B(V).

Let f ∈B(V) and suppose f(ei , ej ) = aij .

Claim: f = ∑ aij fij .

It suffices to show that :

f(es , et ) = (∑ aij fij ) (es , et ).

But (∑ aij fij ) (es , et ) = ∑ aij fij (es , et ).

= ∑ aij φi(es ) φj (et ).

= ∑ aij δis δjt = ast = f(es , et ).

ii) We next show that {fij } is linearly independent.

Suppose, ∑ aij fij = 0. Then, for s,t = 1,2,…,n,
0= 0(es , et ) = (∑ aij fij ) (es , et ) = ast .

72
 Linear Algebra II

Theorem 4.1.2: Let [f] be a matrix representation of f ∈B(V) relative to a basis {e1 , e2, …,en }
of V . Then the mapping f→ [f] is an isomorphism of B(V) on to the vector space Mn (K) of n*n
matrices over K.

Proof: since f is completely determined by the scalars f(ei , ej ), the mapping f→ [f] is one to one
correspondence . It only remains to show that the mapping f→ [f] is a homomorphism.

But

i) (f+g) (ei , ej ) = f (ei , ej ) + g(ei , ej ), showing [f+g] = [f]+[g].

ii) (λf) (ei , ej ) =λ f(ei , ej ) , showing[λf ]= λ[f].

Example-1: For the following bilinear form determine the matrix A?

a) b(x,y)= 5x1 y1 +3x1 y2 + 3 x2 y1 + x3 y1 + 3 x3 y2 .

b) b(x,y)= 5x1 y1 −2x1 y2 + 7x2 y1 - 5 x2 y2 + 6 x2 y3 + x3 y1 +4 x3 y2 + x3 y3 .
Solution:
a)
A= y1 y2
x1 5 3
x 2 [3 0 ]
x3 1 3
b)

A= y1 y2 y3

x1 5 −2 0
x2 [7 −5 6].
x3 1 4 1

Example-2: For the following matrix determine the bilinear form b(x,y)?

3 5 2 1 −3
a) A= [ 7 −5 1] b) A = [9 10 ].
−3 8 1 1 2
Solution:

a) b(x,y)= 3x1 y1 +5x1 y2 + 2x1 y3 +7x2 y1 - 5 x2 y2 + x2 y3 -3x3 y1 + 8 x3 y2 + x3 y3 .

b) b(x,y)= 5x1 y1 −3x1 y2 + 9 x2 y1 + 10x2 y2 + x3 y1 + 2 x3 y2 .

4.2. Alternating bilinear forms

Definition 4.2.1: Let V= a finite dimensional vector space over a field K. Let f = a bilinear form
on V.

f is said to be alternating if f(v,v) = 0, for all v∈V.

73
 Linear Algebra II

Note:

1) Let f be an alternating bilinear form on V.

0 = f(v+w,v+w) = f(v,v) + f(v,w) + f(w,v) +f(w,w).
= f(v,w) + f(w,v).
Therefore, f(v,w) = - f(w,v) ; for all v, w ∈V. ……….(*)
A bilinear form which satisfies condition (*) is called skew symmetric.
Thus, every alternating bilinear form is a skew symmetric bilinear form.
2) Suppose f is a skew symmetric bilinear form and 1+1≠ 0 in K. Then condition (*) implies
f(v,v) = -f(v,v).
→(1+1) f(v,v) =0.
→ f(v,v) = 0, for all v ∈V.
Therefore, f is alternating bilinear form.

Thus, alternating and skew symmetric bilinear forms are equivalent when 1+1≠ 0 in K.

Example: Show that f(v,w) = 2xy ′ -2x ′ y , for v =(x,y) and w= (x ′ , y ′ ) is an alternating bilinear
form?

Solution: f(v,v) = f[(x,y),(x,y)] = 2xy – 2xy = 0, for all v ∈V.

0 2
The matrix representation is skew symmetric: A= [ ].
−2 0

4.3. Symmetric bilinear forms and quadratic forms

4.3.1. Symmetric Bilinear Forms
Definition 4.3.1: A bilinear form : V×V → K is said to be symmetric if φ (v,w) = φ(w,v); for all
v,w ∈V.

Note: A symmetric bilinear form is none other than an inner product on V.

Theorem 4.3.1: An n*n matrix A over a field K represents a symmetric bilinear form if and only
if it is a symmetric matrix.

Proof: (→): Suppose A represents a symmetric bilinear form, i.e.,

X t AY = Y t AX, for all X,Y ∈ F n .

Now, Y t AX= (Y t AX)t = X t At Y.

→ X t AY= X t At Y ; for all X,Y ∈ F n .

→ A = At .

(←): Suppose A is symmetric, i.e., A= At .

74
 Linear Algebra II

The  X,Y∈ F n , X t AY= (X t AY)t = Y t At X = Y t AX.

→A represents a symmetric bilinear form.

Theorem 4.3.2: Let f be a symmetric bilinear form on V over K /in which 1+1 ≠ 0/. Then V has
a basis {v1 , v2 , …,vn }in which f is represented by a diagonal matrix, i.e., f(vi , vj ) =0 for i ≠ j.

Proof: Left as a reading assignment.

Example-1: For the following symmetric bilinear form determine the matrix A?

b(x,y)= 4x1 y1 + x1 y2 + x2 y1 - 2 x2 y2 - 4 x2 y3 - 4 x3 y2 + 7x3 y3 .

Solution:

A= y1 y2 y3

x1 4 1 0
x2 [1 −2 −4].
x3 0 −4 7

Example-2: For the following symmetric matrix determine the symmetric bilinear form, b(x,y)?

3 5 2
A= [5 −5 1] .
2 1 1
Solution:

b(x,y)= 3x1 y1 +5x1 y2 + 2x1 y3 +5x2 y1 - 5 x2 y2 + x2 y3 + 2x3 y1 + x3 y2 + x3 y3 .

Exercise-4.1:

[Link] u=(x1 , x2 ) and u=(y1 , y2 ) .Determine which of the following are bilinear forms on R2 .
i. f(u,v)=x1 + y2 iv. f(u,v)= 1
ii. f(u,v)=2 x1 y1 v. f(u,v)=0
iii. f(u,v)=2x1 x2 + 3y1 y2 vi. f(u,v)= 3x1 y2 + x2 y1 .
2
2. Let f be the bilinear form on R defined by
a)F((x1 , x2 ), (y1 , y2 ))= 2x1 y1 +3x1 y2 + x2 y1 -2 x2 y2 .
b)F((x1 , x2 ), (y1 , y2 ))= 3x1 x2 + 2y1 y2 .
i. Find the matrix A of f in the basis {u1 = (1, 1), u2 = (2,1)}
ii. Find the matrix A of f in the basis {v1 = (1, 3), v2 = (1,1)}
2 5
3. Let V be the vector space of 2  2 matrices over R. Let M= [ ],and let f(A,B)=tr(At MB)
1 1
where A,B  V and ‘’tr’’ denotes trace.

i. Show that f is bilinear form on V.

75
 Linear Algebra II

1 0 0 1 0 0 0 0
ii. Find the matrix of f in the basis {[ ],[ ],[ ],[ ]}.
0 0 0 0 1 0 0 1
4.3.2. Quadratic Forms
Definition 4.2.1: Let V be a finite dimensional vector space over a field K.

A mapping q:V→ K is called a quaradratic form if q(v) = f(v,v) for some symmetric bilinear
form f on V.

Example 4.2.1: Let q: R2 →R be given by q(x,y) = xy + y 2 . q is a quadratic form since q(v)=

f(v,v), where
1 1
f(v,w) = y , y + xy , + x , y, for v = (x,y), w = (x , , y , ). It is not difficult to show
2 2
that f is a symmetric bilinear form on V.

Notes:

a) Notice that if 1+1 ≠ 0 in K , then f is obtainable from q according to the identity.

1
f(v,w) = 2 [q( v+w) - q(v)- q(w) ] …………………(*)
To see this, we observe:
q( v+w) - q(v)- q(w) = f( v+w , v+w) - f(v,v) - f(w,w) =2f(v,w).
The formula (*) is Called the polar form of f.
b) If f is represented by a symmetric matrix A = (aij ) , then q is represented in the form:
n∗n
a11 a12 … a1n x1
a21 a22 … a2n x2
q(X) = f(X,X) = X t AX = (x1 x2 … xn ) [ … … ⋱ … ] [ ⋮ ].
an1 an2 … ann xn
= ∑i.j aij xi xj .
= a11 x1 2 + a22 x2 2 +…+ ann xn 2 + 2 ∑i<j aij xi xj . ………(*)

The formula (**) is called the quadratic polynomial corresponding to the symmetric matrix A.

Observe that if A is diagonal, then q(X) = a11 x1 2 + a22 x2 2 +…+ ann xn 2 is called canonical
form.

Example 4.2.2: Find the symmetric matrix which corresponds to the quadratic polynomial

q(x,y) = xy + y 2 .

Solution:

The symmetric matrix A= A = (aij ) representing q(x1 x2 … xn ) has the diagonal entry
n∗n
aii equal to the coefficient of xi 2 and has the entries aij each equal to half the coefficient of xi xj .

76
 Linear Algebra II

1
0 2
Thus, A= [ 1 ].
1
2

Example 4.2.3: Let q(x,y) = 4x 2 − 6 xy -7 y 2 . Find the symmetric matrix which corresponds to
the quadratic polynomial

Solution:

The symmetric matrix A= A = (aij ) representing q(x1 x2 … xn ) has the diagonal entry
n∗n
aii equal to the coefficient of xi 2 and has the entries aij each equal to half the coefficient of xi xj .

4 −3
Thus, A= [ ].
−3 −7
Example 4.2.4: For the following symmetric matrices determine the quadratic form, q(X)?

2 6
a) A= [ ].
6 1
3 3 1
b) A= [3 5 2] .
1 2 1
Solution:

a) q(X) = q(x,y) = 2x 2 + 12 xy + y 2 .
b) q(X)= q(x1 , x2 , x3 ) = 3x1 2 + 5x2 2 + x3 2 +6 x1 x2 + 2x1 x3 + 4x2 x3 .

Exercise-4.2:

1. Find the symmetric matrix which corresponds to each of the following quadratic
polynomials.
a)q(x,y) = x 2 −xy + y 2 .
b)q(x,y) =x 2 +4yz + 2xz + 5y 2 .
c) q(x,y) =x 2 +2xy-y 2 + 6xz – 4yz + 3z 2 .
2. For each of the following matrices A, find a non-singular matrix P such that P t AP is
diagonal:

2 3
i) A= [ ].
3 4
1 −2 3
ii) A = [−2 6 −9].
3 −9 4

77
 Linear Algebra II

In each case find the rank and signature.

3. Let q be the quadratic form associated with a symmetric bilinear form f. Verify the following
1
alternate polar form of f(u,v)=4 [q(u+v)- q(u-v)].

4.4. Real Symmetric Bilinear Forms and Positive Definite Forms

4.4.1. Real symmetric Bilinear Forms
Let V be a vector space over R.

Theorem 4.4.1: Let f be a symmetric bilinear form on V. Then

i) There is a basis of V in which f is represented by a diagonal matrix.

ii) Every diagonal representation of f has the same number P of positive entries and the
same number N of negative entries.

Proof:

i) Follows from previous theorem since 1+1≠0 in R.

ii) Suppose {u1 , u2 , … , un } is a basis of f in which f is represented by a diagonal
matrix, say, with P positive and N negative entries. Suppose also
{w1 , w2 , … , wn } is another basis of f in which f is represented by a diagonal
matrix , say, with P ′ positive and N′ negative entries.

We may assume without any loss of generality that in each matrix the positive entries appear
first. Since rank(f) = P +N= P ′ + N′ , it suffice to prove that P=P ′ .

Let U = {u1 , u2 , … , up }.

W={wp′ +1 , wp′ +2 , … , wn }.

Then f(v,v) > 0 for every non-zero v ∈U, and f(v,v) ≤ 0 for every non-zero v ∈W.

Hence, U  W={0}.

Note that dim(U)= P.

dim(W)= n-P ′ .

Thus dim [U+W]= dim(U) + dim(W) – dim[U  W].

= P + n-P ′ -0 = P -P ′ + n.

But dim [U+W] ≤ dim(V) =n.

Hence, P -P ′ + n ≤ n or P ≤ P ′ . Similarly, P ′ ≤ P.

78
 Linear Algebra II

Therefore, P= P ′ .

Theorem 4.3.1: Let V= finite dimensional vector space over R.

f = a symmetric bilinear form on V.

p = The number of positive entries of a diagonal representation of f.

n = The number of negative entries of a diagonal representation of f.

We call:

Rank= p + n.

p= The index of f.

p – n = The signature of f.

nature:

. positive definite(PD) = all the diagonal entries are positive.

.positive semi-definite(PSD) = all the diagonal entries are non-negative.

.negative definite(ND) = all the diagonal entries are negative.

.negative semi-definite(NSD) = all the diagonal entries are non-positive.

.No nature = different from the above.

This same terms apply to the associated quadratic form.

Example-1: Consider the quadratic form q on R2 given by:

q(x,y) = x 2 − y 2 . The matrix representation of q with respect to the standard ordered basis of

R2 is:

1 0
A= [ ].
0 −1
A is also the matrix representation of the bilinear form f on R2 corresponding to q.

Therefore, index f = index of q = 1.

Signature of f = signature of q = 0.

Example-2: Reduce the following quadratic forms into canonical form. And determine rank,
signature, index and nature of the quadratic forms.

79
 Linear Algebra II

a) q(x,y,z) =8x 2 +7y 2 + 3z 2 -12xy+ 4xz – 8yz .

b) q(x1 , x2 , x3 ) = x1 2 + 2x2 2 + x3 2 - 2x1 x2 + 2x2 x3 .

Solution:

8 −6 2
a) First , A = [−6 7 −4].
2 −4 3
6−λ −6 2
The characteristic equation is |A − λI| = 0= | −6 7−λ −4 |=0= λ3 - D1 λ2 + λD2 - D3
2 −4 3−λ
D1 = 8+7+3=18.

7 −4 8 2 8 −6
D2 = | |+ | |+| | = 5 + 20 +20 = 45.
−4 3 2 3 −6 7
7 −4 −6 −4 −6 7
D3 =8| | + 6| | + 2| | = 40-60+20=0.
−4 3 2 3 2 −4
This implies: λ3 - 18λ2 + 45λ = 0.

→ λ (λ - 15) (λ - 3) =0.

→ λ =0,3,15.

0 0 0
Therefore , D = [0 3 0 ].
0 0 15
The canonical form is : 3y 2 + 15z 2 .

1)Nature =PSD (λi > 0).

2)Rank =number of positive eigenvectors + number of negative eigenvectors = 2+0=2.

3)Index = number of positive eigenvectors=2.

4)Signature = number of positive eigenvectors - number of negative eigenvectors = 2-0=2.

Exercise-4.3:

Reduce the following quadratic form:

q(x1 , x2 , x3 ) = x1 2 + 2x2 2 − 7x3 2 - 4x1 x2 + 8x1 x3 + 5x2 x3 ,

into canonical form. And determine rank, signature, index and nature of the quadratic form.

4.4.2. Positive Definite Matrices

80
 Linear Algebra II

Definition 4.4.1: Let V be a vector space over R.

Let f be symmetric bilinear form on V.

We call f positive definite if f(v,v) ≥ 0 for all v ∈ V, and f(v,v) > 0 if v ≠ 0.

Notice: If f is positive definite, signature of f = index of f = dim(V).

Definition -4.4.2: A real symmetric n*n matrix A is said to be positive definite if X t AX > 0 for
all X≠ 0.

.Symbolized by A>0.

Example: Let f be the ordinary dot product in Rn ; that is :

f(v,w) = v • w = x1 y1 + x2 y2 + …+ xn yn , where

v= (x1 , x2 ,…, , xn ); w = (y1 , y2 ,…, , yn ) is symmetric, since f(v,w) = f(w,v) = v • w.

f is positive definite, because

f(v,v) = x1 2 + x2 2 + ⋯ + xn 2 > 0, when v ≠ 0.

Definition-4.4.3: An n*n complex matrix A is said to be positive semi-definite, written A≥ 0, if

X*AX ≥ 0 for all X∈ Cn .

Generally, the following methods are test for positive definite and positive semi-definite.

No. Methods Positive definite Positive semi-definite

1 Eigenvalue All λ >0 All λ ≥ 0

2 Pivot elements All >0 All ≥ 0

3 Leading principal minors All >0 All ≥ 0
4 Energy/ perfect square X t AX >0 for all X≠ 0.
X*AX ≥ 0 for all X

Example: Check that the following symmetric matrices are positive definite, positive semi-
definite or not.

4 6
a) A= [ ]
6 10
1 1 2
b) B = [1 2 3]
0 −1 −1

81
 Linear Algebra II

Solution:

4 6
a) A = [ ]
6 10
i) By using pivot element method:

4 6 4 6
[ ]↔[ ]. Therefore, all the pivots are positive:1and 4.
6 10 0 1
ii) By taking left-upper matrices determinants:

|4| =4> 0.

4 6
| |=4*10- 6*6=40-36=4 >0.
6 10
→ A is positive definite.

iii) Energy / perfect square / method:

4 6 x1
X t AX=(x1 x2 ) [ ][ ]
6 10 x2
= 4x1 2 + 12x1 x2 + 10x2 2
= [4x1 2 + 12x1 x2 + 9x2 2 ] + x2 2
= (2x1 + 3x2 )2 + x2 2 > 0 ,for all (x1 , x2 ) ≠ 0.

1 1 2
b) B= [1 2 3 ].By taking left-upper matrices determinants:
0 −1 −1
|1| =5> 0.

1 1
| |=2- 1=1>0.
1 2
1 1 2
1 1 1 1
|1 2 3 |=1 | |- | | = (-2+3) – (-1+2)=1-1=0≥ 0.
1 2 1 2
0 −1 −1
→ A is positive semi-definite.

Exercise-4.4: For which values of c is

2 −1 −1
B = [−1 2 −1 ]
−1 −1 2 + c
Positive definite and positive semi-definite ?

82
 Linear Algebra II

CHAPTER FIVE

DIRECT SUM DECOMPOSITION OF VECTOR SPACES

5.1. Definition of a direct sum of vector spaces

Definition 5.1.1: Let V be a vector space over a field F.

Let W1 , W2 subspace of V.

We say that V is a direct sum of W1 and W2 and write:

V = W1  W2 , if v V there exist unique elements w1 ∈ W1 , w2 ∈ W2 such that

v = w1 + w2 .

Theorem 5.1.1: Let V a vector space over a field F.

Let W1 , W2 subspace of V.

If V= W1 + W2 and W1  W2 = {0}, then V = W1  W2 .

Proof: Let v  V .

Since V= W1 + W2 ,  w1 , w2 are unique.

Suppose  w1 ′ , w2 ′ ∈ W2 such that v = w1 ′ + w2 ′ also.

Then, v = w1 + w2 = w1 ′ + w2 ′ .

→ w1 - w1 ′ = w2 ′ - w2 ∈ W1  W2 .

→ w1 = w1 ′ and w2 ′ = w2 .

Definition 5.1.2:

Let V a vector space over a field F.

Let W1 , W2 ,…, Wk subspace of V.

We say that V is a direct sum of W1 , W2 ,…, Wk , and write:

V = W1  W2  …  Wk , if v V can be expressed uniquely in the form

v = w1 + w2 +…+ wk with wi ∈ Wi , for i= 1, 2,…,k.

Examples on Direct sum

83
 Linear Algebra II

1. V = 𝑅 3 , U = {(a,b,0): a, b ∈ R}; W = = {(0,b,c): b, c ∈ R}.Is V a direct sum of U and W?

Solution:
i) E.g.: (3,5,7) = (3,1,0) + (0,4,7) = (3,-4,0) + (0,9,7)→ is not unique.
ii) (0,1,0) ∈ U  W.
0
U  W = [𝑒 ].
0
→ U  W ≠ {0}.

→ V = 𝑅 3 ≠ U  W.

2. V = 𝑅 3 , U = {(a,b,0): a, b ∈ R}; W = {(0,0,c): c ∈ R}.Is V a direct sum of U and W?

Solution:
i) (a, b, c) = (a , b, 0) + (0,0,c) → is sum.
ii) U  W = (0, 0, 0) = {0}.→ is unique.

→ V = 𝑅 3 = U  W.

3. V = 𝑅 3 , U = {(a, b, c): a= b=c ∈ R}; W = {(0,b,c): b,c ∈ R}.Is V a direct sum of U and W?

Solution:

i) 𝑅3 = U + W .
→ (a, b, c) = (a , a, a) + (0,b-a,c-a) → is sum.

ii) U  W = (0, 0, 0) = {0}.

Since, v = (a, b, c) ∈ U  W.

→ a = b= c and a=0.

→b=c=0.

→ v = ( 0, 0, 0). → is unique.

V = 𝑅 3 = U  W.

Definition 5.1.3: Let V a vector space over a field F.

Let 𝑊1 , 𝑊2 ,…, 𝑊𝑘 subspace of V.

We say 𝑊1 , 𝑊2 ,…, 𝑊𝑘 are independent if

𝑤1 + 𝑤2 +…+ 𝑤𝑘 = 0, where 𝑤𝑖 ∈ 𝑊𝑖 → each 𝑤𝑖 = 0 ; i= 1, 2,…,k.

Theorem 5.1.2: Let V a vector space over a field F.

84
 Linear Algebra II

Let W1 , W2 ,…, Wk subspace of V such that

V = W1 + W2 +…+ Wk .

Then, the following statements are equivalent:

i) W1 , W2 ,…, Wk are independent.

ii) V = W1  W2  …  Wk .
iii) (W1 + W2 +…+ Wj−1 )  Wj = {0} for j = 2,3,…,k.

Proof: (i)→ (ii):

Let v ∈ V, Since V = W1 + W2 +…+ Wk ,  wi ∈ Wi for i= 1,2,…,k such that

v = w1 + w2 +…+ wk . We need to show that W1 , W2 ,…, Wk are unique.

Suppose also  wi ′ ∈ Wi for i= 1,2,…,k such that

v = w1 ′ + w2 ′ + …+ wk ′ .

Then, 0= v-v = (w1 − w1 ′ ) + (w2 − w2 ′ ) + …+ (wk − wk ′ ) .

→ each wi − wi ′ = 0./Since W1 , W2 ,…, Wk are independent/

→ wi = wi ′ for i= 1,2,…,k.

(ii)→ (iii):

Let v ∈ (W1 + W2 +…+ Wj−1 )  Wj for j = 2,3,…,k.

Then, v = w1 + w2 +…+ wj−1 = wj for some wi ∈ Wi for i= 1,2,…,j.

→ w1 + w2 +…+ wj−1 +(- wj ) = 0.

→ w1 = w2 =…= wj−1 = wj = 0./ by unique expression for 0/.

→ v = 0.

(iii)→ (i):

Suppose W1 + W2 +…+ Wk = 0 , whose wi ∈ Wi .

→ w1 + w2 +…+ wk−1 = (−wk ).

→ wk ∈(W1 + W2 +…+ Wk−1 )  Wk = {0}.

→ wk = 0.

85
 Linear Algebra II

→ w1 + w2 +…+ wk−1 = 0.

→ wk−1 = 0./ by the same reasoning as before/.

→ w1 = w2 =…= wk−1 = wk = 0 / continuing the process/.

Therefore, W1 , W2 ,…, Wk are independent.

Theorem 5.1.3: Let V be a finite dimensional vector space over a field F.

Let W1 , W2 ,…, Wk subspace of V such that

V = W1  W2  …  Wk .

Let βi = {vi1 , vi2 ,…, vidi } be a basis of Wi , i= 1,2,…,k.

Then, β = ⋃ki=1 βi is a basis of V.

Proof:

(i) Let v ∈ V. Then v = w1 + w2 +…+ wk uniquely with wi ∈ Wi for i= 1,2,…,k.

d
But wi = ∑j=1
i
aij vij for i= 1,2,…,k.
d d d
1
Therefore, v = ∑j=1 2
a1j v1j +∑j=1 a2j v2j + …+∑j=1
k
akj vkj .
Hence , β generates V.
(ii) Suppose
dk
∑dj=1
1 d2
a1j v1j +∑j=1 a2j v2j + …+∑j=1 akj vkj = 0.

d d d
1
→ ∑j=1 2
a1j v1j = ∑j=1 a2j v2j = …= ∑j=1
k
akj vkj = 0.

/ Since W1 , W2 ,…, Wk are independent /

→ aij = 0 ij , since βi is linearly independent for i= 1,2,…,k.

Therefore, β is linearly independent.

Hence, β is a basis for V.

5.2. Projection and Invariant subspaces of a linear operator

Let V be a finite dimensional vector space over a field F. Let T be a linear operator on V.

Definition 5.2.1: A subspace W of V is called invariant under T if T(W)⊆W.

Note:

86
 Linear Algebra II

(i) V is invariant under T.

(ii) {0} is invariant under T.
(iii) The Null space of T is invariant under T.
(iv) The Range of T is invariant under T.
(v) Any eigenspace of T is invariant under T.

Example-5.2.1: Let V = F[x].

W = The subspace of V of polynomials of deg ≤ n.

D = The differential operator.

Therefore, W is invariant under T.

Definition -5.2.2: Let V= a vector space over a field F.

T= a linear operator on V.

W = a subspace of V invariant under T.

i) The restriction of T to W, denoted by Tw , is a linear operator on W.

Tw (W) = T(w),  w ∈ W.

ii) Suppose V = W1  W2  …  Wk , where each Wi is invariant under T.

T induces a linear operator Ti on each Wi by restriction. If v ∈ V, we have unique
vectors w1 , w2 ,…, wk with wi ∈ Wi such that :
v = w1 + w2 +…+ wk .Then, T(v)= T(w1 ) +T( w2 ) +…+T( wk ).
When T,T1 , T2 ,…, Tk are as above , we say T is a direct sum of the operations
T1 , T2 ,…, Tk .

Definition -5.2.3: Let V= a vector space over a field F.

T= a linear operator on V. T is said to be idempotent if 𝑇 2 = T.

Theorem 5.2.1: Let V= a finite dimensional vector space over a field F.

T= an idempotent linear operator on V.

R = Range of T.

N = Null space of T.

Then, V= R  N.

Proof:

87
 Linear Algebra II

(i) Let 𝑣 ∈ V.
Then, v = T(v) + [v – T(v)], T(v) ∈ R.
v – T(v) ∈ N, since T(v-T(v)) = T(v) - 𝑇 2 (v) = T(v)- T(v) = 0.
Therefore, 0= T(v) = 𝑇 2 (w) = T(w) = v.
Hence, R  N= {0}.
Hence, V= R  N.

Note:

i) For any linear operator T,R and N are invariant under T.

ii) Let T be an idempotent linear operator on V. v ∈ R if v = T(w) for some w ∈
[Link],

𝑇𝑅 (v) = T(v) =𝑇 2 (w) = T(w) = v.

Therefore, 𝑇𝑅 = The identity linear operator.
Hence, 𝑇𝑁 = 0.

Definition-5.2.4: An idempotent linear operator T on a vector space V is called a Projection of

R along N.
𝑥 𝑥
Example-5.2.2: T: 𝑅 2 → 𝑅 2 , given by T(𝑦) = ( ). Clearly, T is an idempotent linear operator
0
1 0
R=[( )] , N= [( )].
0 1
Hence, V= R  N.

Theorems-5.2.2: Let V= a vector space over a field F.

(i) If V = 𝑊1  𝑊2  …  𝑊𝑘 , then there exist linear operators 𝑇1 , 𝑇2 ,…, 𝑇𝑘 on V such

that :
a) Each 𝑇𝑖 is a projection/i.e. 𝑇𝑖 2 = 𝑇𝑖 /
b) 𝑇𝑖  𝑇𝑗 = 0 if i≠j.
c) 𝑇1 + 𝑇2 +…+ 𝑇𝑘 = I = The identity linear operator.
d) Range 𝑇𝑖 = 𝑊𝑖 .
(ii) Conversely, if 𝑇1 , 𝑇2 ,…, 𝑇𝑘 are linear operator on V satisfying conditions (a),(b) and (c)
above, and if we let 𝑊𝑖 = Range 𝑇𝑖 , then V = 𝑊1  𝑊2  …  𝑊𝑘 .

Proof:

(i) Define Tj : V→V by Tj (v) = wj , where v = w1 + w2 +…+ wk .Clearly, Tj is a linear

operator.
Range Tj = Wj -------------(d)

88
 Linear Algebra II

Tj 2 = Tj -------------------------(a)
Null space of Tj = W1 + W2 +…+ Wj + Wj−1 + …+ Wk .
Furthermore, v = T1 (v)+ T2 (v) +…+ Tk (v).
Therefore, T1 + T2 +…+ Tk = I. --------------------(c)
If i≠j, then Ti  Tj = 0, because the Range Tj is the subspace of Wj which is contained
in the Null space of Ti . -------------------(b)
(ii) Clearly, V = W1 + W2 + …+ Wk , for by (c) , we have
v = T1 (v)+ T2 (v) +…+ Tk (v), for all v∈V and Tj (v) ∈ Wj . We need only show that
this expression for v is unique.
Suppose
v = w1 + w2 +…+ wk with wj ∈ Wj . Since Wj =Range Tj , wj = Tj (vj ) for some
vj ∈ V.
Then, Tj (v) = Tj (w1 + w2 +…+ wk ).
= Tj (w1 )+ Tj (w2 )+…+ Tj (wk ).
= Tj T1(v1 )+ Tj T2 (v2 )+…+ Tj Tk (vk ).
= Tj 2 ( vj ).
== Tj ( vj ).
= wj .

5.3. Primary Decomposition Theorem

Theorem: Let T be a linear operator on the finite-dimensional vector space V over the field F.
Let p be the minimal polynomial for T,

P= 𝑝1 𝑟1 𝑝2 𝑟2 .... 𝑝𝑘 𝑟𝑘 ;

Where the 𝑝𝑖 are distinct irreducible monic polynomials over F and ri are positive integers.

Let Wi = ker[pi ri (T)] be null space of pi ri (T), i= 1,2,…,k. Then,

(i) V = W1  W2  …  Wk ;
(ii) Each Wi is invariant under T;
(iii) If Ti is the operator induced on Wi by T, then the minimal polynomial for Ti is pi ri .

Proof: Reading assignment.

89