Home Catalog

Linear Algebra

Linear Systems
Summary

Vector Spaces
Summary

Maps Between Spaces
Summary

Determinants
Summary

Similarity
Summary

Linear Systems

Solving Linear Systems
Summary

Linear Geometry of n-Space
Summary

Reduced Echelon Form
Summary

Topics about Linear Systems
Summary

Vector Spaces

Definition of Vector Spaces
Summary

Linear Independence
Summary

Basis and Dimensions
Summary

Topics about Vector Spaces
Summary

Maps Between Spaces

Isomorphisms
Summary

Homomorphisms
Summary

Computing Linear Maps
Summary

Matrix Operations
Summary

Change of Basis
Summary

Projections
Summary

Topics about Maps Between Spaces
Summary

Determinants

Definition
Summary

Geometry of Determinants
Summary

Other Formulas for Determinants
Summary

Topics about Determinants
Summary

Similarity

Complex Vector Spaces
Summary

Similarity
Summary

Nilpotence
Summary

Jordan Form
Summary

Topics about Similarity
Summary

Solving Linear Systems

Systems of linear equations are common in science and mathematics. These two examples from high school science (O'Nan 1990) give a sense of how they arise.

The first example is from Physics. Suppose that we are given three objects, one with a mass known to be 2 kg, and are asked to find the unknown masses. Suppose further that experimentation with a meter stick produces these two balances.

Linalg balance 1.pngLinalg balance 2.png

Since the sum of moments on the left of each balance equals the sum of moments on the right (the moment of an object is its mass times its distance from the balance point), the two balances give this system of two equations.

The second example of a linear system is from Chemistry. We can mix, under controlled conditions, toluene C7H8 and nitric acid HNO3 to produce trinitrotoluene C7H5O6N3 along with the byproduct water (conditions have to be controlled very well, indeed— trinitrotoluene is better known as TNT). In what proportion should those components be mixed? The number of atoms of each element present before the reaction

20ad415c0f6aa0885ffb9a572d1c7ee6.png

must equal the number present afterward. Applying that principle to the elements C, H, N, and O in turn gives this system.

87e80184d4d73c0bb49cebfba14e3edf.png

To finish each of these examples requires solving a system of equations. In each, the equations involve only the first power of the variables. This chapter shows how to solve any such system.

Gauss' Method
If a linear system is changed to another by one of these operations
     1. an equation is swapped with another
    2. an equation has both sides multiplied by a nonzero constant
    3. an equation is replaced by the sum of itself and a multiple of another
 then the two systems have the same set of solutions.

Gauss' method uses these three row operations to set a system up for back substitution. If any step shows a contradictory equation then we can stop with the conclusion that the system has no solutions. If we reach echelon form without a contradictory equation, and each variable is a leading variable in its row, then the system has a unique solution and we find it by back substitution. Finally, if we reach echelon form without a contradictory equation, and there is not a unique solution (at least one variable is not a leading variable) then the system has many solutions.

Describing the Solution Set
Summary

General = Particular + Homogenous
Summary

Comparing Set Descriptions
Summary

Automation
Summary

Linear Geometry of n-Space

In the first section, we had to do a bit of work to show that there are only three types of solution sets— singleton, empty, and infinite. But in the special case of systems with two equations and two unknowns this is easy to see. Draw each two-unknowns equation as a line in the plane and then the two lines could have a unique intersection, be parallel, or be the same line.

Unique solution

Linalg unique solution.png

3x + 2y = 7
x - y = -1

No solutions

Linalg no solutions.png

3x + 2y = 7
3x + 2y = 4

Infinitely many solutions

Linalg infinitely many solutions.png

3x + 2y = 7
6x + 4y = 14

These pictures don't prove the results from the prior section, which apply to any number of linear equations and any number of unknowns, but nonetheless they do help us to understand those results. This section develops the ideas that we need to express our results from the prior section, and from some future sections, geometrically. In particular, while the two-dimensional case is familiar, to extend to systems with more than two unknowns we shall need some higher-dimensional geometry.

Vectors in Space
Summary

Length and Angle Measures
Summary

Reduced Echelon Form

After developing the mechanics of Gauss' method, we observed that it can be done in more than one way. One example is that we sometimes have to swap rows and there can be more than one row to choose from. Another example is that from this matrix

begin{pmatrix} 2  &2   4  &3 end{pmatrix}

Gauss' method could derive any of these echelon form matrices.

begin{pmatrix} 2  &2   0  &-1 end{pmatrix} qquad begin{pmatrix} 1  &1   0  &-1 end{pmatrix} qquad begin{pmatrix} 2  &0   0  &-1 end{pmatrix}

The first results from -2ρ1 + ρ2. The second comes from following (1/2)ρ1 with -4ρ1 + ρ2. The third comes from -2ρ1 + ρ2 followed by 2ρ2 + ρ1 (after the first pivot the matrix is already in echelon form so the second one is extra work but it is nonetheless a legal row operation).

The fact that the echelon form outcome of Gauss' method is not unique leaves us with some questions. Will any two echelon form versions of a system have the same number of free variables? Will they in fact have exactly the same variables free? In this section we will answer both questions "yes". We will do more than answer the questions. We will give a way to decide if one linear system can be derived from another by row operations. The answers to the two questions will follow from this larger result.

Gauss-Jordan Reduction
Summary

Row Equivalence
Summary

Topics about Linear Systems

Computer Algebra Systems
Summary

Input-Output Analysis
Summary

Accuracy of Computations
Summary

Analyzing Networks
Summary

Speed of Gauss' Method
Summary

Computer Algebra Systems

Content

Input-Output Analysis

Content

Accuracy of Computations

Content

Analyzing Networks

Content

Speed of Gauss' Method

Content

Gauss' Method

Definition 1.1

A linear equation in variables x1 , x2 , ... , xn has the form

a_1x_1+a_2x_2+a_3x_3+cdots+a_nx_n=d

where the numbers a1 , a2 , ... , an ∈ ℜ are the equation's coefficients and d ∈ ℜ is the constant. An n-tuple (s1 , s2 , ... , sn) ∈ ℜ is a solution of, or satisfies, that equation if substituting the numbers s1 , ... , sn for the variables gives a true statement:

a_1s_1+a_2s_2+ldots+a_ns_n=d

A system of linear equations

begin{array}{*{4}{rc}r} a_{1,1}x_1 &+ &a_{1,2}x_2  &+  &cdots &+ &a_{1,n}x_n &=  &d_1   a_{2,1}x_1 &+ &a_{2,2}x_2  &+  &cdots &+ &a_{2,n}x_n &=  &d_2   &  &            &   &       &  &           &vdots    a_{m,1}x_1 &+ &a_{m,2}x_2  &+  &cdots &+ &a_{m,n}x_n &=  &d_m end{array}

has the solution (s1 , s2 , ... , sn) if that n-tuple is a solution of all of the equations in the system.


Example 1.2

The ordered pair (-1 , 5) is a solution of this system

begin{array}{*{2}{rc}r} 3x_1 &+ &2x_2 &= &7   -x_1 &+ &x_2  &= &6 end{array}

In contrast, (5 , -1) is not a solution.


Finding the set of all solutions is solving the system. No guesswork or good fortune is needed to solve a linear system. There is an algorithm that always works. The next example introduces that algorithm, called Gauss' method. It transforms the system, step by step, into one with a form that is easily solved.

Example 1.3

To solve this system

begin{array}{*{3}{rc}r} &   &      &   &3x_3  &=  &9   x_1 &+  &5x_2  &-  &2x_3  &=  &2   frac{1}{3}x_1 &+  &2x_2  &   &      &=  &3   end{array}

we repeatedly transform it until it is in a form that is easy to solve.

begin{array}{rcl} quad &xrightarrow[]{ text{swap row 1 with row 3} } &begin{array}{*{3}{rc}r} frac{1}{3}x_1 &+  &2x_2  &   &      &=  &3   x_1 &+  &5x_2  &-  &2x_3  &=  &2   &   &      &   &3x_3  &=  &9   end{array}                                          &xrightarrow[]{ text{multiply row 1 by 3} } &begin{array}{*{3}{rc}r} x_1 &+  &6x_2  &   &      &=  &9   x_1 &+  &5x_2  &-  &2x_3  &=  &2   &   &      &   &3x_3  &=  &9   end{array}                                           &xrightarrow[]{ text{add }-1text{ times row 1 to row 2} } &begin{array}{*{3}{rc}r} x_1 &+  &6x_2  &   &      &=  &9   &   &-x_2  &-  &2x_3  &=  &-7  &   &      &   &3x_3  &=  &9   end{array} end{array}

The third step is the only nontrivial one. We've mentally multiplied both sides of the first row by -1, mentally added that to the old second row, and written the result in as the new second row.

Now we can find the value of each variable. The bottom equation shows that x3 = 3. Substituting 3 for x3 in the middle equation shows that x2 = 1. Substituting those two into the top equation gives that x1 = 3 and so the system has a unique solution: the solution set is { ( 3 , 1 , 3 ) }.


Most of this subsection and the next one consists of examples of solving linear systems by Gauss' method. We will use it throughout this book. It is fast and easy. But, before we get to those examples, we will first show that this method is also safe in that it never loses solutions or picks up extraneous solutions.


Theorem 1.4 (Gauss' method)

If a linear system is changed to another by one of these operations

  1. an equation is swapped with another
  2. an equation has both sides multiplied by a nonzero constant
  3. an equation is replaced by the sum of itself and a multiple of another

then the two systems have the same set of solutions.


Each of those three operations has a restriction. Multiplying a row by 0 is not allowed because obviously that can change the solution set of the system. Similarly, adding a multiple of a row to itself is not allowed because adding -1 times the row to itself has the effect of multiplying the row by 0. Finally, swapping a row with itself is disallowed to make some results in the fourth chapter easier to state and remember (and besides, self-swapping doesn't accomplish anything).


Proof

We will cover the equation swap operation here and save the other two cases for Problem 14.

Consider this swap of row i with row j

begin{array}{*{4}{rc}r} a_{1,1}x_1  &+  &a_{1,2}x_2 &+  &cdots  &&a_{1,n}x_n  &=  &d_1   &   &           &   &        &   &            &vdots    a_{i,1}x_1  &+  &a_{i,2}x_2 &+  &cdots  &&a_{i,n}x_n  &=  &d_i   &   &           &   &        &   &            &vdots    a_{j,1}x_1  &+  &a_{j,2}x_2 &+  &cdots  &&a_{j,n}x_n  &=  &d_j   &   &           &   &        &   &            &vdots    a_{m,1}x_1  &+  &a_{m,2}x_2 &+  &cdots  &&a_{m,n}x_n  &=  &d_m   end{array} xrightarrow[]{} begin{array}{*{4}{rc}r} a_{1,1}x_1  &+  &a_{1,2}x_2 &+  &cdots  &&a_{1,n}x_n  &=  &d_1   &   &           &   &        &   &            &vdots    a_{j,1}x_1  &+  &a_{j,2}x_2 &+  &cdots  &&a_{j,n}x_n  &=  &d_j   &   &           &   &        &   &            &vdots    a_{i,1}x_1  &+  &a_{i,2}x_2 &+  &cdots  &&a_{i,n}x_n  &=  &d_i   &   &           &   &        &   &            &vdots    a_{m,1}x_1  &+  &a_{m,2}x_2 &+  &cdots  &&a_{m,n}x_n  &=  &d_m   end{array}

The n-tuple (s1, s2, ... , sn) satisfies the system before the swap if and only if substituting the values, the s's, for the variables, the x's, gives true statements:

a_{1,1}s_1+a_{1,2}s_2+cdots+a_{1,n}s_n=d_1

and ...

a_{i,1}s_1+a_{i,2}s_2+cdots+a_{i,n}s_n=d_i

and ...

a_{j,1}s_1+a_{j,2}s_2+cdots+a_{j,n}s_n=d_j

and ...

a_{m,1}s_1+a_{m,2}s_2+cdots+a_{m,n}s_n=d_m

This is exactly the requirement that (s1 , s2 , ... , sn) solves the system after the row swap.


Definition 1.5

The three operations from Theorem 1.4 are the elementary reduction operations, or row operations, or Gaussian operations. They are swapping, multiplying by a scalar or rescaling, and pivoting.


When writing out the calculations, we will abbreviate "row i" by "ρi". For instance, we will denote a pivot operation by kρi + ρj, with the row that is changed written second. We will also, to save writing, often list pivot steps together when they use the same ρi.


Example 1.6

A typical use of Gauss' method is to solve this system.

begin{array}{*{3}{rc}r} x  &+  &y  &   &   &=  &0   2x  &-  &y  &+  &3z &=  &3   x  &-  &2y &-  &z  &=  &3   end{array}

The first transformation of the system involves using the first row to eliminate the x in the second row and the x in the third. To get rid of the second row's 2x, we multiply the entire first row by -2, add that to the second row, and write the result in as the new second row. To get rid of the third row's x, we multiply the first row by -1, add that to the third row, and write the result in as the new third row

begin{array}{rcl} &xrightarrow[-rho_1 +rho_3]{-2rho_1 +rho_2} &begin{array}{*{3}{rc}r} x  &+  &y  &   &   &=  &0   &   &-3y&+  &3z &=  &3   &   &-3y&-  &z  &=  &3   end{array}    end{array}

(Note that the two ρ1 steps -2ρ1 + ρ2 and -ρ1 + ρ3 are written as one operation.) In this second system, the last two equations involve only two unknowns. To finish we transform the second system into a third system, where the last equation involves only one unknown. This transformation uses the second row to eliminate y from the third row.

begin{array}{rcl} &xrightarrow[]{-rho_2 +rho_3} &begin{array}{*{3}{rc}r} x  &+  &y  &   &   &=  &0   &   &-3y&+  &3z &=  &3   &   &   &   &-4z&=  &0 end{array} end{array}

Now we are set up for the solution. The third row shows that z = 0. Substitute that back into the second row to get y = -1, and then substitute back into the first row to get x = 1.


Example 1.7

For the Physics problem from the start of this chapter, Gauss' method gives this.

begin{array}{rcl} begin{array}{*{2}{rc}r} 40h  &+  &15c  &=  &100       -50h &+  &25c  &=  &50          end{array} &xrightarrow[]{5/4rho_1 +rho_2} &begin{array}{*{2}{rc}r} 40h  &+  &15c       &=  &100       &   &(175/4)c  &=  &175  end{array} end{array}

So c = 4, and back-substitution gives that h = 1. (The Chemistry problem is solved later.)


Example 1.8

The reduction

begin{array}{rcl} begin{array}{*{3}{rc}r} x  &+  &y  &+  &z  &=  &9   2x  &+  &4y &-  &3z &=  &1   3x  &+  &6y &-  &5z &=  &0   end{array} &xrightarrow[-3rho_1 +rho_3]{-2rho_1 +rho_2} &begin{array}{*{3}{rc}r} x  &+  &y  &+  &z  &=  &9   &   &2y &-  &5z &=  &-17 &   &3y &-  &8z&=  &-27 end{array}                                     &xrightarrow[]{-(3/2)rho_2+rho_3} &begin{array}{*{3}{rc}r} x  &+  &y  &+  &z            &=  &9   &   &2y &-  &5z           &=  &-17 &   &   &   &-(1/2)z      &=  &-(3/2)  end{array} end{array}

shows that z = 3 , y = -1 , and x = 7.


As these examples illustrate, Gauss' method uses the elementary reduction operations to set up back-substitution.


Definition 1.9

In each row, the first variable with a nonzero coefficient is the row's leading variable. A system is in echelon form if each leading variable is to the right of the leading variable in the row above it (except for the leading variable in the first row).


Example 1.10

The only operation needed in the examples above is pivoting. Here is a linear system that requires the operation of swapping equations. After the first pivot

begin{array}{rcl} begin{array}{*{4}{rc}r} x  &-  &y  &   &   &   &   &=  &0   2x  &-  &2y &+  &z  &+  &2w &=  &4   &   &y  &   &   &+  &w  &=  &0   &   &   &   &2z &+  &w  &=  &5   end{array} &xrightarrow[]{-2rho_1 +rho_2} &begin{array}{*{4}{rc}r} x  &-  &y  &   &   &   &   &=  &0   &   &   &   &z  &+  &2w &=  &4   &   &y  &   &   &+  &w  &=  &0   &   &   &   &2z &+  &w  &=  &5   end{array}     end{array}

the second equation has no leading y. To get one, we look lower down in the system for a row that has a leading y and swap it in.

begin{array}{rcl} &xrightarrow[]{rho_2 leftrightarrowrho_3} &begin{array}{*{4}{rc}r} x  &-  &y  &   &   &   &   &=  &0   &   &y  &   &   &+  &w  &=  &0   &   &   &   &z  &+  &2w &=  &4   &   &   &   &2z &+  &w  &=  &5   end{array}     end{array}

(Had there been more than one row below the second with a leading y then we could have swapped in any one.) The rest of Gauss' method goes as before.

begin{array}{rcl} &xrightarrow[]{-2rho_3 +rho_4} &begin{array}{*{4}{rc}r} x  &-  &y  &   &   &   &   &=  &0   &   &y  &   &   &+  &w  &=  &0   &   &   &   &z  &+  &2w &=  &4   &   &   &   &   &   &-3w&=  &-3  end{array} end{array}

Back-substitution gives w = 1 , z = 2 , y = -1 , and x = -1 .


Strictly speaking, the operation of rescaling rows is not needed to solve linear systems. We have included it because we will use it later in this chapter as part of a variation on Gauss' method, the Gauss-Jordan method.

All of the systems seen so far have the same number of equations as unknowns. All of them have a solution, and for all of them there is only one solution. We finish this subsection by seeing for contrast some other things that can happen.


Example 1.11

Linear systems need not have the same number of equations as unknowns. This system

begin{array}{*{2}{rc}r} x  &+  &3y  &=  &1   2x  &+  &y   &=  &-3  2x  &+  &2y  &=  &-2  end{array}

has more equations than variables. Gauss' method helps us understand this system also, since this

begin{array}{rcl} &xrightarrow[-2rho_1 +rho_3]{-2rho_1 +rho_2} &begin{array}{*{2}{rc}r} x  &+  &3y  &=  &1   &   &-5y &=  &-5  &   &-4y &=  &-4  end{array} end{array}

shows that one of the equations is redundant. Echelon form

begin{array}{rcl} &xrightarrow[]{-(4/5)rho_2 +rho_3} &begin{array}{*{2}{rc}r} x  &+  &3y  &=  &1   &   &-5y &=  &-5  &   &0   &=  &0   end{array} end{array}

gives y = 1 and x = -2 . The "0 = 0" is derived from the redundancy.


That example's system has more equations than variables. Gauss' method is also useful on systems with more variables than equations. Many examples are in the next subsection.

Another way that linear systems can differ from the examples shown earlier is that some linear systems do not have a unique solution. This can happen in two ways.

The first is that it can fail to have any solution at all.


Example 1.12

Contrast the system in the last example with this one.

begin{array}{rcl} begin{array}{*{2}{rc}r} x  &+  &3y  &=  &1   2x  &+  &y   &=  &-3  2x  &+  &2y  &=  &0   end{array} &xrightarrow[-2rho_1 +rho_3]{-2rho_1 +rho_2} &begin{array}{*{2}{rc}r} x  &+  &3y  &=  &1   &   &-5y &=  &-5  &   &-4y &=  &-2 end{array} end{array}

Here the system is inconsistent: no pair of numbers satisfies all of the equations simultaneously. Echelon form makes this inconsistency obvious.

begin{array}{rcl} &xrightarrow[]{-(4/5)rho_2 +rho_3} &begin{array}{*{2}{rc}r} x  &+  &3y  &=  &1   &   &-5y &=  &-5  &   &0   &=  &2  end{array} end{array}

The solution set is empty.


Example 1.13

The prior system has more equations than unknowns, but that is not what causes the inconsistency— Example 1.11 has more equations than unknowns and yet is consistent. Nor is having more equations than unknowns sufficient for inconsistency, as is illustrated by this inconsistent system with the same number of equations as unknowns.

begin{array}{rcl} begin{array}{*{2}{rc}r} x  &+  &2y  &=  &8   2x  &+  &4y  &=  &8   end{array} &xrightarrow[]{-2rho_1 + rho_2} &begin{array}{*{2}{rc}r} x  &+  &2y  &=  &8   &   &0   &=  &-8 end{array} end{array}


The other way that a linear system can fail to have a unique solution is to have many solutions.


Example 1.14

In this system

begin{array}{*{2}{rc}r} x  &+  &y   &=  &4   2x  &+  &2y  &=  &8   end{array}

any pair of numbers satisfying the first equation automatically satisfies the second. The solution set {(x,y)| x + y = 4 } is infinite; some of its members are (0 , 4) , (-1 , 5) , and (2.5 , 1.5). The result of applying Gauss' method here contrasts with the prior example because we do not get a contradictory equation.

begin{array}{rcl} &xrightarrow[]{-2rho_1 + rho_2} &begin{array}{*{2}{rc}r} x  &+  &y   &=  &4   &   &0   &=  &0   end{array} end{array}


Don't be fooled by the "0 = 0" equation in that example. It is not the signal that a system has many solutions.


Example 1.15

The absence of a "0 = 0" does not keep a system from having many different solutions. This system is in echelon form

begin{array}{*{3}{rc}r} x  &+  &y  &+  &z  &=  &0   &   &y  &+  &z  &=  &0   end{array}

has no "0 = 0", and yet has infinitely many solutions. (For instance, each of these is a solution: (0 , 1, -1) , (0 , 1/2 , -1/2 ) ,  (0 , 0 , 0) , and (0 , -π , π) . There are infinitely many solutions because any triple whose first component is 0 and whose second component is the negative of the third is a solution.)

Nor does the presence of a "0 = 0" mean that the system must have many solutions. Example 1.11 shows that. So does this system, which does not have many solutions— in fact it has none— despite that when it is brought to echelon form it has a "0 = 0" row.

begin{array}{rcl} begin{array}{*{3}{rc}r} 2x  &   &   &-   &2z  &=  &6   &   &y  &+   &z   &=  &1   2x  &+  &y  &-   &z   &=  &7   &   &3y &+   &3z  &=  &0   end{array} &xrightarrow[]{-rho_1 +rho_3} &begin{array}{*{3}{rc}r} 2x  &   &   &-   &2z  &=  &6   &   &y  &+   &z   &=  &1   &   &y  &+   &z   &=  &1   &   &3y &+   &3z  &=  &0   end{array}      &xrightarrow[-3rho_2 +rho_4]{-rho_2 +rho_3} &begin{array}{*{3}{rc}r} 2x  &   &   &-   &2z  &=  &6   &   &y  &+   &z   &=  &1   &   &   &    &0   &=  &0   &   &   &    &0   &=  &-3  end{array} end{array}


We will finish this subsection with a summary of what we've seen so far about Gauss' method.

Gauss' method uses the three row operations to set a system up for back substitution. If any step shows a contradictory equation then we can stop with the conclusion that the system has no solutions. If we reach echelon form without a contradictory equation, and each variable is a leading variable in its row, then the system has a unique solution and we find it by back substitution. Finally, if we reach echelon form without a contradictory equation, and there is not a unique solution (at least one variable is not a leading variable) then the system has many solutions.

The next subsection deals with the third case— we will see how to describe the solution set of a system with many solutions.

Exercises
Summary

Describing the Solution Set

Content

General = Particular + Homogenous

Content

Comparing Set Descriptions

Content

Automation

Content

Exercises

Content

Definition of Vector Spaces

We shall study structures with two operations, an addition and a scalar multiplication, that are subject to some simple conditions. We will reflect more on the conditions later, but on first reading notice how reasonable they are. For instance, surely any operation that can be called an addition (e.g., column vector addition, row vector addition, or real number addition) will satisfy all the conditions in Definition 1.1 below.

Definition and Examples
Summary

Subspaces and Spanning sets
Summary

Linear Independence

The prior section shows that a vector space can be understood as an unrestricted linear combination of some of its elements— that is, as a span. For example, the space of linear polynomials { a + bx | a , b } ∈ ℜ is spanned by the set { 1 , x }. The prior section also showed that a space can have many sets that span it. The space of linear polynomials is also spanned by { 1 , 2x } and { 1 , x , 2x } .

At the end of that section we described some spanning sets as "minimal", but we never precisely defined that word. We could take "minimal" to mean one of two things. We could mean that a spanning set is minimal if it contains the smallest number of members of any set with the same span. With this meaning { 1 , x , 2x } is not minimal because it has one member more than the other two. Or we could mean that a spanning set is minimal when it has no elements that can be removed without changing the span. Under this meaning { 1 , x , 2x } is not minimal because removing the 2x and getting { 1 , x } leaves the span unchanged.

The first sense of minimality appears to be a global requirement, in that to check if a spanning set is minimal we seemingly must look at all the spanning sets of a subspace and find one with the least number of elements. The second sense of minimality is local in that we need to look only at the set under discussion and consider the span with and without various elements. For instance, using the second sense, we could compare the span of { 1 , x , 2x } with the span of { 1 , x }   and note that the 2x is a "repeat" in that its removal doesn't shrink the span.

In this section we will use the second sense of "minimal spanning set" because of this technical convenience. However, the most important result of this book is that the two senses coincide; we will prove that in the section after this one.

Definition and Examples
Summary

Basis and Dimensions

The prior section ends with the statement that a spanning set is minimal when it is linearly independent and a linearly independent set is maximal when it spans the space. So the notions of minimal spanning set and maximal independent set coincide. In this section we will name this idea and study its properties.

Basis
Summary

Dimension
Summary

Vector Spaces and Linear Systems
Summary

Combining Subspaces
Summary

Topics about Vector Spaces

Fields
Summary

Crystals
Summary

Voting Paradoxes
Summary

Dimensional Analysis
Summary

Vectors in Space

Content

Length and Angle Measures

Content

Gauss-Jordan Reduction

Content

Row Equivalence

Content

Definition and Examples

Content

Subspaces and Spanning sets

Content

Definition and Examples

Content

Basis

Content

Dimension

Content

Vector Spaces and Linear Systems

Content

Combining Subspaces

Content

Fields

Content

Crystals

Content

Voting Paradoxes

Content

Dimensional Analysis

Content

Isomorphisms

In the examples following the definition of a vector space we developed the intuition that some spaces are "the same" as others. For instance, the space of two-tall column vectors and the space of two-wide row vectors are not equal because their elements—column vectors and row vectors—are not equal, but we have the idea that these spaces differ only in how their elements appear. We will now make this idea precise.

This section illustrates a common aspect of a mathematical investigation. With the help of some examples, we've gotten an idea. We will next give a formal definition, and then we will produce some results backing our contention that the definition captures the idea. We've seen this happen already, for instance, in the first section of the Vector Space chapter. There, the study of linear systems led us to consider collections closed under linear combinations. We defined such a collection as a vector space, and we followed it with some supporting results.

Of course, that definition wasn't an end point, instead it led to new insights such as the idea of a basis. Here too, after producing a definition, and supporting it, we will get two surprises (pleasant ones). First, we will find that the definition applies to some unforeseen, and interesting, cases. Second, the study of the definition will lead to new ideas. In this way, our investigation will build a momentum.

Definition and Examples
Summary

Dimension Characterizes Isomorphism
Summary

Homomorphisms

The definition of isomorphism has two conditions. In this section we will consider the second one, that the map must preserve the algebraic structure of the space. We will focus on this condition by studying maps that are required only to preserve structure; that is, maps that are not required to be correspondences.

Experience shows that this kind of map is tremendously useful in the study of vector spaces. For one thing, as we shall see in the second subsection below, while isomorphisms describe how spaces are the same, these maps describe how spaces can be thought of as alike.

Definition of Homomorphism
Summary

Rangespace and Nullspace
Summary

Computing Linear Maps

The prior section shows that a linear map is determined by its action on a basis. In fact, the equation

h(vec{v}) =h(c_1cdotvec{beta}_1+dots+c_ncdotvec{beta}_n) =c_1cdot h(vec{beta}_1)+dots +c_ncdot h(vec{beta}_n)

shows that, if we know the value of the map on the vectors in a basis, then we can compute the value of the map on any vector

vec{v}

at all. We just need to find the

c

's to express

vec{v}

with respect to the basis.

This section gives the scheme that computes, from the representation of a vector in the domain

{rm Rep}_{B}(vec{v})

, the representation of that vector's image in the codomain

{rm   Rep}_{D}(h(vec{v}))

, using the representations of

h(vec{beta}_1)

, ...,

h(vec{beta}_n)

.

Representing Linear Maps with Matrices
Summary

Any Matrix Represents a Linear Map
Summary

Matrix Operations

The prior section shows how matrices represent linear maps. A good strategy, on seeing a new idea, is to explore how it interacts with some already-established ideas. In the first subsection we will ask how the representation of the sum of two maps f + g is related to the representations of the two maps, and how the representation of a scalar product r · h of a map is related to the representation of that map. In later subsections we will see how to represent map composition and map inverse.

Sums and Scalar Products
Summary

Matrix Multiplication
Summary

Mechanics of Matrix Multiplication
Summary

Inverses
Summary

Change of Basis

Representations, whether of vectors or of maps, vary with the bases. For instance, with respect to the two bases

mathcal{E}_2

and

B=langle begin{pmatrix} 1  1 end{pmatrix},begin{pmatrix} 1  -1 end{pmatrix} rangle

for

mathbb{R}^2

, the vector

vec{e}_1

has two different representations.

{rm Rep}_{mathcal{E}_2}(vec{e}_1)=begin{pmatrix} 1  0 end{pmatrix} qquad {rm Rep}_{B}(vec{e}_1)=begin{pmatrix} 1/2  1/2 end{pmatrix}

Similarly, with respect to

mathcal{E}_2,mathcal{E}_2

and

mathcal{E}_2,B

, the identity map has two different representations.

{rm Rep}_{mathcal{E}_2,mathcal{E}_2}(text{id})= begin{pmatrix} 1  &0   0  &1 end{pmatrix} qquad {rm Rep}_{mathcal{E}_2,B}(text{id})= begin{pmatrix} 1/2  &1/2   1/2  &-1/2 end{pmatrix}

With our point of view that the objects of our studies are vectors and maps, in fixing bases we are adopting a scheme of tags or names for these objects, that are convienent for computation. We will now see how to translate among these names— we will see exactly how representations vary as the bases vary.

Changing Representations of Vectors
Summary

Changing Map Representations
Summary

Projections

We have described the projection

pi

from

mathbb{R}^3

into its

xy

plane subspace as a "shadow map". This shows why, but it also shows that some shadows fall upward.

Linalg projection 1.png Linalg projection 2.png

So perhaps a better description is: the projection of

vec{v}

is the

vec{p}

in the plane with the property that someone standing on

vec{p}

and looking straight up or down sees

vec{v}

. In this section we will generalize this to other projections, both orthogonal (i.e., "straight up and down") and nonorthogonal.

Orthogonal Projection Onto a Line
Summary

Gram-Schmidt Orthogonalization
Summary

Projection Onto a Subspace
Summary

Topics about Maps Between Spaces

Line of Best Fit
Summary

Geometry of Linear Maps
Summary

Markov Chains
Summary

Orthonormal Matrices
Summary

Line of Best Fit

Content

Geometry of Linear Maps

Content

Markov Chains

Content

Orthonormal Matrices

Content

Definition and Examples

Content

Dimension Characterizes Isomorphism

Content

Definition of Homomorphism

Content

Rangespace and Nullspace

Content

Representing Linear Maps with Matrices

Content

Any Matrix Represents a Linear Map

Content

Sums and Scalar Products

Content

Matrix Multiplication

Content

Mechanics of Matrix Multiplication

Content

Inverses

Content

Changing Representations of Vectors

Content

Changing Map Representations

Content

Orthogonal Projection Onto a Line

Content

Gram-Schmidt Orthogonalization

Content

Projection Onto a Subspace

Content

Definition

For 1 × 1 matrices, determining nonsingularity is trivial.

begin{pmatrix} a end{pmatrix} is nonsingular iff a neq 0

The 2 × 2 formula came out in the course of developing the inverse.

begin{pmatrix} a  &b   c  &d end{pmatrix} is nonsingular iff ad-bc neq 0

The 3 × 3 formula can be produced similarly (see Problem 9).

begin{pmatrix} a  &b  &c   d  &e  &f   g  &h  &i end{pmatrix} is nonsingular iff aei+bfg+cdh-hfa-idb-gec neq 0

With these cases in mind, we posit a family of formulas,

a

,

ad-bc

, etc.

For each

n

the formula gives rise to a determinant function

detnolimits_{n ! times ! n}:mathcal{M}_{n ! times ! n}to mathbb{R}

such that an

n ! times ! n

matrix

T

is nonsingular if and only if

detnolimits_{n ! times ! n}(T)neq 0

. (We usually omit the subscript because if

T

is

n ! times ! n

then "

det(T)

" could only mean "

detnolimits_{n ! times ! n}(T)

".)

Exploration
Summary

Properties of Determinants
Summary

The Permutation Expansion
Summary

Determinants Exist
Summary

Geometry of Determinants

The prior section develops the determinant algebraically, by considering what formulas satisfy certain properties. This section complements that with a geometric approach. One advantage of this approach is that, while we have so far only considered whether or not a determinant is zero, here we shall give a meaning to the value of that determinant. (The prior section handles determinants as functions of the rows, but in this section columns are more convenient. The final result of the prior section says that we can make the switch.)

Determinants as Size Functions
Summary

Other Formulas for Determinants

Determinants are a fount of interesting and amusing formulas. Here is one that is often seen in calculus classes and used to compute determinants by hand.

Laplace's Expansion
Summary

Topics about Determinants

Cramer's Rule
Summary

Speed of Calculating Determinants
Summary

Projective Geometry
Summary

Cramer's Rule

Content

Speed of Calculating Determinants

Content

Projective Geometry

Content

Exploration

Content

Properties of Determinants

Content

The Permutation Expansion

Content

Determinants Exist

Content

Determinants as Size Functions

Content

Laplace's Expansion

Content

Complex Vector Spaces

This chapter requires that we factor polynomials. Of course, many polynomials do not factor over the real numbers; for instance, x2 + 1 does not factor into the product of two linear polynomials with real coefficients. For that reason, we shall from now on take our scalars from the complex numbers.

That is, we are shifting from studying vector spaces over the real numbers to vector spaces over the complex numbers— in this chapter vector and matrix entries are complex.

Any real number is a complex number and a glance through this chapter shows that most of the examples use only real numbers. Nonetheless, the critical theorems require that the scalars be complex numbers, so the first section below is a quick review of complex numbers.

In this book we are moving to the more general context of taking scalars to be complex only for the pragmatic reason that we must do so in order to develop the representation. We will not go into using other sets of scalars in more detail because it could distract from our goal. However, the idea of taking scalars from a structure other than the real numbers is an interesting one. Delightful presentations taking this approach are in (Halmos 1958) and (Hoffman & Kunze 1971).

Factoring and Complex Numbers: A Review
Summary

Complex Representations
Summary

Similarity

Definition and Examples
Summary

Diagonalizability
Summary

Eigenvalues and Eigenvectors
Summary

Nilpotence

The goal of this chapter is to show that every square matrix is similar to one that is a sum of two kinds of simple matrices. The prior section focused on the first kind, diagonal matrices. We now consider the other kind.

Self-Composition
Summary

Strings
Summary

Jordan Form

This section uses material from three optional subsections: Direct Sum, Determinants Exist, and Other Formulas for the Determinant.

The chapter on linear maps shows that every

h:Vto W

can be represented by a partial-identity matrix with respect to some bases

Bsubset V

and

Dsubset W

. This chapter revisits this issue in the special case that the map is a linear transformation

t:Vto V

. Of course, the general result still applies but with the codomain and domain equal we naturally ask about having the two bases also be equal. That is, we want a canonical form to represent transformations as

{rm Rep}_{B,B}(t)

.

After a brief review section, we began by noting that a block partial identity form matrix is not always obtainable in this

B,B

case. We therefore considered the natural generalization, diagonal matrices, and showed that if its eigenvalues are distinct then a map or matrix can be diagonalized. But we also gave an example of a matrix that cannot be diagonalized and in the section prior to this one we developed that example. We showed that a linear map is nilpotent— if we take higher and higher powers of the map or matrix then we eventually get the zero map or matrix— if and only if there is a basis on which it acts via disjoint strings. That led to a canonical form for nilpotent matrices.

Now, this section concludes the chapter. We will show that the two cases we've studied are exhaustive in that for any linear transformation there is a basis such that the matrix representation

{rm Rep}_{B,B}(t)

is the sum of a diagonal matrix and a nilpotent matrix in its canonical form.

Polynomials of Maps and Matrices
Summary

Jordan Canonical Form
Summary

Topics about Similarity

Geometry of Eigenvalues
Summary

The Method of Powers
Summary

Stable Populations
Summary

Linear Recurrences
Summary

Geometry of Eigenvalues

Content

The Method of Powers

Content

Stable Populations

Content

Linear Recurrences

Content

Factoring and Complex Numbers: A Review

Content

Complex Representations

Content

Definition and Examples

Content

Diagonalizability

Content

Eigenvalues and Eigenvectors

Content

Self-Composition

Content

Strings

Content

Polynomials of Maps and Matrices

Content

Jordan Canonical Form

Content