Math 2050 Fall 2022

Homework 1

Homework 1 is due by Friday, 28th Oct at the recitation class. The total number of points is 30.

1 Subspaces

1. (2 points) Consider the set S of points x ∈ R3 such that

x1 + 2x2 + 3x3 = 0, 3x1 + 2x2 + x3 = 0.

Show that S is a subspace. Determine its dimension, and find a basis for it.
Solution:
By solving the system of 2 linear equations, we obtain:
(
x1 = x3
x2 = −2x1
So for every x ∈ S, x = (x1 , −2x1 , x1 ) = x1 (1, −2, 1)
Therefore, S is a subspace of R3 , has dimension 1 and S = span(1, −2, 1).
2. (4 points) Consider the set in R3 , defined by the equation

P := {x ∈ R3 : x1 + 2x2 + 3x3 = 1}.

(a) Show that the set P is an affine subspace of dimension 2. To this end, express it as x0 +
span(x1 , x2 ), where x0 in P , and x1 , x2 are independent vectors.
Solution:
We have: x1 = 1 − 2x2 − 3x3
So x = (1 − 2x2 − 3x3 , x2 , x3 ) = (1, 0, 0) + x2 (−2, 1, 0) + x3 (−3, 0, 1)
Furthermore, the origin is not in P so P is an affine subspace of dimension 2.
(b) Find the minimum Euclidean distance from 0 to the set P . Find a point that achieves the
minimum distance. (Hint: using the Cauchy-Schwartz inequality, prove that the minimum-
distance point is proportional to a := (1, 2, 3)).
Solution:
We need to find the minimum of x21 + x22 + x23 such that x1 + 2x2 + 3x3 = 1
Apply Cauchy-Schwartz inequality for x = (x1 , x2 , x3 ) and a = (1, 2, 3):

(xT a)2 ≤ ∥x∥2 . ∥a∥2

This is equivalent to
(x1 + 2x2 + 3x3 )2 ≤ 14. ∥x∥2
√ √
Therefore ∥x∥ ≥ 1/ 14, and ∥x∥ = 1/ 14 when x proportional to a = (1, 2, 3).
We can find this point by assuming x is of the form x = α(1, 2, 3) for some α ∈ R, to be
1
determined as follows. Replace the coordinate of x in x1 + 2x2 + 3x3 = 1, we obtain α = 14
1
so x = 14 (1, 2, 3).

1
2 Generalized Cauchy-Schwartz inequalities

1. (2 points)
Show that the following inequalities hold for any vector x:

∥x∥∞ ≤ ∥x∥1 ≤ n ∥x∥∞ .

Solution:
This is obvious, it follows from the definition of these norms.

2. (2 points) Show that following inequalities hold for any vector x:

√
∥x∥2 ≤ ∥x∥1 ≤ n ∥x∥2 .
Hint: use the Cauchy-Schwartz inequality for the second inequality.
Solution:
For the first inequality, we just need to square both sides.
For the second inequality: Apply Cauchy-Schwartz inequality for x and a = (1, . . . , 1), we have:

|xT a| ≤ ∥x∥2 . ∥a∥2

This is equivalent to: √
∥x∥1 ≤ n ∥x∥2

3. (2 points) In a generalized version of the above inequalities, show that for any non-zero vector x,

∥x∥21
≤ Card(x),
∥x∥22
where Card(x) is the cardinality of the vector x, defined as the number of non-zero elements in x.
For which vectors x is the equality attained?
Solution:

Consider vector a such that, for i ∈ {1, . . . , n}:

(
ai = 1 if xi ̸= 0,
ai = 0 if xi = 0.

Apply Cauchy-Schwartz inequality to x and a, we have:

∥x∥21 ≤ ∥x∥22 .Card(x).

The equality attained when all the coordinates that different from 0 of x are equal.

2
3 Orthogonalization

(2 points) Let x, y ∈ Rn be two unit-norm vectors, that is, such that ∥x∥2 = ∥y∥2 = 1. Show that the
vectors x − y and x + y are orthogonal. Use this to find an orthogonal basis for the subspace spanned by
x and y.
Solution:
We calculate the inner product of x − y and x + y:

(x − y)T (x + y) = ∥x∥22 + (xT y − y T x) − ∥y∥22 = 0.

We also have
1
x = [(x + y) + (x − y)],
2
1
y = [(x + y) − (x − y)].
2
Therefore, x − y and x + y is an orthogonal basis for the subspace spanned by x and y.

3
4 Linear Functions

1. (2 points) For a n-vector x, with n = 2m − 1 odd, we define the median of x as xm . Now consider
the function f : Rn → R, with values

n
1X
f (x) = xm − xi .
n i=1

Express f as a scalar product, that is, find a in Rn such that f (x) = aT x for every x. Find a basis
for the set of points x such that f (x) = 0.
Solution:
Vector a satisfies:
(
− n1 if i ̸= m
ai = n−1
n
if i = m

For the second part, we have f (x) = 0 so:

n
1X
xm = xi
n i=1

equivalent to:
n
1 X
xm = xi
n − 1 i=1,i̸=m

Therefore, the basis of the space orthogonal to a is the set of n − 1 vectors which are the column
vectors of the following matrix (n × (n − 1) dimensions):
 
1 0 ... 0
 0 1 ... 0 
 
... ... ... ...
A=
 1 1 1 

 n−1 n−1
... n−1 
... ... ... ...
0 0 ... 1
1
where n−1
is the value for all elements of row m. For example, when n = 5:
 
1 0 0 0
0 1 0 0
1 1 1 1

A=
4 4 4 4
.
0 0 1 0 
0 0 0 1

2. (2 points) For α in R2 , we consider the ”power law” function f : R2++ → R, with values

f (x) = xα1 1 xα2 2 .

4
 Justify the statement: ”the coefficients αi provide the ratio between the relative error in f to a relative
error in xi ”.
Solution:
We have:
df ∆f
= α1 xα1 1 −1 xα2 2 =
dx1 ∆x1
Thus:
∆f x1 α1 .f
= = α1
f ∆x1 f
Similarly, we obtain:
∆f x2
= α2
f ∆x2

3. (2 points) Find the gradient of the function f : R2 → R that gives the distance from a given point p
in R2 to a point x in R2 .
Solution:
Let p = (p1 , p2 ), x = (x1 , x2 ). We have:
p
f (x1 , x2 ) = g(x), where g(x) := (x1 − p1 )2 + (x2 − p2 )2 .

We have
∇g = (2(x1 − p1 ), 2(x2 − p2 )) = 2(x − p).
Since g(x) = f (x)2 , we have
∇g(x) = 2f (x)∇f (x).
Therefore:
1 x−p
∇f = ((x1 − p1 ), (x2 − p2 )) = .
f ∥x − p∥

5
5 Norm and Distance

(1) (3 points) Nearest point to a line. Let a and b be the different n-vectors. The line passing through a
and b is given by the set of vectors of the norm (1 − θ)a + θb, where θ is a scalar that determines the
particular point on the line.
Let x be any n-vector. Find a formula for the point p on the line that is closest (in the Euclidean norm
sense) to x. The point p is called the projection of x onto the line. Show that (p − x) ⊥ (a − b), and
draw a simple picture illustrating this in 2-D. Hint. Work with the square of the distance between a
point on the line and x, i.e. ∥(1 − θ)a + θb − x∥2 . Expand this, and minimize over θ.
Solution:

Let:
f (θ) = ∥(1−θ)a+θb−x∥2 = (1−θ)2 ∥a∥2 +θ2 ∥b∥2 +∥x∥2 +2θ(1−θ)aT b−2(1−θ)aT x−2θbT x

f ′ (θ) = −2(1 − θ) ∥a∥2 + 2θ ∥b∥2 + 2abT (1 − 2θ) + 2axT − 2bxT

If f ′ = 0, we have:
(aT − xT )(a − b)
θ=
∥a − b∥2
Thus:
(xT − bT )(a − b)
1−θ =
∥a − b∥2
Inner product of a − b and p − x as follows:
(a − b)T (p − x) = (xT − bT )a + (aT − xT )b − (aT − bT )x = 0

(2) (3 points) Regression model sensitivity. Consider the regression model ŷ = xT β + v, where ŷ is the
prediction, x is a feature vector, β is a coefficient vector, and v is the offset term. If x and x̃ are
feature vectors with corresponding prediction ŷ and ỹ, show that |ŷ − ỹ| ≤ ∥β∥∥x − x̃∥. This means
that when ∥β∥ is small, the prediction is not very sensitive to a change in the feature vector.
Solution:
We have |ŷ − ỹ| = |(xT β + v) − (x̃T β + v)| = |(x − x̃)T β|. Apply the Cauchy-Schwartz inequality,
we obtain:
|ŷ − ỹ| = |(x − x̃)T β| ≤ ∥x − x̃∥ . ∥β∥ .

6
(3) (3 points) Changes in coefficient vector in a scoring function. We consider a scoring function f
defined as
f : x ∈ Rn → aT x + b
where a ∈ Rn , b ∈ R. Assume that the coefficients of the vector a are only known up to intervals.
Precisely, for every i ∈ {1, . . . , n}, we have |ai − âi | ≤ σi , where â ∈ Rn and σ ∈ Rn+ are given.
For a given x ∈ Rn , what are the corresponding largest and smallest values f (x) can take? It may
be useful to define |x|, the n-vector with entries equal to the absolute values in x.
Solution:
We just need to find the maximum and minimum of aT x. We have:

|ai − âi | ≤ σi , i = 1, . . . , n.

Multiplying both sides with |xi |, we obtain:

|(ai − âi )xi | ≤ σi |xi |, i = 1, . . . , n.

Thus:
âi xi − σi |xi | ≤ ai xi ≤ âi xi + σi |xi |, i = 1, . . . , n.
Summing:
n
X n
X n
X
(âi xi − σi |xi |) ≤ (ai xi ) ≤ (âi xi + σi |xi |).
i=1 i=1 i=1

Let |x| = (|x1 |, |x2 |, . . . , |xn |). We can write

âT x − σ T |x| ≤ aT x ≤ âT x + σ T |x|.