Optimization and algorithms

Course overview

João Xavier and Cláudia Soares

jxavier@[Link],[Link]@[Link]

Instituto Superior Técnico, Universidade de Lisboa

September 21, 2020

1 / 15
 Outline

• What is the goal of optimization?

• What is the most important fact about optimization?

• Your first optimization problem: where to place a fire station?

• How is your grade computed?

2 / 15
 What is the goal of optimization?

To save the world, one optimization at a time. . .

real-world problem

formulation

optimization problem

theory and algorithms

solution

3 / 15
Why study optimization?

Optimization is at the heart of numerous fields:

• communications
• control
• power systems
• computer vision
• machine learning
• finance
• networks
• data science
• ...

4 / 15
An optimization problem is a mathematical object of the following form:

minimize f (x)
x
subject to h1 (x) =0
..
.
hp (x) =0
g1 (x) ≤0
..
.
gm (x) ≤ 0

• x ∈ Rn is the optimization variable

• f : Rn → R is the objective or cost function that we want to
minimize
• h1 , . . . , hp , g1 , . . . , gm : Rn → R are constraint functions

5 / 15
Solving an optimization problem means finding a global minimizer

An unconstrained optimization problem:

cost function f

global minimizer (solution that we want)

local minimizer (“fake” minimizer)

6 / 15
 A constrained optimization problem:

cost function f

unfeasible point (no longer valid)

global minimizer (solution that we want)

constraint set (we can only search in here)

7 / 15
What is the most important fact about optimization?

1
100
0.8

0.6 80

0.4
60
0.2

0 40

−0.2
20
−0.4

−0.6
0
−0.8

−1 4 4
−20
3 2 3 2
2 2
1 0 1 0
0 0
−1 −2 −1 −2
−2 −2
−3 −4 −3 −4

Nonconvex problem Convex problem

• Algorithms that solve typical nonconvex problems are very slow

• Algorithms that solve typical convex problems are very fast

8 / 15
Your first optimization problem: where to place a fire
station?
• A fire station is going to serve K villages
• The K villages are located at given positions p1 , p2 , . . . , pK ∈ R2
• Where should you place the fire station?

p1

x =? pK
fire station

9 / 15
• A possible problem formulation is as follows:

minimize max {kx − p1 k , kx − p2 k , . . . , kx − pK k}

x | {z }
f (x)

with optimization variable x ∈ R2 (x is location of fire station)

• Is this a convex or a nonconvex problem?

10 / 15
1 % firestation.m; uses package CVX from [Link]
2 KK = 20; % choose K = number of villages
3 p1 = 0.5*randn(2,KK); % generate random positions
4 p2 = p1+[5 ; 0]*ones(1,KK);
5 p = [ p1 , p2 , [ 5 ; 5 ] ]; K = size(p,2);
6
7 % plot the villages
8 figure(1); clf;
9 plot(p(1,:),p(2,:),'ˆ','MarkerSize',8,'MarkerFaceColor','b');
10 grid on;
11
12 % solve the optimization problem
13 cvx begin quiet
14 variable x(2,1);
15
16 % build cost function
17 f = norm(x − p(:,1));
18 for i = 2:K
19 f = max(f,norm(x−p(:,i)));
20 end;
21
22 minimize(f);
23 cvx end;
24
25 %plot solution
26 hold on; plot(x(1),x(2),'rx','MarkerSize',15,'LineWidth',2);
11 / 15
• Here is a typical output:

−1

−2
−2 −1 0 1 2 3 4 5 6

12 / 15
The course consists of three parts:

• Part 1: the art of formulating optimization problems

• Part 2: unconstrained optimization

I how to tell if an unconstrained optimization problem is convex
I numerical algorithms for convex and nonconvex problems

• Part 3: constrained optimization

I how to tell if a constrained optimization problem is convex
I numerical algorithms for convex and nonconvex problems

13 / 15
 Grading

• Your grade is computed as follows:

grade = 50% project + 50% max{exam 1, exam 2}

• The minimum grade for either the project or the exam is 10 points

• Project is done by groups of four students

14 / 15
 Timeline for the project

• Sep. 25: We post the link for you to register your group

• Sep. 25–Sep. 28: You register your group

• Sep. 29: We remove the link for you to register your group

• Oct. 1: We post the list of groups and Part 1 of the project

• Oct. 21: We post part 2 of the project

• Nov. 14: We post part 3 of the project

• Dec. 8: You send us by email your project report (written in a

LaTeX template that we’ll provide)

• Dec. 10–Dec. 18: You present orally your project (20 min)

15 / 15