NUMERICAL ALGEBRA, doi:10.3934/naco.2012.2.

395
CONTROL AND OPTIMIZATION
Volume 2, Number 2, June 2012 pp. 395–412

A NONMONOTONE SPECTRAL PROJECTED GRADIENT

METHOD FOR LARGE-SCALE TOPOLOGY OPTIMIZATION
PROBLEMS

Rouhollah Tavakoli1
Department of Material Science and Engineering
Sharif University of Technology
Tehran, P.O. Box 11365-9466, Iran

Hongchao Zhang
Department of Mathematics
Louisiana State University
Baton Rouge, LA, 70808, USA

(Communicated by Xiaoling Sun)

Abstract. An efficient gradient-based method to solve the volume constrained

topology optimization problems is presented. Each iterate of this algorithm is
obtained by the projection of a Barzilai-Borwein step onto the feasible set con-
sisting of box and one linear constraints (volume constraint). To ensure the
global convergence, an adaptive nonmonotone line search is performed along
the direction that is given by the current and projection point. The adaptive
cyclic reuse of the Barzilai-Borwein step is applied as the initial stepsize. The
minimum memory requirement, the guaranteed convergence property, and al-
most only one function and gradient evaluations per iteration make this new
method very attractive within common alternative methods to solve large-scale
optimal design problems. Efficiency and feasibility of the presented method are
supported by numerical experiments.

1. Introduction. The goal of topology optimization is to find optimal materi-

al distribution in the given design domain subject to some constraints governed
by certain physical properties and/or some other practical constraints during the
design. In the past two decades, advances in the theory of homogenization, op-
timization, numerical analysis as well as newly developed engineering approaches
make topology optimization techniques to become a standard tool of engineering de-
sign, in particular in the field of structural mechanics. For more detailed literature
review, one may refer [1, 3] and the references therein. In topology optimization,
the design parameters are often material properties (e.g., conductivity or stiffness
tensor) and the objective is often to minimize an integral functional defined on the
spatial domain with state variables satisfying a partial differential equation (PDE)

2000 Mathematics Subject Classification. Primary: 90C06, 90C26; Secondary: 65Y20.

Key words and phrases. Barzilai-Borwein step-size, distributed parameter identification, large-
scale topology optimization, method of moving asymptotic (MMA), nonmonotone line search,
volume constraint.
The work of second author is supported by NSF grant 1016204.
1 The corresponding author.

395
396 ROUHOLLAH TAVAKOLI AND HONGCHAO ZHANG

corresponding to certain physical law. In addition, some bound constraints on the

design variables and usually a global material resource constraint are also often
enforced during the design.
One typical large class of topology optimization problems have the structure
that a nonlinear non-convex objective functional is minimized over a feasible region
defined by a second order elliptic PDE together with bilateral bound and a single
equality constraints. In this paper, we focus on solving this class of optimal de-
sign problems which have a wide range of applications in engineering design, such
as compliance mechanics, fluid dynamics, heat transfer, functionally graded and
composite materials [1, 3].
In the nonlinear optimization literature, there are many well-known optimization
methods for finding solutions of general optimization problem. Among them, the
sequential quadratic programming (SQP) [15] method is widely used and generally
considered to be an efficient method for smooth nonlinear optimization problems
with constraints. Trust region methods [8] are another class of well-studied meth-
ods which have strong global and local convergence properties. More recently,
interior-point methods receives much more attention because of their polynomial
complexity. However, most of the above mentioned methods require the evaluation
or a certain type of approximations of the hessian at each iteration. In addition,
these methods also often require to solve a linear system of equations (usually in-
definite, dense and is difficult to solve by iterative methods) to a certain level at
each iteration. Therefore, when the problem size is very large, which often occurs
after the discretization of the topology optimization problem, obtaining the Hessian
information as well as solving very large linear system of equations at each iteration
could be very expensive. Hence, although those second order methods enjoy fast
local convergence properties, they are generally not efficient to solve very large-
scale problems, especially when a solution with very high accuracy is not strictly
required.
On the other hand, some classes of optimization algorithms are developed with-
in engineering community to solve large-scale engineering design problems. The
first method of this kind is known as CONLIN or convex linearization [13]. More
advanced version of CONLIN, called method of moving asymptotes (MMA), was
introduced by Svanberg [22] . Within each iteration of this method the optimiza-
tion problem is approximated by a convex separable sub-problem, for which efficient
solvers are available. The globalization of the method is performed by either line-
search [25] procedures or conservative sub-iterations [23].
Despite the improvements of these methods for general structural optimization
problems, a key issue to design an efficient optimization method is to exploit the
specific structure of the desired problem. The optimality criteria (OC) method [3]
is the first method developed particularly to solve the resource constrained topology
optimization problems. Recently, OC becomes very popular and is the most widely
used method in the engineering community for such problems. However, OC is
not globally convergent and its application is limited to some special problems like
thermal/structural compliance minimization, see chapter 5 of [1]. Moreover, the
convergence rate of OC is not very promising.
By applying the recent techniques developed in the field of nonlinear optimization
(cf. [24]), in this paper we would like to design an optimization algorithm for solving
very large-scale topology optimization problems. Each iterate of this algorithm is
obtained by performing an adaptive nonmonotone line search along the line segment
 SPECTRAL PROJECTED GRADIENT METHOD 397

connected by the current point and the projection point of a cyclic Barzilai-Borwein
(CBB) step onto the feasible set. Hence, the method can be called the projected
cyclic Barzilai-Borwein (PCBB) method. Because of the special structure of the
feasible set of the problem, which consists of box constants and a single linear con-
straint, it is possible to do the projection on the feasible set very efficiently with
linear time complexity. The main attractive features of the presented algorithm are:
producing strictly feasible iterations; using almost one objective function and gra-
dient evaluations per iteration; O(n) memory consumption (6n working memory);
easily to be implemented; only first order (gradient) information being required.

2. Barzilai-Borwein methods. Consider to solve the following finite-dimensional

unconstrained optimization problem,
min f (x), x ∈ Rn , (1)
where f : Rn → R is continuously differentiable, and Rn denotes the Euclidean
space with dimension n. Suppose that x0 is the starting point, xk is the current
point, and gk is the gradient of f at xk , i.e., gk = ∇f (xk ). Then gradient methods
generate the next iterative point by
xx+1 = xk − αk gk , k = 0, 1, . . . , (2)
where the stepsize αk is computed by some line search techniques. Two classical
ways of selecting initial stepsize in the line searches are given by the so called Steep-
est Descent (SD) and Minimal Gradient (MG) methods, which minimize f (xk −αgk )
and kg(xk − αgk )k along the search direction gk , respectively:
αSD
k = arg min f (xk − αgk ), (3)
α∈R

and
αMG
k = arg min kg(xk − αgk )k. (4)
α∈R
where k · k denotes the Euclidean norm of a vector. However, it is well-known
that SD and MG methods can be very slow when the Hessian of f is singular or
nearly singular at the local minimum. In this case the iterates could approach the
minimum very slowly in a zigzag fashion [14].
The basic idea of Barzilai-Borwein (BB) [2] method is to use the matrix D(αk ) =
1 2
αk I, where I denotes the identity matrix, to approximate of the Hessian ∇ f (xk )
by imposing a quasi-Newton condition on D(αk ):
αBB
k = arg min kD(α) sk−1 − yk−1 k2 , (5)
α∈R

where sk−1 = xk − xk−1 , yk−1 = gk − gk−1 , and k > 2. By straightforward

calculation, BB stepsize obtained from (5) is
sTk−1 sk−1
αBB
k = . (6)
sTk−1 yk−1
By symmetry, another alternative BB stepsize could be computed by:
αBB2
k = arg min ksk−1 − D−1 (α) yk−1 k2 , (7)
α∈R

which gives
sTk−1 yk−1
αBB2
k = T y
. (8)
yk−1 k−1
398 ROUHOLLAH TAVAKOLI AND HONGCHAO ZHANG

In contrast to the SD or MG methods in which the non-trivial and expensive op-

timization problem (3) or (4) need to be solved to obtain the initial stepsize, the
BB stepsize is readily available during the iterations by formula (6) or (8). In prac-
tice, to keep the stability of the numerical procedure, it is often to project the BB
stepsize onto a safeguard interval, that is to set
ᾱBB
k = min{αmax , max{αmin , αBB
k }} (9)
where αmin , αmax ∈ R and 0 < αmin << 1 << αmax < ∞.
The following Lemma shows the spectral property of Barzilai-Borwein methods
and it is due to this property that these methods are usually called spectral gradient
methods.
Lemma 2.1. The Barzilai-Borwein stepsize, αBB k , is the inverse of the Rayleigh
quotient, related to vector sk−1 , of the averaged Hessian of the objective function
between two consecutive iterations k − 1 and k.
Proof. By the Mean-Value Theorem and straightforward computations, one has
Z 1
gk − gk−1
= ∇2 f (txk + [1 − t]xk−1 )dt.
xk − xk−1 0
Therefore,  
T
R1 2
sk−1 0 ∇ f (xk−1 + tsk−1 )dt sk−1
sTk−1 yk−1
= ,
sTk−1 sk−1 sTk−1 sk−1
which complete the proof.
By Lemma 2.1, one has Λmin 6 (αBB k )
−1
6 Λmax , where Λmax and Λmin are the
R1
maximum and minimum eigenvalues of the averaged Hessian matrix 0 ∇2 f (xk +
tsk ) dt respectively. Therefore, αBBk I can be considered as an approximation of
the inverse of the averaged Hessian of the objective function. This shows that the
Barzilai-Borwein methods could incorporate certain useful second order information
with extremely little additional expense of computational and memory cost com-
pared with SD or MG methods. Due to its easy implementation, efficiency and low
storage requirement, BB-type methods have been widely used in many applications.
Exceptional good performances of BB-type methods have been observed for solv-
ing large-scale problems in particular when only approximate (not very accurate)
solutions are desired, cf. [12].
It has been shown that if the exact steepest descent step (3) is reused in a cyclic
fashion, the convergence speed of gradient methods can be greatly accelerated.
However, it is often very expensive or impractical to find the exact stepsize along
the steepest descent searching direction for large dimensional problems, unless the
objective function is a quadratic function. Hence, for non-quadratic objective func-
tions, it is unrealistic to apply some cyclic fashion of the steepest descent method.
On the contrary, the BB stepsize (6) or (8) can still be easily calculated. Hence,
analogous to the cyclic steepest descent method, the cyclic BB (CBB) has been
introduced in [10] for general nonlinear optimization. The numerical results given
in [10] show the CBB methods have excellent numerical performances compared
with SD methods and even be competitive to some well-known nonlinear conjugate
gradient methods. Given an integer m > 1, which is the cycle length, CBB stepsize
can be expressed as
αCBB BB
ml+i = αml+1 for i = 1, . . . , m, l = 0, 1, . . . , (10)
 SPECTRAL PROJECTED GRADIENT METHOD 399

The R-linear convergence of CBB method for a strongly convex quadratic objec-
tive function has been proved in [9], while the local R-linear convergence for the
CBB method at a local minimizer for general nonlinear objective function has been
established in [10].

3. Projected Barzilai-Borwein methods. Consider the following constrained

counterpart of problem (1)
min f (x), x ∈ D, (11)
where D ⊆ Rn is a closed non-empty convex set. Because of the constraints in (11),
the iterations generated by (2) may lie outside of the feasible set D. Therefore,
(2) need to be modified in order to maintain the feasibility of iterations. Gradient
projection methods (see: [21]) keep the feasibility of iterates by frequently projecting
trial steps generated from (2) onto the feasible set. Considering x as the trial step,
the projection point y of x onto D, denoted by PD [x], can be computed by solving
the following minimization problem
1 2
y := PD [x] = arg min kx − zk2 . (12)
z∈D 2

Since the feasible set is convex, problem (12) always has an unique solution. How-
ever, in general (12) is still a convex constrained quadratic programming problem,
which could be as difficult as the original problem. And for general convex con-
strained large-scale problems, this projection step at each iteration could be very
time consuming and is normally the most expensive part of gradient projection
methods. Hence, there is little interests on applying the gradient projection meth-
ods for large-scale problem unless the gradient projection step can be performed
very efficiently. However, in some special cases when efficient algorithms for cal-
culating the projection (12) exist, for example there are only box or single ball
constraints, these gradient projection methods will be attractive. Fortunately, as
we will see in the next section, the projection step can be performed very efficiently
for the volume constrained topology optimization problems. This makes it possible
for us to design gradient projection type algorithms for volume constrained topology
optimization.
Combining the Barzilai-Borwein stepsize rule and the gradient projection method,
the projected Barzilai-Borwein (PBB) method was first introduced in [4]. In this
method the iteration updating formula (2) is modified to
xx+1 = xk + βk dα
k, k = 0, 1, . . . , (13)
where β ∈ R+ and the search direction dα k (descent direction) is computed by
connecting the current point to the projection point of the trial iterate (2) based
on the BB stepsize, that is
dα BB
k = PD [xk − αk gk ] − xk . (14)
In [4],dα
k was called spectral projected gradient. It is not difficult to show that
dαk is a descent direction (see Lemma 3.1). This together with the convexity of the
feasible set D would imply that for a sufficiently small βk , an iterate of (13) will
reduce the objective function value while simultaneously preserve the feasibility of
the iterates.
Lemma 3.1. for all xk ∈ D and αBB
k > 0,
2
(i) hgk (x), dα
k (x)i 6
1
αBB
kdα
k (x)k .
k
400 ROUHOLLAH TAVAKOLI AND HONGCHAO ZHANG

(ii) dα
k (x) = 0 if and only if x is a stationary point for (11).

Proof. see the Proposition 2.1 in [17].

4. Globalization by nonmonotone line search. Using descent directions and

simply applying the SD, MG or BB stepsizes in gradient projection methods is
generally not sufficient to ensure global convergence of the iterates starting from
an arbitrary initial point. Hence, to deal with general nonlinear objective func-
tion, a globalization strategy is required to guarantee the global convergence of the
algorithm.
Monotonic gradient-based methods usually generate a sequence of iterates for
which a sufficient decrease in the objective function (or the related merit function)
is enforced at every iteration. In many cases, the globalization strategy accepts the
stepsize in the search direction, if it satisfy the well-known Wolfe or Armijo type
conditions (cf. chapter 3 [18]). This can be accomplished using either of monotonic
line searches or trust region methods.
Since the search direction is parallel to the negative direction of the gradient
(projected gradient) in BB (PBB) methods, the globalization of BB (PBB) meth-
ods by monotonic function value reduction often reduces them to the classic SD
(projected SD) method. Hence, these monotonic methods will often distroy all of
the advantages of BB (PBB) methods in contrast to SD (projected SD) method. To
maintain the inherit spirits of BB-type methods, it is essential to accept the initial
BB-type stepsize as frequently as possible while simultaneously ensure the global
convergence. Hence, some nonmonotone line search techniques need to be develope-
d to globalize the BB-type methods. The first nonmonotone line search technique
was developed in [16] in which the main goal was to accept the full Newton step
as much as possible. Combing the type of nonmonotone line search in [16], the
first globalized version of BB nethod (GBB) was introduced in [20]. Following [20],
the globalized PBB method (GPBB) was suggested in [4]. In these methods the
following (weaker) objective function value decrease condition is enforced during
each iteration
f (xk+1 ) 6 max f (xk−j ) + δ gTk dk , (15)
06j6mk
where δ ∈ (0, 1), dk is the search direction and mk is a nonnegative nondecreasing
integer, bounded by some fixed integer M . More precisely
m0 = 0 and 0 6 mk 6 min{mk−1 + 1, M } for k > 0.
Based on the same motivations, the more efficient and adaptive nonmonotone line
searches were particularly designed for BB-type methods in [17]. The globalized
projected cyclic Barzilai-Borwein (PCBB) algorithm based on these new nonmono-
tone line search techniques can be described as follows:
Algorithm 4.1.

Parameters:
• ǫ ∈ [0, ∞), error tolerance.
• δ ∈ (0, 1), descent parameter used in Armijo line search.
• η ∈ (0, 1), decay factor for stepsize in Armijo line search.
• αmin , αmax ∈ (0, ∞), safeguarding interval for BB stepsize.

Initialization:
 SPECTRAL PROJECTED GRADIENT METHOD 401

r
• k = 0, x0 = starting guess, and f−1 = f (x0 ).

Main Loop: While kPD [xk − gk ] − xk k∞ > ǫ

1. Choose ᾱk ∈ [αmin , αmax ].
2. Compute dk = PD [xk − ᾱk gk ] − xk .
3. Choose fkr such that f (xk ) 6 fkr 6 max{fk−1 r
, fkmax } and fkr 6 fkmax
max
infinitely often, where fk = max{f (xk−i ) : 0 6 i 6 min(k, M − 1)}.
4. Let f R be either fkr or min{fkr , fkmax }.
5. Nonmonotone line search:
5.1. If f (xk + dk ) 6 f R + δ gTk dk then βk = 1.
5.2. Else βk = η j , where j > 0 is the smallest integer such that f (xk +
η j dk ) 6 f R + η j δ gTk dk .
6. Set xk+1 = xk + βk dk and k = k + 1.
End Main Loop.

The variable fkr in Algorithm 4.1 denotes the so called “reference” function value
in nonmonotone line search. It can be seen that the traditional monotone line
search simply corresponds to the choice of setting fkr = f (xk ) at each iteration.
And the nonmonotone line search developed in [16] corresponds to the choice of
setting fkr = fkmax . In our present study, fkr is chosen based on Algorithm 4.2
adapted from [17]. Let fk denote f (xk ). In the algorithm 4.2, the integer a counts
the number of consecutive iterations for which βk = 1 in Algorithm 4.1 is accepted
and the Armijo line search in step 5 is skipped. The integer l counts the number of
iterations since the function value is strictly decreased by an amount ∆ > 0.
Algorithm 4.2.

R0. If k = 0, choose parameters: A > L > 0, γ1 , γ2 > 1, and ∆ > 0; initialize

a = l = 0 and f0min = f0maxmin = f0r = f−1r
= f0 .
r
R1. Update fk as follows:
R1.1. If l = L, then set l = 0, and
fkmax −fkmin
(
r fkmaxmin if f maxmin −fkmin
> γ1 ,
fk = k
max
fk otherwise.
R1.2. Else If a > A, then set
( r
fk−1 −fk
r fkmax if fkmax > fk and fkmax −fk > γ2 ,
fk = r
fk−1 otherwise.
R1.3. Else set fkr = fk−1
r
.
R
R2. Set f as follows in step 4 of Algorithm 4.1:
R2.1. If j = 0 (the first iterate in a CBB cycle) then f R = fkr .
R2.2. Else (j > 0) f R = min{fkr , fkmax }
R3. If βk = 1 in Algorithm 4.1 then a = a + 1, Else (βk < 1) a = 0.
R4. If fk+1 6 fkmin − ∆ then set fk+1 maxmin min
= fk+1 = fk+1 and l = 0; Else set
min min maxmin
l = l + 1, fk+1 = fk and fk+1 = max{fkmaxmin , fk+1 }.
The variable fkmax in Algorithm 4.2 stores the maximum of recent function val-
ues and fkmin stores the minimum function value within the tolerance ∆. The
402 ROUHOLLAH TAVAKOLI AND HONGCHAO ZHANG

variable fkmaxmin stores the maximum function value since the last new minimum
was recorded in fkmin .
The condition f (xk ) < fkr in step 3 of Algorithm 4.1 guarantees that the Armijo
line search in step 5 can be satisfied. Notice that the requirement “fkr < fkmax infin-
itely often” in step 3 which is required to ensure the global convergence is a weaker
condition. Besides Algorithm 4.2 which satisfies this condition, this condition can
be satisfied by many other strategies. For example, it is possible to set fkr = fkmax
at every L iterations. In Algorithm 4.2, fkr = fkmax if f (xk−L ) − f (xk ) 6 ∆ for
given decrease parameter ∆ > 0 and integer L > 0.
Now lets give more details about computation of ᾱk in step 1 of Algorithm 4.1.
This parameter is computed based on the safeguarded CBB scheme as follows. Let
j as an integer that counts the number of times in which the current BB step has
been reused and let m as the CBB memory in (10), i.e., the maximum number of
times the BB step will be reused.

Algorithm 4.3.

S0. If k = 0 choose ᾱ0 ∈ [αmin , αmax ] and a parameter θ < 1 near 1; set j = 0
and flag=1. If k > 0 set flag = 0.
S1. 0 < |dki | < ᾱk |gki | for some i (component of vector) then set flag = 1.
S2. If βk = 1 in Algorithm 4.1 then set j=j+1; Else (βk < 1) set flag =1.
S3. If j > m or flag=1 or sTk yk /ksk kkyk k > θ then:
S3.1. If sTk yk 6 0 then
S3.1.1. If j > 1.5m then set t = min{kxk k∞ , 1}/kd1 (xk )k∞ ,
ᾱk+1 = min{αmax , max{αmin , βk }} and j = 0;
where d1 (xk ) = PD [xk − gk ] − xk
S3.1.2. Else ᾱk+1 = ᾱk
S3.2 Else set ᾱk+1 = min{αmax , max{αmin , αBB k }} and j = 0.

In Algorithm 4.3, the former BB stepsize is reused for the current iterate unless
one of the following conditions happens in which the new BB stepsize is computed
(see S3.2) : (I) the previous BB stepsize is truncated by the projection step, i.e.,
when the trial point using BB stepsize lies outside of the feasible domain and the
gradient projection was performed (see S1); (II) the previous BB stepsize is truncat-
ed by the line search step (see S2 where βk < 1); (III) the number of times the BB
stepsize was reused reaches to its bound, i.e., j > m (see S3); (IV) sTk yk /ksk kkyk k
is close to 1 (see section 4 [10] for details about the justification for this decision).
The condition sTk yk < 0 (see S3.1) is equivalent to detection of the negative curva-
ture in the searching direction. Assuming that the objective function can be well
approximated by a quadratic function in the vicinity of the current iterate, a rela-
tively large stepsize should be used in the next iteration (see S3.1.1) to reduce the
function as much as possible once a negative curvature is detected. This strategy
is similar to the original SPG algorithm (see section 2 [5]).
Now lets briefly review the convergence theory of Algorithm 4.1.

Theorem 4.4. Let L be the level set defined by

L = {x ∈ D : f (x) ≤ f (x0 ) }.
We assume the following conditions hold:
G1. f is bounded from below in L and dmax = supk kdk k < ∞.
 SPECTRAL PROJECTED GRADIENT METHOD 403

G2. If L̄ is the collection of x ∈ D whose distance to L is at most dmax , then ∇f

is Lipschitz continuous on L̄.
Then either algorithm 4.1 with ǫ = 0 terminates in a finite number of iterations at
a stationary point, or we have lim inf kd1 (xk )k∞ = 0.
k→∞
Proof. see the Theorem 2.2 in [17].
When f is a strongly convex function, (11) has a unique minimizer x∗ and the
conclusion of the global convergence Theorem 4.4 can be strengthened as follows.
Theorem 4.5. Suppose f is strongly convex and twice continuously differentiable
on D, and there is a positive integer L with the property that for each k, there exists
j ∈ [k, k + L) such that fjr ≤ fjmax . Then the iterates xk of Algorithm 4.1 with ǫ = 0
converge to the global minimizer x∗ .
Proof. see the Corollary 2.3 in [17].

Figure 1. Projection onto the feasible set for n = 2 related to the

volume constrained topology optimization problem; the feasible set
is a potion of line aT x = b which is located inside the box l 6 x 6 u.

5. Projection onto the feasible set. The introduced algorithm in the previous
section is general and can be applied to any type of convex constrained optimization
problems which satisfy the requirements of Theorem 4.4. However, the performance
of the method is significantly affected by the efficiency of the projection step, see
[7]. In this section, we explore the special structure of the feasible set in the volume
constrained topology optimization and introduce a very efficient algorithm to do
the projection step.
After discretization, the feasible set in the volume constrained topology opti-
mization problems has the following form (see Figure 1)
D = {x ∈ Rn : aT x = b, l 6 x 6 u}, (16)
n n
where a ∈ R , ai ∈ R+ , b ∈ R+ , l, u ∈ R , 0 < li 6 ui < ∞. Henceforth, in this
section we denote by D the feasible domain defined in (16).
Considering (12) together with (16), for a given trial vector x, PD [x] is the unique
minimizer of the following box constrained Lagrangian:
1 1
L(z; λ) = kzk22 − xT z + kxk22 + λ(aT z − b), z ∈ B, (17)
2 2
404 ROUHOLLAH TAVAKOLI AND HONGCHAO ZHANG

where λ ∈ R is a proper Lagrange multiplier related to the volume constraint and

the box constrained set B is defined as B = {z ∈ Rn : l 6 z 6 u}. For any fixed
value of λ, (17) is a convex separable minimization problem with respect to z, which
has the following explicit solution:
z(λ) = max{l, min{x − λa, u}}, (18)
where the max and min operators are understood as componentwise. Since solutions
resulted from (18) satisfy the bound constraints, to solve the projection problem
(12) the remaining task then turns out to find the Lagrange multiplier λ∗ such
that (18) satisfies the volume constraint. Hence, an efficient algorithm is needed
to find the (unique) root λ∗ of the following nonlinear non-smooth one-dimensional
equation:
g(λ) = aT z(λ) − b = 0. (19)
Considering (18) together with (19), it can be seen that the graph of g(λ) has 2n
breakpoints at:
λli = (xi − li )/ai , λui = (xi − ui )/ai , i = 1, . . . , n.
Since li 6 ui and ai > 0 for all i, we have λui 6 λli . So, each zi (λ) in (18) can be
expressed in the following form:

 ui , if λ 6 λui ,
zi (λ) = xi − λai , if λui 6 λ 6 λli , (20)
li , if λ > λli .


From (20), it is obvious that for each i, zi (λ) is a continuous piecewise linear and
non-increasing function of λ (see Figure 2). Therefore, by ai > 0 for all i, g(λ) is
a continuous piecewise linear and non-increasing function of λ. Then, with some
straitforward calculations, we can see that the root λ∗ of g(λ) is unique and λ∗ ∈
[λmin , λmax ] with λmin 6 0 6 λmax , where λmin = min{λui } and λmax = max{λli }.
So, starting from interval [λmin , λmax ] and using some classical one dimensional root
finding method, it is possible to find λ∗ up to the machine precision with a priori
known computational complexity. However, it is possible to use more advanced root
founding methods to improve the performance of this step.

Figure 2. Plot of zi (λ) as a function of λ, cf. equation 20.

In our approach, the Brent’s root finding method [6] is employed to solve (19).
The Brent’s method does not assume the function differentiability and is enable to
manage the limited precision of the computed arithmetic very well. In the worst
 SPECTRAL PROJECTED GRADIENT METHOD 405

condition, the convergence of this method is never slower that of the bisection
method. The Brent’s method has been proved to be very efficient and robust in
practice and it is currently accepted as a standard method for one dimensional
root finding problem (cf. chapter 9 of [19]). The specific implementation details of
Brent’s method are available in the Numerical Recipe (see chapter 9 of [19])..

6. Numerical solution of topology optimization problems. To show efficien-

cy of the proposed method for the volume constrained topology optimization [3], in
this section we solve the standard model problems presented in [11]. The descrip-
tion of the problem is as follows: considering two isotropic conducting materials,
with thermal conductivities kα and kβ , 0 < kα < kβ , in a simply connected design
domain Ω ⊂ Rd (d = 2, 3), the goal is to mix these materials with a fixed ratio in
Ω, such that the total temperature gradient in Ω is minimized under the thermal
load f ∈ L2 (Ω). More specifically, the problem can be formulated as
arg minw J(w) = 21 Ω ∇θ · ∇θ dx,
 R






subject to:





(P )

 −∇ · (k(w)∇θ) = f (x) in Ω,
θ(x) = θ (x) on ∂Ω,

0


 p p
k(w) = w k + (1 − w )k in Ω,

β α


 R
w dx = R|Ω|, 0 < R < 1, 0 6 w 6 1,

Ω

where θ ∈ H 1 (Ω) is the state variable, p > 1 is a penalization factor and w ∈ L2 (Ω)
is the control parameter (topology indicator field). By some standard derivations,
the first order optimality condition of problem P can be written as the following:


 −∇ · (k(w)∇θ) = f (x) in Ω,
θ(x) = θ0 (x) on ∂Ω,




−∇ · (k(w)∇η) = −∇ · (∇θ) in Ω,



(OC) η(x) = 0 on ∂Ω,
k(w)
= wp kβ + (1 − wp )kα in Ω,




P G(x) = 0 in Ω,

D



G(x) = −pwp−1 (kβ − kα )∇θ · ∇η in Ω,


where η ∈ H01 (Ω) is the adjoint state, G is the L2 gradient of the objective functional
with respect to w and PD (u) denotes the L2 projection of function u onto the
admissible set D,
Z
2
D = {w ∈ L (Ω) | w(x)dx = R|Ω|, 0 < R < 1, 0 6 w 6 1}.
Ω
By discretization of Ω into an n control volumes, we have the finite dimensional
counterpart of problem (P). We also assume that the state variable and the design
parameter are defined at the center of each control volume. Under these assump-
tions, the admissible design domain D forms a simplex in Rn which is identical to the
continuous knapsack constraints in (16). At each iterate of the control parameter
w, we solve the associated state PDE. Then the discretized optimization problem
would have the general format of problem (11), which has a nonlinear objective
function with convex continuous knapsack constraints.
406 ROUHOLLAH TAVAKOLI AND HONGCHAO ZHANG

In our numerical experiments, the problem (P) was solved in two and three
dimensions for Ω = [0, 1]2 and Ω = [0, 1]3 . The spatial domain is divided into
1272 and 313 control volumes in two and three dimensions respectively. In all of
experiments, we set kα = 1, f (x) = 1 and R = 0.4. And in these experiments two
conductivity ratios 2 and 100 were tested, which is equivalent to setting kβ = 2, 100
respectively. The penalization factor p is taken to be 1 and 10 for conductivity
ratio 2 and 100 respectively. The governing PDE are solved by cell centered finite
volume method using central difference scheme and the related system of linear
equations are solved by a preconditioned conjugate gradient method with relative
convergence threshold 10−20 . The optimization is performed for 15 iterations in
these numerical experiments. Notice that using finite volume method we do not
observe any topological instability phenomena (the checkerboard pattern), which
often occurs by using finite element method for topology optimization problems.
The input parameters related to optimization algorithms used in this study are
as follows: δ = 10−4 , η = 0.5, αmin = 10−30 , αmax = 1030 , A = 40, L = 10,
M = 20, m = 4, γ1 = γ2 = 2, θ = 0.975. Moreover, the initial value α0 for spectral
step-size was taken to be 1/kPD [xk − gk ] − xk k∞ . Another alternative choice for
this parameter could be α0 = 1/kPD [xk − gk ] − xk k2 . However, for the testing
problems in our numerical experiments the former choice was considerably more
efficient.
To evaluate the efficiency of the presented method, we compare our results with
the results obtained by the method of moving asymptotic (MMA) [22]. The imple-
mentation of MMA available in SCPIP code [26]2 is used in our experiments. All
default parameters are used in SCPIP code, except the parameter for the constrain-
t violation, for which the threshold 10−7 is used in this study. That SCPIP code
used in MMA algorithm has two globalization strategies. The first one is identical
to that of the original MMA by Svanberg [23], while the second one is globalization
by monotone line search method (see: [25]). The first strategy is employed in our
experiments, since our results with second strategy was significantly worse in terms
of the computational cost. Note that in our procedure the PDE constraint is solved
upto the accuracy of the finite volume method we have applied for solving this
PDE, and the constraints (16) in PCBB algorithm are satisfied upto the machine
precision.
The results of our numerical experiments including the variation of the objective
functional during the optimization process and the final resulted topology (w-field)
are shown in figures 3, 4, 5 and 6. The plots in figures 3, 4 show the success of the
presented method to solve these topology optimization problems. Roughly speaking,
both methods behave very competitively in terms of computational cost and final
results. The main differences in the results are related to the conductivity ratio
100, in which the PCBB performs superior and its final objective function values
are considerably lower than that of MMA. The sign of the differences is clear in the
figures 5 and 6 related to the final topologies. But for the conductivity ratio equal
to 2, it seems MMA behaves slightly better than PCBB. However, the differences
in terms of the objective function values as well as the final topologies are almost
negligible .
In all our numerical experiments, the final total number of function and gradient
evaluations was 15 which is equal to the number of optimization cycles. This result
2 The SCPIP code (in Fortran) is freely available through personal request from its original

author (Christian Zillober: [Link](at)[Link]).

 SPECTRAL PROJECTED GRADIENT METHOD 407

1 1

0.95 0.95
scaled objective function

scaled objective function

0.9 0.9

0.85 0.85

0.8 0.8

0.75 0.75
0 2 4 6 8 10 12 14 16 0 2 4 6 8 10 12 14 16
iterations iterations

2D result: PCBB with conductivity ratio = 2 2D result: MMA with conductivity ratio = 2

1 1

0.9 0.9

0.8
0.8

0.7
scaled objective function

scaled objective function

0.7
0.6
0.6
0.5
0.5
0.4
0.4
0.3

0.3
0.2

0.1 0.2

0 0.1
0 2 4 6 8 10 12 14 16 0 2 4 6 8 10 12 14 16
iterations iterations
2D result: PCBB with conductivity ratio = 100 2D result: MMA with conductivity ratio = 100

Figure 3. Variations of scaled objective function values during

optimization cycles for 2D examples.

shows that both methods used only one function and gradient evaluations per it-
eration in practice. We believe that this is a key property for the success of MMA
and makes the method well accepted in the engineering design community. In fact,
MMA behaves very conservatively and uses small steps to proceed toward a local
minimum. More clearly, it does not use (expensive) line search globalization strate-
gy (unlike alternative methods), but uses reasonably small steps such that hopefully
the merit function decreases sufficiently during each iteration. Of course, whenever
the merit function increases (or sufficient decrease in merit function value violates),
which rarely happens in practice, it performs sub-cycles to ensure the desired mono-
tonic behavior. On the other hand, the nonmonotone PCBB methods enjoys such
property using an alternative strategy. In practice, by applying the nonmonotonic
line search, PCBB often uses only one function evaluation per optimization cycle.
As our results clearly show, this property makes the nonmonotone PCBB method
as a very competitive alternative to MMA for these class of problems.
It is important to note that in all of our experiments presented here, the pro-
jection step is used actively in PCBB method. Therefore, the cyclic reuse of step
size never employed in our testing problems (cf. algorithm 4.3). Moreover, for
the conductivity ratio 100, PCBB method explored directions of negative curvature
after a few iterations, and so used large step-sizes which considerably accelerate
408 ROUHOLLAH TAVAKOLI AND HONGCHAO ZHANG

the convergence. This property plays an important role for the superior results of
PCBB compared with MMA in these cases (cf. table 1).

1 1

0.98 0.98

0.96 0.96
scaled objective function

scaled objective function

0.94 0.94

0.92 0.92

0.9 0.9

0.88 0.88

0.86 0.86

0.84 0.84

0.82 0.82
0 2 4 6 8 10 12 14 16 0 2 4 6 8 10 12 14 16
iterations iterations
3D result: PCBB with conductivity ratio = 2 3D result: MMA with conductivity ratio = 2

1 1

0.9 0.9

0.8
0.8

0.7
scaled objective function

scaled objective function

0.7
0.6
0.6
0.5
0.5
0.4
0.4
0.3

0.3
0.2

0.1 0.2

0 0.1
0 2 4 6 8 10 12 14 16 0 2 4 6 8 10 12 14 16
iterations iterations
3D result: PCBB with conductivity ratio = 100 3D result: MMA with conductivity ratio = 100

Figure 4. Variations of scaled objective function values during

optimization cycles for 3D examples.

Besides the presented numerical results here, we have also applied the presented
method successfully to many families of topology optimization problems with very
satisfied results. But we omit the details of these experiments here due to the
limited space for this paper.

Table 1. Variations of step size αk during the fist 10 optimiza-

tion cycles in PCBB algorithm for 2D examples (the decimals are
dropped in the table). α(2) and α(100) denote values of αk for the
conductivity ratio 2 and 100 respectively.

iter 1 2 3 4 5 6 7 8 9 10
α(2) 37 48 124 133 52 44 54 147 305 286
α(100) 53 1030 1030 243 164 100 104 1030 1030 1093
 SPECTRAL PROJECTED GRADIENT METHOD 409

Figure 5. Final distribution of material field, w, for 2D examples

related to PCBB (top) and MMA (bottom) for conductivity ratio
2 (left) and 100 (right).
410 ROUHOLLAH TAVAKOLI AND HONGCHAO ZHANG

Figure 6. Final distribution of material field, w, for 3D examples,

related to PCBB (top) and MMA (bottom) conductivity ratio 2
(left) and 100 (right).
 SPECTRAL PROJECTED GRADIENT METHOD 411

7. Closing remarks. Recent studies on the spectral projected gradient methods

show that these class of methods are very promising for solving the large-scale con-
vex constrained optimization problems when the projection on the feasible set can
be performed efficiently. In this paper, we presented a particular spectral projected
gradient method called PCBB (Projected Cyclic Barzilai-Borwein) for solving vol-
ume constrained topology optimization problems. This method applies the cyclic
Barzilai-Borwein stepsizes and uses the most recent adaptive nonmonotone line
search techniques, which greatly improves the efficiency of the method as well as
ensures its global convergence. By exploring the structure of the admissible set of
the volume constrained topology optimization problem, the projection step can be
performed very efficiently. In addition to high efficiency, our presented method also
enjoys the following features: easy implementation, minimum memory requirement,
no need for second order (Hessian) information, not sensitive to data noise, feasi-
bility being strictly maintained during optimization process. All these features are
essential for a successful numerical method to solve large-scale topology optimiza-
tion problems.
Our numerical results indicate our presented methods are very promising and
well-suited for the class of topology optimization problems considered in this paper.
Comparing our results with those of MMA, a well accepted method in shape and
topology optimization community, it seems the presented methods could be a very
competitive alternative choice for solving the class of topology design problems.
Finally, our results suggest that including spectral step size and a nonmonotone
globalization strategy in MMA, the MMA algorithm could be further significantly
improved. This would be a topic for our continuing research.

Acknowledgments. The authors would like to thank Prof. Christian Zillober for
sharing SCPIP code with them. The work of second author is supported by the
National Science Foundation under grants 1016204.

REFERENCES

[1] G. Allaire, “Shape Optimization by the Homogenization Method,” Springer, 2002.

[2] J. Barzilai and J. M. Borwein, Two-point step size gradient methods, IMA Journal of Numer-
ical Analysis, 8 (1988), 141–148.
[3] M. P. Bendsøe and O. Sigmund, “Topology Optimization: Theory, Methods, and Applica-
tions,” Springer Verlag, 2003.
[4] E. G. Birgin, J. M. Martı́nez and M. Raydan, Nonmonotone spectral projected gradient meth-
ods on convex sets, SIAM Journal on Optimization, 10 (2000), 1196–1211.
[5] E. G. Birgin, J. M. Martı́nez, and M. Raydan, Algorithm 813: SPGsoftware for convex-con-
strained optimization, ACM Transactions on Mathematical Software, 27 (2001), 340–349.
[6] R. P. Brent, An algorithm with guaranteed convergence for finding a zero of a function, The
Computer Journal, 14 (1971), 422–425.
[7] L. Ceng, Q. H. Ansari and J. Yao, Extragradient-projection method for solving constrained
convex minimization problems, Numerical Algebra, Control and Optimization, 1 (2011), 341–
359.
[8] A. R. Conn, N. I. M. Gould and Ph. L. Toint, “Trust-Region Methods,” SIAM, Philadelphia,
PA, USA, 2000.
[9] Y. H. Dai, Alternate step gradient method, Optimization, 52 (2003), 395–415.
[10] Y. H. Dai, W. W. Hager, K. Schittkowski and H. Zhang, The cyclic Barzilai–Borwein method
for unconstrained optimization, IMA Journal of Numerical Analysis, 26 (2006), 604–627.
[11] A. Donoso, Numerical simulations in 3D heat conduction: Minimizing the quadratic mean
temperature gradient by an optimality criteria method, SIAM Journal on Scientific Comput-
ing, 28 (2006), 929–941.
412 ROUHOLLAH TAVAKOLI AND HONGCHAO ZHANG

[12] R. Fletcher, On the Barzilai-Borwein method, Optimization and Control with Applications,
96 (2005), 235–256.
[13] C. Fleury, CONLIN: An efficient dual optimizer based on convex approximation concepts,
Structural and Multidisciplinary Optimization, 1 (1989), 81–89.
[14] G. E. Forsythe, On the asymptotic directions of the s-dimensional optimum gradient method,
Numerische Mathematik, 11 (1968), 57–76.
[15] P. E. Gill, W. Murray, and M. A. Saunders, SNOPT: An SQP algorithm for large-scale
constrained optimization, SIAM J. Optim., 12 (2002), 979–1006.
[16] L. Grippo, F. Lampariello and S. Lucidi, A nonmonotone line search technique for Newton’s
method, SIAM Journal on Numerical Analysis, (1986), 707–716.
[17] W. W. Hager and H. Zhang, A new active set algorithm for box constrained optimization,
SIAM Journal on Optimization, 17 (2006), 526–557.
[18] J. Nocedal and S. J. Wright, “Numerical Optimization,” Springer, 2006.
[19] W. H. Press, S. A. Teukolsky, W. T. Vetterling and B. P. Flannery, “Numerical Recipes 3rd
edition: The Art of Scientific Computing,” 2007.
[20] M. Raydan, The Barzilai and Borwein gradient method for the large scale unconstrained
minimization problem, SIAM Journal on Optimization, 7 (1997), 26–33.
[21] J. B. Rosen, The gradient projection method for nonlinear programming. Part I. Linear
constraints, Journal of the Society for Industrial and Applied Mathematics, (1960), 181–217.
[22] K. Svanberg, The method of moving asymptotes- A new method for structural optimization,
International Journal for Numerical Methods in Engineering, 24 (1987), 359–373.
[23] K. Svanberg, A class of globally convergent optimization methods based on conservative con-
vex separable approximations, SIAM J. Optim., 12 (2002), 555–573.
[24] Y. X. Yuan, Recent advances in numerical methods for nonlinear equations and nonlinear
least squares, Numerical Algebra Control and Optimization, 1 (2011), 15–34.
[25] C. Zillober, A globally convergent version of the method of moving asymptotes, Structural
and Multidisciplinary Optimization, 6 (1993), 166–174.
[26] C. Zillober, SCPIP: an efficient software tool for the solution of structural optimization
problems, Struct. Multidisc. Optim., 24 (2002), 362–371.

Received October 2011; revised March 2012.

E-mail address: rtavakoli@[Link]
E-mail address: hozhang@[Link]