1.

a. F(x)=x²-3x+5

Parabolic :Found minimum (x=1.5) in one iteration. This is not suprising given the function is parabolic.

Newton’s: ditto

b. F(x)=x⁴-3x³+x-3.14

Parabolic:

Using x0=-.5,x1=0 and x2=0.5 gave x_min=-0.5 an incorrect minimum.

I need to choose the x-values such that f(x0)>f(x1) and f(x2)>f(x1) for x0<x1<x2

I next tried x0=-.75,x1=0 and x2=.25. The program found the maximum at x=.364 after 6 iterations.

I next tried x0=-0.50,x1=-.3 and x2=-.15. The program found a minimum at x=-.312356 after 6 iterations (It was already at -.312 after 3 iterations)

Next tried x0=1.25,x1=2.0 and x2=2.75 as they bordered a minimum by instpection of the graph. Found x_min=2.198266 after 6 iterations

This demonstrates the importance of knowing the shape of the function as the parabolic method success is sensitive to the input values

Newton’s: Initial guess, x_min=0. Stuck at our guess value. We are at an inflection point (second derivative is zero) and hence the method does not go up or down

Initial guess, x_min=-.25 gave a minimum of -.312356 after 3 iterations.

Initial guess, x_max=.1 gave a maximum of at x_max=.364091 after 4 iterations

Initial guess,x_min=-2 gave a minimum at x_min=-.312356 after 7 iterations

Initial guess, x_min=3 gave a minimum at x_min=2.198266 after 5 iterations

Newton’s method is better if you know little about the shape of the function as the criteria of the relative function values does not have to be met as in the parabolic method.

F(x,y) = -.5x 2+.25x 4+.25y 4

Using Steeepest Descent and start of (0.5,0.5) it had reached close to minimum (0.999670,0.081252) Notice as one approaches the minimum the steps get smaller (as they are proportional to the gradient) so in theory the minimum will only be reached after an infinite # of evaluations.

Using conjugate gradient method we reach the same value of the minimum seen in steepest descent but only after 11 iterations (1.0035,0.0899). After 27 iterations we have (1.000,0.0177).

Starting at (4.0,4.0):

Conjugate Gradient reaches (1.000,0.0870) after 17 iterations and (1.0000,0.0177) after 96 iterations. It reaches the x_min=1.000 after 18 iterations but takes considerable more iterations to reach y_min=0.0177.

Steepest Descent reaches (0.999649,0.0869) after 65 iterations and (1.000089,0.057463) after 150 iterations

F(x,y)=-.5x 2+.25x 4+.25xy 2+.1y 4

Starting at (0.5,0.5)

Steepest Descent:

Reached (1.000493,0.002490) after 10 iterations

Conjugate Gradient:

Reached (1.0009,0.0016) after 9 iterations and f(x,y) = -.25

Starting at (4.0,4.0)

Conjugate Gradient found the second minima (-1.1456,1.1967) after 5 iterations.

Steepest Descent found it as well (-1.1456,1.1967) after 5 iterations as well

Clearly, when multiple extrema are present the final extrema reached depends on the initial conditions.

Let’s try a starting point close to the other extrema

The other extrema is at (-1.1456,1.196681) after 17 iterations using steepest descent and f(x,y)=-0.4306

Hence, the global minimum is here and not the first minimum reached.