Description

Hanao Li hl3202
Instructions
Make sure that you upload the PDF or HTML output after you have knitted the Rmd file. The file you upload to the Canvas page should be updated with commands you provide to answer each of the questions below. You can edit this file directly to produce your final solutions.
Optimization
The goal of this lab is to write a simple optimization function in R which estimates the global minimum of a convex differentiable function f(x). Specifically, consider the function
− log(x)
f(x) = , x > 0, 1 + x
where log(x) is the natural logarithm of x. We seek to estimate the value of x > 0 such that f(x) achieves its global minimum. For example, the global minimum of the function g(x) = x2 − 4x + 3 is at x = 2. The minimum of g(x) can easily be computed using the vertex formula for quadratic functions, i.e.,
x = −b/(2a) = 4/(2 ∗ 1) = 2. In most cases, the minimum does not have a closed form solution and must be computed numerically. Hence we seek to estimate the global minimum of f(x) numerically via gradient descent.

Tasks
1. Using R, define the function
f(x) =
Test the points f(0) and f(2).
− log(x) , x > 0.
1 + x

2. Plot the function f(x) over the interval (0, 6].

3. By inspection, were do you think global minimum is located at?
By inspection, the global minimum is within the range of (0,6] and it should be around 3.59.
4. Define a R function which computes the difference quotient of f(x), i.e., for h > 0,
f(x + h) − f(x)
.
h
This function should have two inputs; h and x. Name the difference quotient function diff.quot. Note that for small h, this function is the approximate derivative of f(x).

5. Plot both the difference quotient function diff.quot and f(x) over the interval (0, 6]. Fix h = .0001 to construct this plot. Comment on any interesting features.

These two functions has a intersection point at around 1.5. They are inverse to each other.
6. Write a R function named basic.grad.descent that runs the basic gradient descent algorithm on the function f(x). The function should have inputs:
The function should have outputs:

7. Check the optimal value using the base R function nlm().