# quasinewton.m

This function calculates the quasi-Newton update giving an approximation to the Hessian matrix. The supported quasi-Newton methods are BFGS and SR1. This functions also provides two options; one for calculating an approximation to the Hessian matrix, and one for calculating the approximation to the inverse of the Hessian matrix.

## Syntax

    B_new=quasinewton(B_old,x_new,x_old,df_new,df_old,inv_method,algo)


## Description

### BFGS update formula

#### Approximating the Hessian

The BFGS formula to update the Hessian matrix is proper is

$\left[\nabla^{2}J\right]_{n+1}=\left[\nabla^{2}J\right]_{n}+ \left(\frac{\left| \Delta g\middle\rangle\middle\langle \Delta g\right|}{\left\langle \Delta g\middle| \Delta c\right\rangle} \right) - \left( \frac{\left[\nabla^{2}J\middle]_{n} \middle| \Delta c\middle\rangle\middle\langle \Delta c\middle|\middle[\nabla^{2}J\right]_{n}}{\left\langle \Delta c\middle| \left[\nabla^{2}J\right]_{n}\middle| \Delta c\right\rangle}\right)$

where $\left[\nabla^{2}J\right]_{n}$ is the approximation to the Hessian matrix, $\Delta c=c_{n+1}-c_{n}$ is the change in $c$ (the waveform in optimal control of magnetic resonance), and $\Delta g=\nabla J(c_{n+1})-\nabla J(c_{n})$ is the change in gradient

#### Approximating the inverse Hessian

The BFGS formula to update the inverse Hessian matrix is proper is

$\left[\nabla^{2}J\right]_{n+1}^{-1}=\left[\nabla^{2}J\right]_{n}^{-1}+ \left(1+\frac{\left\langle\Delta g\middle|\left[\nabla^{2}J\right]_{n}^{-1}\middle|\Delta g\right\rangle}{\left\langle \Delta c\middle|\Delta g\right\rangle} \right)\frac{\left| \Delta c\middle\rangle\middle\langle \Delta c\right|}{\left\langle \Delta c\middle|\Delta g\right\rangle} - \left(\frac{\left| \Delta c\middle\rangle\middle\langle \Delta g\middle|\middle[\nabla^{2}J\right]_{n}^{-1}+\left[\nabla^{2}J\middle]_{n}^{-1}\middle| \Delta g\middle\rangle\middle\langle \Delta c\right|}{\left\langle \Delta c\middle|\Delta g\right\rangle} \right)$

where $\left[\nabla^{2}J\right]_{n}^{-1}$ is the matrix approximating the inverse Hessian at an iterate $n$.

### Symmetric rank-1 update formula

#### Approximating the Hessian

The symmetric rank-1 (SR1) formula to update the Hessian matrix is proper is

$\left[\nabla^{2}J\right]_{n+1}=\left[\nabla^{2}J\right]_{n} - \left(\frac{\left(\left|\Delta g \right\rangle - \left[\nabla^{2}J\middle]_{n}\middle|\Delta c\right\rangle\right)\left(\left\langle\Delta c\middle|\middle[\nabla^{2}J\right]_{n} - \left\langle\Delta g \right|\right)}{\left\langle\Delta g \middle|\Delta c\right\rangle - \left\langle\Delta c\middle|\left[\nabla^{2}J\right]_{n}\middle|\Delta c\right\rangle }\right)$

#### Approximating the inverse Hessian

The SR1 formula to update the inverse Hessian matrix is proper is

$\left[\nabla^{2}J\right]_{n+1}^{-1}=\left[\nabla^{2}J\right]_{n}^{-1} - \left(\frac{\left(\left|\Delta c \right\rangle - \left[\nabla^{2}J\middle]_{n}^{-1}\middle|\Delta g\right\rangle\right)\left(\left\langle\Delta g\middle|\middle[\nabla^{2}J\right]_{n}^{-1} - \left\langle\Delta c \right|\right)}{\left\langle\Delta c \middle|\Delta g\right\rangle - \left\langle\Delta g\middle|\left[\nabla^{2}J\right]_{n}^{-1}\middle|\Delta g\right\rangle }\right)$

## Arguments

    B_old          - Hessian approximation at the previous iterate.

x_new          - new waveform at this iterate.

x_old          - old waveform from the previous iterate.

df_new         - new gradient at this iterate.

df_old         - old gradient from the previous iterate.

inv_method     - Boolean. Perform inverse Hessian update (true) or Hessian update (false).

algo           - update mathod, either 'bfgs' or 'sr1'.


## Returns

    B_new          - updated Hessian approximation


## Notes

It is normal to use an initial Hessian approximation equal the the identity matrix, this is default in fminnewton.m.