quasinewton.m

From Spinach Documentation Wiki
Jump to: navigation, search


This function calculates the quasi-Newton update giving an approximation to the Hessian matrix. The supported quasi-Newton methods are BFGS and SR1. This functions also provides two options; one for calculating an approximation to the Hessian matrix, and one for calculating the approximation to the inverse of the Hessian matrix.

Syntax

    B_new=quasinewton(B_old,x_new,x_old,df_new,df_old,inv_method,algo)

Description

BFGS update formula

Approximating the Hessian

The BFGS formula to update the Hessian matrix is proper is

[math] \left[\nabla^{2}J\right]_{n+1}=\left[\nabla^{2}J\right]_{n}+ \left(\frac{\left| \Delta g\middle\rangle\middle\langle \Delta g\right|}{\left\langle \Delta g\middle| \Delta c\right\rangle} \right) - \left( \frac{\left[\nabla^{2}J\middle]_{n} \middle| \Delta c\middle\rangle\middle\langle \Delta c\middle|\middle[\nabla^{2}J\right]_{n}}{\left\langle \Delta c\middle| \left[\nabla^{2}J\right]_{n}\middle| \Delta c\right\rangle}\right) [/math]

where [math]\left[\nabla^{2}J\right]_{n}[/math] is the approximation to the Hessian matrix, [math]\Delta c=c_{n+1}-c_{n}[/math] is the change in [math]c[/math] (the waveform in optimal control of magnetic resonance), and [math]\Delta g=\nabla J(c_{n+1})-\nabla J(c_{n})[/math] is the change in gradient

Approximating the inverse Hessian

The BFGS formula to update the inverse Hessian matrix is proper is

[math] \left[\nabla^{2}J\right]_{n+1}^{-1}=\left[\nabla^{2}J\right]_{n}^{-1}+ \left(1+\frac{\left\langle\Delta g\middle|\left[\nabla^{2}J\right]_{n}^{-1}\middle|\Delta g\right\rangle}{\left\langle \Delta c\middle|\Delta g\right\rangle} \right)\frac{\left| \Delta c\middle\rangle\middle\langle \Delta c\right|}{\left\langle \Delta c\middle|\Delta g\right\rangle} - \left(\frac{\left| \Delta c\middle\rangle\middle\langle \Delta g\middle|\middle[\nabla^{2}J\right]_{n}^{-1}+\left[\nabla^{2}J\middle]_{n}^{-1}\middle| \Delta g\middle\rangle\middle\langle \Delta c\right|}{\left\langle \Delta c\middle|\Delta g\right\rangle} \right) [/math]

where [math]\left[\nabla^{2}J\right]_{n}^{-1}[/math] is the matrix approximating the inverse Hessian at an iterate [math]n[/math].

Symmetric rank-1 update formula

Approximating the Hessian

The symmetric rank-1 (SR1) formula to update the Hessian matrix is proper is

[math] \left[\nabla^{2}J\right]_{n+1}=\left[\nabla^{2}J\right]_{n} - \left(\frac{\left(\left|\Delta g \right\rangle - \left[\nabla^{2}J\middle]_{n}\middle|\Delta c\right\rangle\right)\left(\left\langle\Delta c\middle|\middle[\nabla^{2}J\right]_{n} - \left\langle\Delta g \right|\right)}{\left\langle\Delta g \middle|\Delta c\right\rangle - \left\langle\Delta c\middle|\left[\nabla^{2}J\right]_{n}\middle|\Delta c\right\rangle }\right) [/math]


Approximating the inverse Hessian

The SR1 formula to update the inverse Hessian matrix is proper is

[math] \left[\nabla^{2}J\right]_{n+1}^{-1}=\left[\nabla^{2}J\right]_{n}^{-1} - \left(\frac{\left(\left|\Delta c \right\rangle - \left[\nabla^{2}J\middle]_{n}^{-1}\middle|\Delta g\right\rangle\right)\left(\left\langle\Delta g\middle|\middle[\nabla^{2}J\right]_{n}^{-1} - \left\langle\Delta c \right|\right)}{\left\langle\Delta c \middle|\Delta g\right\rangle - \left\langle\Delta g\middle|\left[\nabla^{2}J\right]_{n}^{-1}\middle|\Delta g\right\rangle }\right) [/math]


Arguments

    B_old          - Hessian approximation at the previous iterate.
    
    x_new          - new waveform at this iterate.
    
    x_old          - old waveform from the previous iterate.
    
    df_new         - new gradient at this iterate.
    
    df_old         - old gradient from the previous iterate.
    
    inv_method     - Boolean. Perform inverse Hessian update (true) or Hessian update (false).
    
    algo           - update mathod, either 'bfgs' or 'sr1'.

Returns

    B_new          - updated Hessian approximation

Notes

It is normal to use an initial Hessian approximation equal the the identity matrix, this is default in fminnewton.m.

See also

fminnewton.m, fminkrotov.m, hessreg.m, hessprep.m


Version 1.9, authors: Ilya Kuprov, David Goodwin