Implementation of limited Memory BFGS matrix for arbitrary vector spaces. More...

#include <ConstrainedOptPack_MatrixSymPosDefLBFGS.hpp>

Inheritance diagram for ConstrainedOptPack::MatrixSymPosDefLBFGS:

Classes
class	PostMod
	PostMod class to use with `MemMngPack::AbstractFactorStd`. More...

Public Member Functions
	MatrixSymPosDefLBFGS (size_type max_size=0, size_type m=10, bool maintain_original=true, bool maintain_inverse=true, bool auto_rescaling=false)
	Calls initial_setup(,,,) More...

	STANDARD_MEMBER_COMPOSITION_MEMBERS (bool, auto_rescaling)
	Set whether automatic rescaling is used or not. More...

void	initial_setup (size_type max_size=0, size_type m=10, bool maintain_original=true, bool maintain_inverse=true, bool auto_rescaling=false)
	Initial setup for the matrix. More...

size_type	m () const

size_type	m_bar () const

size_type	k_bar () const

value_type	gamma_k () const

const DMatrixSlice	S () const

const DMatrixSlice	Y () const

bool	maintain_original () const

bool	maintain_inverse () const

size_type	num_secant_updates () const
	Returns the total number of successful secant updates performed. More...

size_type	rows () const

Public types
typedef Teuchos::RCP< const MultiVector >	multi_vec_ptr_t

Constructors and initializers
	MatrixSymPosDefLBFGS (size_type m=10, bool maintain_original=true, bool maintain_inverse=true, bool auto_rescaling=false)
	Calls `this->initial_setup()` More...

	STANDARD_MEMBER_COMPOSITION_MEMBERS (bool, auto_rescaling)
	Set whether automatic rescaling is used or not. More...

void	initial_setup (size_type m=10, bool maintain_original=true, bool maintain_inverse=true, bool auto_rescaling=false)
	Initial setup for the matrix. More...

size_type	m () const

size_type	m_bar () const

value_type	gamma_k () const

const multi_vec_ptr_t	S () const

const multi_vec_ptr_t	Y () const

bool	maintain_original () const

bool	maintain_inverse () const

size_type	num_secant_updates () const
	Returns the total number of successful secant updates performed since `this->init_identity()` was called. More...

Overridden from MatrixOp
const VectorSpace &	space_cols () const

std::ostream &	output (std::ostream &out) const

MatrixOp &	operator= (const MatrixOp &mwo)

void	Vp_StMtV (VectorMutable *v_lhs, value_type alpha, BLAS_Cpp::Transp trans_rhs1, const Vector &v_rhs2, value_type beta) const

std::ostream &	output (std::ostream &out) const

MatrixOp &	operator= (const MatrixOp &m)

void	Vp_StMtV (DVectorSlice *vs_lhs, value_type alpha, BLAS_Cpp::Transp trans_rhs1, const DVectorSlice &vs_rhs2, value_type beta) const

Overridden from MatrixOpNonsing
void	V_InvMtV (VectorMutable *v_lhs, BLAS_Cpp::Transp trans_rhs1, const Vector &v_rhs2) const

Overridden from MatrixSymSecant
void	init_identity (const VectorSpace &space_diag, value_type alpha)

void	init_diagonal (const Vector &diag)
	Actually this calls init_identity( diag.space(), diag.norm_inf() ). More...

void	secant_update (VectorMutable s, VectorMutable y, VectorMutable *Bs)

void	init_identity (size_type n, value_type alpha)

void	init_diagonal (const DVectorSlice &diag)
	Actually this calls init_identity( (&diag)->size(), norm_inf(diag) ). More...

void	secant_update (DVectorSlice s, DVectorSlice y, DVectorSlice *Bs)

Overridden from MatrixWithOpFactorized
void	V_InvMtV (DVectorSlice *v_lhs, BLAS_Cpp::Transp trans_rhs1, const DVectorSlice &vs_rhs2) const

Overridden from MatrixSymAddDelUpdateble
void	initialize (value_type alpha, size_type max_size)
	This is fine as long as alpha > 0.0. More...

void	initialize (const DMatrixSliceSym &A, size_type max_size, bool force_factorization, Inertia inertia, PivotTolerances pivot_tols)
	Sorry, this will throw an exception! More...

size_type	max_size () const

Inertia	inertia () const
	Returns (0,0,rows()) More...

void	set_uninitialized ()
	Will set rows() == 0. More...

void	augment_update (const DVectorSlice *t, value_type alpha, bool force_refactorization, EEigenValType add_eigen_val, PivotTolerances pivot_tols)
	Augment the matrix to add a row and column. More...

void	delete_update (size_type jd, bool force_refactorization, EEigenValType drop_eigen_val, PivotTolerances pivot_tols)
	Should always succeed unless user gives a wrong value for drop_eigen_val. More...

Detailed Description

Implementation of limited Memory BFGS matrix for arbitrary vector spaces.

Implementation of limited Memory BFGS matrix.

The function set_num_updates_stored() must be called first to set the maximum number of the most recent updates that can be stored. The storage requirements for this class are O( n*m + m*m ) which is O(n*m) when n >> m which is expected (where n is the dimension of the vector space and m is the maximum number of updates stored).

This implementation is based on:

Byrd, Nocedal, and Schnabel, "Representations of quasi-Newton matrices and their use in limited memory methods", Mathematical Programming, 63 (1994)

Consider BFGS updates of the form:

( B^{k-1}, s^{k-1}, y^{k-1} ) -> B^{k}

where:

B^{k} = B^{k-1} - ( (B*s)*(B*s)' / (s'*B*s) )^{k-1} + ( (y*y') / (s'*y) )^{k-1}

B <: R^(n x n)
s <: R^(n)
y <: R^(n)

Now let us consider limited memory updating. For this implementation we set:

Bo = ( 1 / gamma_k ) * I

where:
              / (s^{k-1}'*y^{k-1})/(y^{k-1}'*y^{k-1})           :  if auto_rescaling() == true
    gamma_k = |
            \  1/alpha from last call to init_identity(n,alpha) :  otherwise

Now let us define the matrices S and Y that store the update vectors s^{i} and y^{i} for i = 1 ... m_bar:

S = [ s^{1}, s^{2},...,s^{m_bar} ] <: R^(n x m)
Y = [ y^{1}, y^{2},...,y^{m_bar} ] <: R^(n x m)

Here we are only storing the m_bar <= m most recent update vectors and their ordering is significant. The columns S(:,m_bar) and Y(:,m_bar) contain the most recent update vectors. This is all client needs to know in order to reconstruct the updates themselves.

This class allows matrix-vector products x = B*y and the inverse matrix-vector products x = inv(B)*y to be performed at a cost of about O(n*m_bar^2).

The function set_num_updates_stored(l) must be called first to set the maximum number of the most recent updates that can be stored. The storage requirements for this class are O( n_max*l + l*l ) which is O(n_max*l) when n_max >> l which is expected.

This implementation is based on:

Byrd, Nocedal, and Schnabel, "Representations of quasi-Newton matrices and their use in limited memory methods", Mathematical Programming, 63 (1994)

Consider BFGS updates of the form: {verbatim}

( B^{k-1}, s^{k-1}, y^{k-1} ) -> B^{k}

where:

B^{k} = B^{k-1} - ( (B*s)*(B*s)' / (s'*B*s) )^{k-1} + ( (y*y') / (s'*y) )^{k-1}

B <: R^(n x n) s <: R^(n) y <: R^(n)

{verbatim} Given that we start from the same initial matrix #Bo#, the updated matrix #B^{k}# will be the same independent of the order the #(s^{i},y^{i})# updates are added.

Now let us consider limited memory updating. For this implementation we set: {verbatim}

Bo = ( 1 / gamma_k ) * I

where: / (s^{k-1}'*y^{k-1})/(y^{k-1}'*y^{k-1}) : if auto_rescaling() == true gamma_k = | \ alpha from last call to init_identity(n,alpha) : otherwise

{verbatim} Now let us define the matrices S# and Y# that store the update vectors #s^{i}# and #y^{i}# for #i = 1 ... m_bar#: {verbatim}

S = [ s^{1}, s^{2},...,s^{m_bar} ] <: R^(n x m) Y = [ y^{1}, y^{2},...,y^{m_bar} ] <: R^(n x m)

{verbatim} Here we are only storing the m_bar <= m# most recent update vectors and their ordering is arbitrary. The columns #S(:,k_bar)# and #Y(:,k_bar)# contain the most recent update vectors. The next most recent vectors are to the left (i.e. #p = k_bar-1#) and so forth until #p = 1#. Then the next most recent update vectors start at m_bar# and move to the left until you reach the oldest update vector stored at column k_bar+1#. This is all client need to know in order to reconstruct the updates themselves.

This class allows matrix vector products x = B*y and the inverse matrix vector products x = inv(B)*y to be performed at a cost of about O(n*m^2).

In addition, the class supports the MatixSymAddDelUpdateable interface with a few major restrictions. This allows the client to add and remove rows and columns from the matrix.

Definition at line 110 of file ConstrainedOptPack_MatrixSymPosDefLBFGS.hpp.

Member Typedef Documentation

typedef Teuchos::RCP<const MultiVector> ConstrainedOptPack::MatrixSymPosDefLBFGS::multi_vec_ptr_t

Definition at line 120 of file ConstrainedOptPack_MatrixSymPosDefLBFGS.hpp.

Constructor & Destructor Documentation

ConstrainedOptPack::MatrixSymPosDefLBFGS::MatrixSymPosDefLBFGS	(	size_type	m = `10`,
		bool	maintain_original = `true`,
		bool	maintain_inverse = `true`,
		bool	auto_rescaling = `false`
	)

Calls this->initial_setup()

Definition at line 161 of file ConstrainedOptPack_MatrixSymPosDefLBFGS.cpp.

ConstrainedOptPack::MatrixSymPosDefLBFGS::MatrixSymPosDefLBFGS	(	size_type	max_size = `0`,
		size_type	m = `10`,
		bool	maintain_original = `true`,
		bool	maintain_inverse = `true`,
		bool	auto_rescaling = `false`
	)

Calls initial_setup(,,,)

Member Function Documentation

ConstrainedOptPack::MatrixSymPosDefLBFGS::STANDARD_MEMBER_COMPOSITION_MEMBERS	(	bool	,
		auto_rescaling
	)

Set whether automatic rescaling is used or not.

This function must be called before a BFGS update is performed in order for it to take effect for that update.

void ConstrainedOptPack::MatrixSymPosDefLBFGS::initial_setup	(	size_type	m = `10`,
		bool	maintain_original = `true`,
		bool	maintain_inverse = `true`,
		bool	auto_rescaling = `false`
	)

Initial setup for the matrix.

This function must be called before init_identity(n) is called in order for these setting to have affect. When this function is called all current updates are lost and the matrix becomes uninitialized.

Parameters

m	[in] Max number of recent update vectors s and `y` stored.
maintain_original	[in] If `true` then quantities needed to compute `x = Bk*y` will be maintained, otherwise they will not be unless needed. This is to save computational costs in case matrix-vector products will never be needed. However, if a matrix vector product is needed then these quantities will be computed on the fly in order to satisfy the request.
maintain_inverse	[in] If `true` then quantities needed to compute `x = inv(Bk)y = x = Hky` will be maintained, otherwise they will not be unless needed. This is to save computational costs in case inverse matrix-vector products will never be needed. However, if the inverse product is ever needed then the needed quantities will be computed on the fly in order to satisfiy the request. Because it takes so little extra work to maintain the quantities needed for `Hk` it is recommended to always set this to true.
auto_rescaling	[in] See intro.

Definition at line 171 of file ConstrainedOptPack_MatrixSymPosDefLBFGS.cpp.

size_type ConstrainedOptPack::MatrixSymPosDefLBFGS::m ( ) const

inline

Definition at line 360 of file ConstrainedOptPack_MatrixSymPosDefLBFGS.hpp.

size_type ConstrainedOptPack::MatrixSymPosDefLBFGS::m_bar ( ) const

inline

Definition at line 366 of file ConstrainedOptPack_MatrixSymPosDefLBFGS.hpp.

value_type ConstrainedOptPack::MatrixSymPosDefLBFGS::gamma_k ( ) const

inline

Definition at line 372 of file ConstrainedOptPack_MatrixSymPosDefLBFGS.hpp.

const DMatrixSlice ConstrainedOptPack::MatrixSymPosDefLBFGS::S ( ) const

inline

Definition at line 379 of file ConstrainedOptPack_MatrixSymPosDefLBFGS.hpp.

const DMatrixSlice ConstrainedOptPack::MatrixSymPosDefLBFGS::Y ( ) const

inline

Definition at line 386 of file ConstrainedOptPack_MatrixSymPosDefLBFGS.hpp.

bool ConstrainedOptPack::MatrixSymPosDefLBFGS::maintain_original ( ) const

inline

Definition at line 392 of file ConstrainedOptPack_MatrixSymPosDefLBFGS.hpp.

bool ConstrainedOptPack::MatrixSymPosDefLBFGS::maintain_inverse ( ) const

inline

Definition at line 398 of file ConstrainedOptPack_MatrixSymPosDefLBFGS.hpp.

size_type ConstrainedOptPack::MatrixSymPosDefLBFGS::num_secant_updates ( ) const

inline

Returns the total number of successful secant updates performed since this->init_identity() was called.

Definition at line 404 of file ConstrainedOptPack_MatrixSymPosDefLBFGS.hpp.

const VectorSpace & ConstrainedOptPack::MatrixSymPosDefLBFGS::space_cols ( ) const

virtual

Implements AbstractLinAlgPack::MatrixBase.

Definition at line 198 of file ConstrainedOptPack_MatrixSymPosDefLBFGS.cpp.

std::ostream & ConstrainedOptPack::MatrixSymPosDefLBFGS::output ( std::ostream & out ) const

virtual

Reimplemented from AbstractLinAlgPack::MatrixOp.

Definition at line 203 of file ConstrainedOptPack_MatrixSymPosDefLBFGS.cpp.

MatrixOp & ConstrainedOptPack::MatrixSymPosDefLBFGS::operator= ( const MatrixOp & mwo )

Definition at line 227 of file ConstrainedOptPack_MatrixSymPosDefLBFGS.cpp.

void ConstrainedOptPack::MatrixSymPosDefLBFGS::Vp_StMtV	(	VectorMutable *	v_lhs,
		value_type	alpha,
		BLAS_Cpp::Transp	trans_rhs1,
		const Vector &	v_rhs2,
		value_type	beta
	)		const

Definition at line 263 of file ConstrainedOptPack_MatrixSymPosDefLBFGS.cpp.

void ConstrainedOptPack::MatrixSymPosDefLBFGS::V_InvMtV	(	VectorMutable *	v_lhs,
		BLAS_Cpp::Transp	trans_rhs1,
		const Vector &	v_rhs2
	)		const

Definition at line 353 of file ConstrainedOptPack_MatrixSymPosDefLBFGS.cpp.

void ConstrainedOptPack::MatrixSymPosDefLBFGS::init_identity	(	const VectorSpace &	space_diag,
		value_type	alpha
	)

Definition at line 477 of file ConstrainedOptPack_MatrixSymPosDefLBFGS.cpp.

void ConstrainedOptPack::MatrixSymPosDefLBFGS::init_diagonal ( const Vector & diag )

Actually this calls init_identity( diag.space(), diag.norm_inf() ).

This initialization is not convienent for this implementation. Besides, when we are using automatric rescaling (auto_rescaling == true) then this will really not matter much anyway.

Definition at line 509 of file ConstrainedOptPack_MatrixSymPosDefLBFGS.cpp.

void ConstrainedOptPack::MatrixSymPosDefLBFGS::secant_update	(	VectorMutable *	s,
		VectorMutable *	y,
		VectorMutable *	Bs
	)

Definition at line 514 of file ConstrainedOptPack_MatrixSymPosDefLBFGS.cpp.

ConstrainedOptPack::MatrixSymPosDefLBFGS::STANDARD_MEMBER_COMPOSITION_MEMBERS	(	bool	,
		auto_rescaling
	)

Set whether automatic rescaling is used or not.

This function must be called before a BFGS update is performed in order for it to take effect for that update.

void ConstrainedOptPack::MatrixSymPosDefLBFGS::initial_setup	(	size_type	max_size = `0`,
		size_type	m = `10`,
		bool	maintain_original = `true`,
		bool	maintain_inverse = `true`,
		bool	auto_rescaling = `false`
	)

Initial setup for the matrix.

This function must be called before init_identity(n) is called. When this function is called all current updates are lost and the matrix becomes uninitialized.

Parameters

max_size	[in] If max_size > 0 then this is the max size the matrix is allowed to become. If max_size == 0 then this size will be determined by one of the initialization methods.
m	[in] Max number of recent update vectors stored
maintain_original	[in] If true then quantities needed to compute x = Bk*y will be maintained, otherwise they will not be unless needed. This is to save computational costs in case matrix vector products will never be needed. However, if a matrix vector product is needed then these quantities will be computed on the fly in order to satisfy the request.
maintain_inverse	[in] If true then quantities needed to compute x = inv(Bk)y = x = Hky will be maintained , otherwise they will not be unless needed. This is to save computational costs in case inverse matrix vector products will never be needed. However, if the inverse product is ever needed then the needed quantities will be computed on the fly in order to satisfiy the request. Because it takes so little extra work to maintain the quantities needed for Hk it is recommended to always set this to true.
auto_rescaling	[in] See intro.

size_type ConstrainedOptPack::MatrixSymPosDefLBFGS::m ( ) const

size_type ConstrainedOptPack::MatrixSymPosDefLBFGS::m_bar ( ) const

size_type ConstrainedOptPack::MatrixSymPosDefLBFGS::k_bar ( ) const

inline

Definition at line 398 of file ConstrainedOptPack_MatrixSymPosDefLBFGSSerial.hpp.

value_type ConstrainedOptPack::MatrixSymPosDefLBFGS::gamma_k ( ) const

const DMatrixSlice ConstrainedOptPack::MatrixSymPosDefLBFGS::S ( ) const

const DMatrixSlice ConstrainedOptPack::MatrixSymPosDefLBFGS::Y ( ) const

bool ConstrainedOptPack::MatrixSymPosDefLBFGS::maintain_original ( ) const

bool ConstrainedOptPack::MatrixSymPosDefLBFGS::maintain_inverse ( ) const

size_type ConstrainedOptPack::MatrixSymPosDefLBFGS::num_secant_updates ( ) const

Returns the total number of successful secant updates performed.

size_type ConstrainedOptPack::MatrixSymPosDefLBFGS::rows ( ) const

virtual

Reimplemented from AbstractLinAlgPack::MatrixBase.

std::ostream& ConstrainedOptPack::MatrixSymPosDefLBFGS::output ( std::ostream & out ) const

virtual

Reimplemented from AbstractLinAlgPack::MatrixOp.

MatrixOp& ConstrainedOptPack::MatrixSymPosDefLBFGS::operator= ( const MatrixOp & m )

void ConstrainedOptPack::MatrixSymPosDefLBFGS::Vp_StMtV	(	DVectorSlice *	vs_lhs,
		value_type	alpha,
		BLAS_Cpp::Transp	trans_rhs1,
		const DVectorSlice &	vs_rhs2,
		value_type	beta
	)		const

void ConstrainedOptPack::MatrixSymPosDefLBFGS::V_InvMtV	(	DVectorSlice *	v_lhs,
		BLAS_Cpp::Transp	trans_rhs1,
		const DVectorSlice &	vs_rhs2
	)		const

void ConstrainedOptPack::MatrixSymPosDefLBFGS::init_identity	(	size_type	n,
		value_type	alpha
	)

void ConstrainedOptPack::MatrixSymPosDefLBFGS::init_diagonal ( const DVectorSlice & diag )

Actually this calls init_identity( (&diag)->size(), norm_inf(diag) ).

This initialization is not convienent for this implementation. Besides, when we are using automatric rescaling (auto_rescaling == true) then this will really not matter much anyway.

void ConstrainedOptPack::MatrixSymPosDefLBFGS::secant_update	(	DVectorSlice *	s,
		DVectorSlice *	y,
		DVectorSlice *	Bs
	)

void ConstrainedOptPack::MatrixSymPosDefLBFGS::initialize	(	value_type	alpha,
		size_type	max_size
	)

This is fine as long as alpha > 0.0.

void ConstrainedOptPack::MatrixSymPosDefLBFGS::initialize	(	const DMatrixSliceSym &	A,
		size_type	max_size,
		bool	force_factorization,
		Inertia	inertia,
		PivotTolerances	pivot_tols
	)

Sorry, this will throw an exception!

size_type ConstrainedOptPack::MatrixSymPosDefLBFGS::max_size ( ) const

Inertia ConstrainedOptPack::MatrixSymPosDefLBFGS::inertia ( ) const

Returns (0,0,rows())

void ConstrainedOptPack::MatrixSymPosDefLBFGS::set_uninitialized ( )

Will set rows() == 0.

void ConstrainedOptPack::MatrixSymPosDefLBFGS::augment_update	(	const DVectorSlice *	t,
		value_type	alpha,
		bool	force_refactorization,
		EEigenValType	add_eigen_val,
		PivotTolerances	pivot_tols
	)

Augment the matrix to add a row and column.

This function is very limited in what it will do. It will throw exceptions if alpha <= 0.0 or t != NULL or add_eigen_val == EIGEN_VAL_NEG or this->rows() == this->max_size(). The obvious postconditions for this function will only technically be satisfied if alpha == this->gamma_k().

void ConstrainedOptPack::MatrixSymPosDefLBFGS::delete_update	(	size_type	jd,
		bool	force_refactorization,
		EEigenValType	drop_eigen_val,
		PivotTolerances	pivot_tols
	)

Should always succeed unless user gives a wrong value for drop_eigen_val.

The documentation for this class was generated from the following files:

Classes

Public Member Functions

Public types

Constructors and initializers

Overridden from MatrixOp

Overridden from MatrixOpNonsing

Overridden from MatrixSymSecant

Overridden from MatrixWithOpFactorized

Overridden from MatrixSymAddDelUpdateble

Detailed Description

Member Typedef Documentation

Constructor & Destructor Documentation

Member Function Documentation