Developer Notes

Description

This is a description Pat Notz called his elevator speech on Phalanx:

The {NAME} package is a library to help decompose a complex problem
into a set of simpler problems with managed dependencies. Two benefits
of decomposing a problem using {NAME} are (1) increased flexibility
because each simpler piece becomes an extension point that can be
swapped out with different implementations and (2) easier to craft
code because each piece is simpler, more focused and easier to test in
isolation.  The counter-benefits to these are (1) a potential
performance loss due to fragmentation of the over-all algorithm (e.g.,
several small loops instead of one large loop) and (2) a potential
loss of visibility of the original, composite problem (since the code
is scattered into multiple places).  Managing these trade-offs can
result in application code that both performs well and supports rapid
development and extensibility.  

Contiguous Allocator

Simple concept for chunk-layout of heterogeneous members with proper data alignment.

A 'malloc'ed chunk of memory is always suitable to hold any data type, or array of data types (do a 'man malloc').

        unsigned char * chunk = malloc( chunk_size );

If a 'double' is to occupy a portion of the chunk then is must be aligned such that:

        double * d = (double *)( chunk + offset_d );

where:

        0 == offset_d % sizeof(double);

The key point is for the offset of each member within the chunk to have the correct alignment, and of course not overlap any other member.

Old email with description of variable manager

Email description of Phalanx (formerly known as the variable manager):

Phalanx is a tool for controlling the evaluation of data used in the assembly of a residual/Jacobian fill for PDE solvers. I implemented the one in Charon and one also exists in Aria (although it is used for a different purpose). The idea behind the VM is to allow for fast changes to dependency trees. The VM provides variable quantities. A variable consists of a name (string), a type (scalar, vector, tensor - all templated for ad), and the place that it lives (quadrature point, node, edge, face, element, etc...). A variable is evaluated using a provider. The provider contains lists of variables it provides (evlauates) and a list of variables it requires to perform the evaluation (its dependencies). The VM figures out the correct order to call the providers to get the dependency chain correct.

I developed this for Charon to address the following issues:

1. Determine the order of variable evaluation so that the correct evaluation is performed: For example, to compute the density, it could be a constant, it could be a function of temperature or it could be a function of temperature and pressure. A factory creates the density provider and from this, the manager automatically figures out the correct evaluation order - whether T and/or P will have to be evaluated before the density. This is an easy example, but there are much more complex dependency chains.
2. Provides a single place to store all evaluation variables so that the procedure by which the variable is evaluated is not known by the objects using the variable. For example, in Charon a quantity of interest is the velocity components. The velocity could be a degree of freedom or it could be a user specified constant, or it could be a user specified field (read from an exodus file). Before we had the variable manager, everywhere in Charon that used velocity, there were repeated loops that looked like:

if (velocity is DOF)
  // get values from dof structure
else if (velocity is auxiliary data)
  // go get from aux data structure
else if (...)

There were hard coded loops in multiple places that had to be updated every time we added a new way to specify velocity. Now, when velocity is required, you just go to the VM and get it. No questions asked.

1. By having a single place to store the variables, we eliminated the reevaluation of temporary quantities that are used in different PDE operators and different equations. Prior to using the variable manager, the reuse of variables like density was ignored. It was recomputed for each operator evaluation. After implementing the variable manager, my residual evaluation times were reduced by a factor of 7 for reacting flow problems!
2. By having the evaluation order, one can perform automatic differentiation without any library like saccado. When the user writes a provider for a function, they can also write a local Jacobian for the individual function. Then by using the chain rule, the local element Jacobian can be assembled by traversing the dependency tree generated by the variable manager. Ariawrote their VM to do this, since Sacado was not around when they started. Aria does use Sacado for user defined functions, but uses the VM for internal functions.

There are other benefits, but you get the idea. A primary goal of my current ASC funding is to generalize the variable manager in Charon (there are many things I can do better now to make it more efficient and easier to use) as a trilinos package to be used with Intrepid to aid in element assembly for nonlinear equation sets.