Index

Overview

Phalanx is a local graph-based field evaluation toolkit. While the intended use case is for solving general partial differential equations (PDEs), there is NO specific code implemented for PDEs in Phalanx. It can be applied to any system that requires function evaluation. In terms of PDE discretization schemes it can be used for finite element, finite difference and finite volume.

Phalanx is a local node evaluation tool. Phalanx relies on the Kokkos package for performance portability and provides a simple performant interface to multicore and manycore architectures. While its main use is for large scale parallel high performance computing, the MPI communication (possibly required for ghosting) must be handled by other packages in the toolchain (separation of concerns). Users can handle this manually as Phalanx places no requirements on this but we recomend the Tpetra package in Trilinos be leveraged.

The main goal of Phalanx is to decompose a complex problem into a number of simpler problems with managed dependencies to support rapid development and extensibility. Through the use of template metaprogramming concepts, Phalanx supports arbitrary user defined data types and evaluation types. This allows for extreme flexibility in integration with user applications and provides extensive support for embedded technology such as automatic differentiation for sensitivity analysis, optimization, and uncertainty quantification. This approach, coupled with the template capabilities of C++ offers a number of unique and powerful capabilities:

Fast Integration with Flexible and Extensible Models:
- Increased flexibility because each simpler piece of the decomposed problem becomes an extension point that can be swapped out with different implementations.
- Easier to implement new code because each piece is simpler, more focused and easier to test in isolation.
- Easier for users to add their own models. While Phalanx is designed for maximum flexibility and efficiency, the interfaces exposed to users are very simple, requiring minimal training and/or knowledge of the C++ language to use.
- Through the use of template metaprogramming concepts, Phalanx supports arbitrary user defined data types. This allows for unprecedented flexibility for direct integration with user applications and opens the door to embedded technology.
Support for Advanced Embedded Technology: Phalanx is fully compatible with advanced embedded technologies. Embedded technology consists of replacing the default scalar type (i.e., "double" values) in a code with an object that overloads the typical mathematical operators. By replacing the scalar type, we can reuse the same code base to produce different information. For example, if we were to compute a function $F(x) \in R^n$ , where $F: R^n \rightarrow R^n$ and $x \in R^n$ . We could automatically compute the sensitivities of the function, such as the Jacobian, $J_{ij} = \frac{\partial F_i}{\partial x_j}$ merely by replacing the scalar type of double with an automatic differentiation (AD) object. With the C++ template mechanism, this is a trivial task. Embedded technology provides:
```
 <ul> 

 <li> Applications can reuse the exact same code base written to
 evaluate a function yet produce new information including
 arbitrary precision, sensitivities (via embedded automatic
 differentiation), and uncertainty quantification.  

 <li> By reusing the same code for both function and sensitivity
 evaluation, developers avoid having to hand code time consuming
 and error prone analytic derivatives and ensure that the equation
 sets are consistent.

 </ul>
```
Consistent field evaluations via dependency chain management: When users switch models, the dependencies for fields evaluated by the model can change. This can force field to be evaluated in a different order based on the new dependencies. Phalanx will automatically perform the sorting and ordering of the evaluator routines. For example, if density were a field to be evaluated, different models could have different dependencies. Model A could depend on temperature and pressure, whille model B could depend on temperature, pressure, and species mass fractions of a multicomponent mixture. The order of evaluation of fields could change based on the change in dependencies. Using Model B will require that the species mass fractions be evaluated and available for the density evaluation while Model A does not even require them. This is a simple example, but the dependency chain can become quite complex when solving thousands of coupled PDE equations.
Efficient evaluation of field data: Phalanx was designed to support the concept of worksets, although this is NOT REQUIRED. A workset is an arbitrarily sized block of cells (or egdesor faces or nodes) to be evaluated on the local processor. Phalanx evaluates all fields of interest at once for each block of cells. By using the contiguous allocator and correctly sizing the workset to fit in processor cache (if possible), one can arrange all fields to exist in a contiguous block of memory, independent of the data type objects used to store the data (one must still be careful of data alignement issues). By keeping all fields in cache, the code should run much faster. This workset idea will also, in the future, allow for multi-core distrubution of the cell evaluations.

NOTE: with the transition to the Kokkos::View as the underlying data type, the contiguous memory allocation is no longer supported as Kokkos will control data placement to maximize hardware locality (NUMA).

Phalanx is a hammer. It's use should be carefully considered. We recommend its use when writing a general PDE framework where one needs support for flexibility in equation sets, discretizations, and material models. It should not be used for a fixed set of equations and a single discretization that never changes (Although the template metaprogramming support could still be leveraged). There are some drawbacks to using Phalanx that should be considered:

A potential performance loss due to fragmentation of the over-all algorithm (e.g., several small loops instead of one large loop). Since the granularity of the directed acyclic graph is controlled by the user, a poor choice in functional decomposition is possible. This could result in inefficient use of memory and costly kernel launches if there is little work in the evaluator.
A potential loss of visibility of the original, composite problem (since the code is scattered into multiple places).

Managing these trade-offs can result in application code that both performs well and supports rapid development and extensibility.

News

Phalanx has just completed the transition from shards::Array to the Kokkos::View. Leveraging Kokkos, Phalanx is now performance portable to GPUs. The examples and user guide, however, have not been updated to reflect the transition. Please look at the MultiDimensionalArray example and the unit tests for examples on how to use Phalanx. We will update the user guide and other examples asap.

Reporting Bugs and Making Enhancement Requests

To reports bugs or make enhancement requests, visit the Trilinos Bugzilla (Bug Tracking) Database, and use the following instructions.

Click on "Enter a new bug report"
Choose "Phalanx"
Either login or create a new account
Submit your bug report

History

Phalanx grew out of the Variable Manger in the Charon code and the Expression Manager in the Aria code at Sandia National Laboratires. It is an attempt at merging the two capabilities.

Authors and Contributors

The following have contributed to the design through ideas, discussions, and/or code development of Phalanx:

Roger Pawlowski (Lead Developer), SNL
Irina Demeshko, SNL
Eric Phipps, SNL
Pat Notz, SNL

Copyright

// @HEADER
// ************************************************************************
//
//        Phalanx: A Partial Differential Equation Field Evaluation 
//       Kernel for Flexible Management of Complex Dependency Chains
//                    Copyright 2008 Sandia Corporation
//
// Under terms of Contract DE-AC04-94AL85000, there is a non-exclusive
// license for use of this work by or on behalf of the U.S. Government.
//
// Redistribution and use in source and binary forms, with or without
// modification, are permitted provided that the following conditions are
// met:
//
// 1. Redistributions of source code must retain the above copyright
// notice, this list of conditions and the following disclaimer.
//
// 2. Redistributions in binary form must reproduce the above copyright
// notice, this list of conditions and the following disclaimer in the
// documentation and/or other materials provided with the distribution.
//
// 3. Neither the name of the Corporation nor the names of the
// contributors may be used to endorse or promote products derived from
// this software without specific prior written permission.
//
// THIS SOFTWARE IS PROVIDED BY SANDIA CORPORATION "AS IS" AND ANY
// EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
// IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
// PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL SANDIA CORPORATION OR THE
// CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
// EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
// PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
// PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF
// LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING
// NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
// SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
//
// Questions? Contact Roger Pawlowski (rppawlo@sandia.gov), Sandia
// National Laboratories.
//
// ************************************************************************
// @HEADER

For All Questions and Comments...

Please contact Roger Pawlowski (rppaw.nosp@m.lo@s.nosp@m.andia.nosp@m..gov).