Phalanx  Development
 All Classes Functions Variables Typedefs Enumerations Friends Pages
Public Member Functions | Protected Member Functions | Protected Attributes | List of all members
PHX::DagManager< Traits > Class Template Reference

Class to generate the directed acyclic graph (DAG) for evaluation. Determined which Evaluators should be called and the order in which to call them such that all dependencies are met with consistency. More...

#include <Phalanx_DAG_Manager.hpp>

Collaboration diagram for PHX::DagManager< Traits >:
Collaboration graph
[legend]

Public Member Functions

 DagManager (const std::string &evaluator_type_name="???")
 
void requireField (const PHX::FieldTag &v)
 Require a variable to be evaluated.
 
void registerEvaluator (const Teuchos::RCP< PHX::Evaluator< Traits > > &p)
 Registers an evaluator with the manager.
 
void setDefaultGraphvizFilenameForErrors (const std::string &file_name)
 Sets the default filename for graphiz file generation for DAG construction errors.
 
void setWriteGraphvizFileOnError (bool write_file)
 If set to true, a graphviz file will be written during for DAG construction errors.
 
void sortAndOrderEvaluators ()
 
void postRegistrationSetup (typename Traits::SetupData d, PHX::FieldManager< Traits > &vm, const bool &buildDeviceDAG)
 
void evaluateFields (typename Traits::EvalData d)
 
void evaluateFieldsDeviceDag (const int &work_size, const int &team_size, const int &vector_size, typename Traits::EvalData d)
 
void preEvaluate (typename Traits::PreEvalData d)
 This routine is called before each residual/Jacobian fill. More...
 
void postEvaluate (typename Traits::PostEvalData d)
 This routine is called after each residual/Jacobian fill. More...
 
void setEvaluationTypeName (const std::string &evaluation_type_name)
 
const std::vector
< Teuchos::RCP< PHX::FieldTag > > & 
getFieldTags ()
 
bool sortingCalled () const
 Returns true if sortAndOrderEvaluators has been called.
 
void writeGraphvizFile (const std::string filename, bool writeEvaluatedFields, bool writeDependentFields, bool debugRegisteredEvaluators) const
 Write the DAG to file in graphviz/dot format. This is the deprecated version.
 
void writeGraphvizFileNew (const std::string filename, bool writeEvaluatedFields, bool writeDependentFields) const
 Write the DAG to file in graphviz/dot format.
 
void writeGraphviz (std::ostream &os, bool writeEvaluatedFields, bool writeDependentFields) const
 Write the DAG to std::ostream in graphviz/dot format.
 
void print (std::ostream &os) const
 Printing.
 
const std::vector< int > & getEvaluatorInternalOrdering () const
 Returns the Topological sort ordering. Used for unit testing.
 
const std::vector
< PHX::DagNode< Traits > > & 
getDagNodes () const
 Returns the internally registered nodes. Used for unit testing.
 
void analyzeGraph (double &speedup, double &parallelizability) const
 Returns the speedup and parallelizability of the graph. More...
 
std::vector< Teuchos::RCP
< PHX::Evaluator< Traits > > > & 
getEvaluatorsBindingField (const PHX::FieldTag &ft)
 Returns all evaluators that either evaluate or require the given field. This is used to bind memory for unmanaged views. More...
 
const std::unordered_map
< std::string, std::pair< int,
int > > & 
getFieldUseRange ()
 Returns the evaluator range that the field needs to exist over. More...
 
const std::unordered_map
< std::string, Teuchos::RCP
< PHX::FieldTag > > & 
getUnsharedFields ()
 
void printEvaluatorStartStopMessage (const Teuchos::RCP< std::ostream > &ostr)
 Print to user specified ostream when each evaluator starts and stops. Useful for debugging. Enabled only in debug builds. More...
 
const std::vector
< Teuchos::RCP< PHX::FieldTag > > & 
getRequiredFields () const
 Returns all fields that the user requested to to be evaluated by the field manager.
 
const std::unordered_map
< std::string, int > & 
queryRegisteredFields () const
 
const std::vector
< PHX::DagNode< Traits > > & 
queryRegisteredEvaluators () const
 

Protected Member Functions

void dfsVisit (PHX::DagNode< Traits > &node, int &time)
 Depth-first search algorithm.
 
void writeGraphvizDfsVisit (PHX::DagNode< Traits > &node, std::vector< PHX::DagNode< Traits >> &nodes_copy, std::ostream &os, const bool writeEvaluatedFields, const bool writeDependentFields) const
 Depth-first search algorithm specialized for writing graphviz output.
 
void printEvaluator (const PHX::Evaluator< Traits > &e, std::ostream &os) const
 Helper function.
 
void createEvaluatorBindingFieldMap ()
 

Protected Attributes

std::vector< Teuchos::RCP
< PHX::FieldTag > > 
required_fields_
 Fields required by the user.
 
std::vector< PHX::DagNode
< Traits > > 
nodes_
 Vector of all registered evaluators. More...
 
std::unordered_map
< std::string, int > 
field_to_node_index_
 Hash map of field key to evaluator index.
 
std::unordered_map
< std::string,
std::unordered_set< int > > 
contributed_field_to_node_index_
 Hash map of contributed field key to evaluator index.
 
std::vector< Teuchos::RCP
< PHX::FieldTag > > 
fields_
 All fields that are needed for the evaluation.
 
std::vector< Teuchos::RCP
< Teuchos::Time > > 
evalTimers
 
Evaluation Order Objects

Stores results from a topological sort on the evaluator DAG: the order to call evaluators to evaluate fields correctly.

std::vector< int > topoSortEvalIndex
 
std::string graphviz_filename_for_errors_
 Use this name for graphviz file output for DAG construction errors.
 
bool write_graphviz_file_on_error_
 If set to true, will write graphviz file for DAG construction errors.
 
std::string evaluation_type_name_
 
bool sorting_called_
 Flag to tell the setup has been called.
 
bool allow_multiple_evaluators_for_same_field_
 Backwards compatibility option: set to true to disable a check that throws if multiple registered evaluators can evaluate the same field. Original DFS algortihm allowed this. Refactor checks and throws.
 
std::unordered_map
< std::string, std::vector
< Teuchos::RCP< PHX::Evaluator
< Traits > > > > 
field_to_evaluators_binding_
 A map that returns all evaluators that bind the memory of a particular field. Key is unique field identifier.
 
bool build_device_dag_
 If set to true, allocated DeviceEvaluators for Device DAG for evaluation.
 
Kokkos::View
< PHX::DeviceEvaluatorPtr
< Traits > *, PHX::Device > 
device_evaluators_
 Contians pointers to DeviceEvaluators for Device DAG support.
 
Teuchos::RCP< std::ostream > start_stop_debug_ostream_
 If non-null, in debug builds, the DAG manager will print when an evaluator starts and stops.
 
std::unordered_map
< std::string, std::pair< int,
int > > 
field_use_range_
 Field use range for topologically sorted evalautors. Key is field identifier, value is inclusive start/stop range.
 
bool field_use_range_evaluated_
 True if the field use range has been evaluated.
 
std::unordered_map
< std::string, Teuchos::RCP
< PHX::FieldTag > > 
unshared_
 Fields the user has requested to NOT share memory.
 
bool unshared_evaluated_
 True if the unshared fields have been evaluated.
 

Detailed Description

template<typename Traits>
class PHX::DagManager< Traits >

Class to generate the directed acyclic graph (DAG) for evaluation. Determined which Evaluators should be called and the order in which to call them such that all dependencies are met with consistency.

Member Function Documentation

template<typename Traits >
void PHX::DagManager< Traits >::analyzeGraph ( double &  speedup,
double &  parallelizability 
) const

Returns the speedup and parallelizability of the graph.

Estimates are based on execution times. This will return garbage if the evaluateFields() call has not been made to log execution times.

template<typename Traits >
void PHX::DagManager< Traits >::evaluateFields ( typename Traits::EvalData  d)

Evaluate the required fields using data parallel evaluation on topological sort of tasks. Calls parallel_for for each node in DAG.

template<typename Traits >
void PHX::DagManager< Traits >::evaluateFieldsDeviceDag ( const int &  work_size,
const int &  team_size,
const int &  vector_size,
typename Traits::EvalData  d 
)

Evaluate the required fields using data parallel evaluation on topological sort of tasks. Uses Device DAG support, calling a single parallel_for for the entire DAG. This could be faster than the call to evaluateFields, but all nodes in the DAG are restricted to the same work_size. This is intended for CUDA builds where kernel launch overhead can be significant.

The parallel_for kernel launch below will not compile on CUDA unless relocatable device code (RDC) is enabled for the nvcc compiler. We also want to build and run phalanx without Device DAG support on CUDA (i.e. RDC off), so this ifdef will hide the RDC required code.

template<typename Traits >
std::vector< Teuchos::RCP< PHX::Evaluator< Traits > > > & PHX::DagManager< Traits >::getEvaluatorsBindingField ( const PHX::FieldTag ft)

Returns all evaluators that either evaluate or require the given field. This is used to bind memory for unmanaged views.

CAUTION: The returned vector is non-const to rebind memory for fields in evaluators. Be careful not to corrupt the actual vector.

References PHX::FieldTag::identifier().

template<typename Traits >
const std::vector< Teuchos::RCP< PHX::FieldTag > > & PHX::DagManager< Traits >::getFieldTags ( )

Returns the FieldTags for all fields involved in the evaluation. Will return an empty vector unless the user has built the DAG using one of the following calls: postRegistrationSetup(), postRegistrationSetupForType() or buildDagForType().

WARNING: This is a dangerous power user feature. It returns non-const field tags so that the fields can be sized after the DAG has been created.

template<typename Traits >
const std::unordered_map< std::string, std::pair< int, int > > & PHX::DagManager< Traits >::getFieldUseRange ( )

Returns the evaluator range that the field needs to exist over.

Once a topological sort of evalautors is performed, we have N evalautors in a specific order to traverse for the evaluation. Each field is used over a subset of the range of evaluators. We can reuse field memory if the use range between two fields does not overlap. This function returns the range over which each field needs to exist. The MemoryManager will use this information when binding fields.

Function is non-const due to lazy evalaution to construct.

Returns
a map where the key is the field identifier and the value is a pair of integers representing the inclusive use range [0,N-1] over which the field requires memory.
template<typename Traits >
const std::unordered_map< std::string, Teuchos::RCP< PHX::FieldTag > > & PHX::DagManager< Traits >::getUnsharedFields ( )

Returns a set of field tags for fields that the user has requested to NOT be shared with any other field. Unshared fields are used to trade off increased memory use for a reduction in flops for an evalautor. Unshared fields are a corner case where the user can leverage special knowledge about how data in a field changes across evaluations. One example use case is for FAD types during a Gather operation, where we know the off diagonal entries are always zero. The evaluator can zero out the FAD array during initialization and only change the diagonal (seed value) during an evalaution.

template<typename Traits >
void PHX::DagManager< Traits >::postEvaluate ( typename Traits::PostEvalData  d)

This routine is called after each residual/Jacobian fill.

This routine is called ONCE on the evaluator after the fill loop over elements is completed. This allows us to evaluate any post fill data. An example is to print out some statistics such as the maximum grid peclet number in a cell.

template<typename Traits >
void PHX::DagManager< Traits >::postRegistrationSetup ( typename Traits::SetupData  d,
PHX::FieldManager< Traits > &  vm,
const bool &  buildDeviceDAG 
)

Calls post registration setup on all evaluators.

template<typename Traits >
void PHX::DagManager< Traits >::preEvaluate ( typename Traits::PreEvalData  d)

This routine is called before each residual/Jacobian fill.

This routine is called ONCE on the evaluator before the fill loop over elements is started. This allows us to reset global objects between each fill. An example is to reset an evaluator that monitors the maximum grid peclet number in a cell. This call would zero out the maximum for a new fill.

template<typename Traits >
void PHX::DagManager< Traits >::printEvaluatorStartStopMessage ( const Teuchos::RCP< std::ostream > &  ostr)

Print to user specified ostream when each evaluator starts and stops. Useful for debugging. Enabled only in debug builds.

Parameters
[in]ostrRCP to output stream. If set to null, this disables printing.
template<typename Traits >
const std::vector< PHX::DagNode< Traits > > & PHX::DagManager< Traits >::queryRegisteredEvaluators ( ) const

Returns all nodes registered with the DagManager. This function is provided only for query, debug and unit testing.

template<typename Traits >
const std::unordered_map< std::string, int > & PHX::DagManager< Traits >::queryRegisteredFields ( ) const

Returns the internal fields this DagManager can currently evaluate and the index into the DAG node vector. This function is provided only for query, debug and unit testing.

template<typename Traits >
void PHX::DagManager< Traits >::sortAndOrderEvaluators ( )

Builds the evaluation DAG. This should only be called after all required fields and evaluators are registered. Must be called prior to making calls to postRegistrationSetup(), evaluateFields(), preEvaluate(), and postEvaluate(). This can be called multiple times to build a new DAG if requirements have changed or more evaluators have been added.

Member Data Documentation

template<typename Traits>
std::vector<PHX::DagNode<Traits> > PHX::DagManager< Traits >::nodes_
protected

Vector of all registered evaluators.

This list may include more nodes than what is needed for the DAG evaluation of required fields.


The documentation for this class was generated from the following files: