Description of Tpetra's behavior. More...

#include <Tpetra_Details_Behavior.hpp>

Static Public Member Functions
static bool	debug ()
	Whether Tpetra is in debug mode. More...

static bool	debug (const char name[])
	Whether the given Tpetra object is in debug mode. More...

static bool	verbose ()
	Whether Tpetra is in verbose mode. More...

static bool	verbose (const char name[])
	Whether the given Tpetra object is in verbose mode. More...

static void	disable_verbose_behavior ()
	Disable verbose mode, programatically. More...

static void	enable_verbose_behavior ()
	Enable verbose mode, programatically. More...

static bool	timing ()
	Whether Tpetra is in timing mode. More...

static bool	timing (const char name[])
	Whether the given Tpetra object is in timing mode. More...

static void	disable_timing ()
	Disable timing, programatically. More...

static void	enable_timing ()
	Enable timing, programatically. More...

static bool	assumeMpiIsCudaAware ()
	Whether to assume that MPI is CUDA aware. More...

static int	TAFC_OptimizationCoreCount ()
	MPI process count above which Tpetra::CrsMatrix::transferAndFillComplete will attempt to do advanced neighbor discovery. More...

static size_t	verbosePrintCountThreshold ()
	Number of entries below which arrays, lists, etc. will be printed in debug mode. More...

static size_t	rowImbalanceThreshold ()
	Threshold for deciding if a local matrix is "imbalanced" in the number of entries per row. The threshold is compared against the difference between maximum row length and average row length. More...

static bool	useMergePathMultiVector ()
	Whether to use the cuSPARSE merge path algorithm to perform sparse matrix-multivector products, one vector at a time. Depending on the matrix and the number of vectors in the multivector, this may be better than just applying the default SpMV algorithm to the entire multivector at once. More...

static bool	hierarchicalUnpack ()
	Unpack rows of a matrix using hierarchical unpacking. More...

static size_t	hierarchicalUnpackBatchSize ()
	Size of batch for hierarchical unpacking. More...

static size_t	hierarchicalUnpackTeamSize ()
	Size of team for hierarchical unpacking. More...

static size_t	multivectorKernelLocationThreshold ()
	the threshold for transitioning from device to host More...

static bool	profilingRegionUseTeuchosTimers ()
	Use Teuchos::Timer in Tpetra::ProfilingRegion. More...

static bool	profilingRegionUseKokkosProfiling ()
	Use Kokkos::Profiling in Tpetra::ProfilingRegion. More...

Detailed Description

Description of Tpetra's behavior.

"Behavior" means things like whether to do extra debug checks or print debug output. These depend both on build options and on environment variables. Build options generally control the default behavior.

This class' methods have the following properties:

They may read the environment.
They read the environment at most once, on first call.
They are idempotent.
They are reentrant. For example, they may use std::call_once if appropriate.

We intended for it to be inexpensive to call this class' methods repeatedly. The idea is that you don't have to cache variables; you should just call the functions freely. In the common case, the bool methods should just perform an 'if' test and just return the bool value. We spent some time thinking about how to make the methods reentrant without a possibly expensive mutex-like pthread_once / std::call_once cost on each call.

Tpetra does not promise to see changes to environment variables made after using any Tpetra class or calling any Tpetra function. Best practice would be to set any environment variables that you want to set, before starting the executable.

Our main goal with this class is to give both users and developers more run-time control in determining Tpetra's behavior, by setting environment variables. This makes debugging much more efficient, since before, enabling debugging code would have required reconfiguring and recompiling. Not all of Tpetra has bought into this system yet; some debug code is still protected by macros like HAVE_TPETRA_DEBUG. However, our goal is that as much Tpetra debugging code as possible can be enabled or disabled via environment variable. This will have the additional advantage of avoiding errors due to only building and testing in debug or release mode, but not both.

The behavior of Tpetra can be modified at runtime through two environment variables:

TPETRA_DEBUG: flags Tpetra to turn on debug checking. TPETRA_VERBOSE: flags Tpetra to turn on debug output. TPETRA_TIMING: flags Tpetra to turn on timing code.

These are two different things. For example, TPETRA_DEBUG may do extra MPI communication in order to ensure correct error state propagation, but TPETRA_DEBUG should never print copious debug output if no errors occurred. The idea is that if users get a mysterious error or hang, they can rerun with TPETRA_DEBUG set. TPETRA_VERBOSE is for Tpetra developers to use for debugging Tpetra. TPETRA_TIMING is for Tpetra developers to use for timing Tpetra.

The environment variables are understood to be "on" or "off" and recognized if specified in one of two ways. The first is to specify the variable unconditionally ON or OFF. e.g., TPETRA_[VERBOSE,DEBUG,TIMING]=ON or TPETRA_[VERBOSE,DEBUG,TIMING]=OFF. The default value of TPETRA_VERBOSE and TPETRA_TIMING is always OFF. The default value for TPETRA_DEBUG is ON if Tpetra is configured with Tpetra_ENABLE_DEBUG, otherwise it is OFF.

The second is to specify the variable on a per class/object basis, e.g., TPETRA_VERBOSE=CrsGraph,CrsMatrix,Distributor means that verbose output will be enabled for CrsGraph, CrsMatrix, and Distributor classes. For this second method, the default values of both TPETRA_VERBOSE and TPETRA_DEBUG is OFF.

Definition at line 122 of file Tpetra_Details_Behavior.hpp.

Member Function Documentation

bool Tpetra::Details::Behavior::debug ( )

static

Whether Tpetra is in debug mode.

"Debug mode" means that Tpetra does extra error checks that may require more MPI communication or local computation. It may also produce more detailed error messages, and more copious debug output.

Definition at line 298 of file Tpetra_Details_Behavior.cpp.

bool Tpetra::Details::Behavior::debug ( const char name[] )

static

Whether the given Tpetra object is in debug mode.

Parameters

name	[in] Name of the Tpetra object. Typically, the object would be a class name, e.g., "CrsGraph" or method, e.g., "CrsGraph::insertLocalIndices".

Definition at line 466 of file Tpetra_Details_Behavior.cpp.

bool Tpetra::Details::Behavior::verbose ( )

static

Whether Tpetra is in verbose mode.

"Verbose mode" means that Tpetra prints copious debug output to std::cerr on every MPI process. This is a LOT of output! You really don't want to do this when running on many MPI processes.

Definition at line 311 of file Tpetra_Details_Behavior.cpp.

bool Tpetra::Details::Behavior::verbose ( const char name[] )

static

Whether the given Tpetra object is in verbose mode.

Parameters

name	[in] Name of the Tpetra object. Typically, the object would be a class name, e.g., "CrsGraph" or method, e.g., "CrsGraph::insertLocalIndices".

Definition at line 478 of file Tpetra_Details_Behavior.cpp.

void Tpetra::Details::Behavior::disable_verbose_behavior ( )

static

Disable verbose mode, programatically.

Definition at line 496 of file Tpetra_Details_Behavior.cpp.

void Tpetra::Details::Behavior::enable_verbose_behavior ( )

static

Enable verbose mode, programatically.

Definition at line 492 of file Tpetra_Details_Behavior.cpp.

bool Tpetra::Details::Behavior::timing ( )

static

Whether Tpetra is in timing mode.

"Timing mode" means that Tpetra enables code that instruments internal timing.

Definition at line 326 of file Tpetra_Details_Behavior.cpp.

bool Tpetra::Details::Behavior::timing ( const char name[] )

static

Whether the given Tpetra object is in timing mode.

Parameters

name	[in] Name of the Tpetra object. Typically, the object would be a class name, e.g., "CrsGraph" or method, e.g., "CrsGraph::insertLocalIndices".

Definition at line 500 of file Tpetra_Details_Behavior.cpp.

void Tpetra::Details::Behavior::disable_timing ( )

static

Disable timing, programatically.

Definition at line 518 of file Tpetra_Details_Behavior.cpp.

void Tpetra::Details::Behavior::enable_timing ( )

static

Enable timing, programatically.

Definition at line 514 of file Tpetra_Details_Behavior.cpp.

bool Tpetra::Details::Behavior::assumeMpiIsCudaAware ( )

static

Whether to assume that MPI is CUDA aware.

An MPI implementation is "CUDA aware" if it can accept CUDA device buffers (Kokkos::CudaSpace) as send and receive buffers. You may control this behavior at run time via the TPETRA_ASSUME_CUDA_AWARE_MPI environment variable.

For a discussion, see Trilinos GitHub issues #1571 and #1088.

Definition at line 341 of file Tpetra_Details_Behavior.cpp.

int Tpetra::Details::Behavior::TAFC_OptimizationCoreCount ( )

static

MPI process count above which Tpetra::CrsMatrix::transferAndFillComplete will attempt to do advanced neighbor discovery.

This is platform dependent, and the user/developer should test each new platform for the correct value. You may control this at run time via the MM_TAFC_OptimizationCoreCount environment variable.

Definition at line 354 of file Tpetra_Details_Behavior.cpp.

size_t Tpetra::Details::Behavior::verbosePrintCountThreshold ( )

static

Number of entries below which arrays, lists, etc. will be printed in debug mode.

You may control this at run time via the TPETRA_VERBOSE_PRINT_COUNT_THRESHOLD environment variable.

Definition at line 368 of file Tpetra_Details_Behavior.cpp.

size_t Tpetra::Details::Behavior::rowImbalanceThreshold ( )

static

Threshold for deciding if a local matrix is "imbalanced" in the number of entries per row. The threshold is compared against the difference between maximum row length and average row length.

The threshold is measured in max number of entries in excess of the average (it is not a proportion between max and average).

If the "imbalance" of a local matrix is greater than this threshold, a different algorithm may be used for some operations like sparse matrix-vector multiply, packAndPrepare, and unpackAndCombine. You may control this at run time via the TPETRA_ROW_IMBALANCE_THRESHOLD environment variable.

Definition at line 379 of file Tpetra_Details_Behavior.cpp.

bool Tpetra::Details::Behavior::useMergePathMultiVector ( )

static

Whether to use the cuSPARSE merge path algorithm to perform sparse matrix-multivector products, one vector at a time. Depending on the matrix and the number of vectors in the multivector, this may be better than just applying the default SpMV algorithm to the entire multivector at once.

Note: full support for merge path SPMV on multivectors is coming soon.

You may control this at run time via the TPETRA_MULTIVECTOR_USE_MERGE_PATH environment variable (default: false)

Definition at line 390 of file Tpetra_Details_Behavior.cpp.

bool Tpetra::Details::Behavior::hierarchicalUnpack ( )

static

Unpack rows of a matrix using hierarchical unpacking.

Definition at line 522 of file Tpetra_Details_Behavior.cpp.

size_t Tpetra::Details::Behavior::hierarchicalUnpackBatchSize ( )

static

Size of batch for hierarchical unpacking.

Definition at line 412 of file Tpetra_Details_Behavior.cpp.

size_t Tpetra::Details::Behavior::hierarchicalUnpackTeamSize ( )

static

Size of team for hierarchical unpacking.

Definition at line 428 of file Tpetra_Details_Behavior.cpp.

size_t Tpetra::Details::Behavior::multivectorKernelLocationThreshold ( )

static

the threshold for transitioning from device to host

If the number of elements in the multivector does not exceed this threshold and the data is on host, then run the calculation on host. Otherwise, run on device. By default this is 10000, but may be altered by the environment variable TPETRA_VECTOR_DEVICE_THRESHOLD

Definition at line 401 of file Tpetra_Details_Behavior.cpp.

bool Tpetra::Details::Behavior::profilingRegionUseTeuchosTimers ( )

static

Use Teuchos::Timer in Tpetra::ProfilingRegion.

This is disabled by default. You may control this at run time via the TPETRA_USE_TEUCHOS_TIMERS environment variable.

Definition at line 443 of file Tpetra_Details_Behavior.cpp.

bool Tpetra::Details::Behavior::profilingRegionUseKokkosProfiling ( )

static

Use Kokkos::Profiling in Tpetra::ProfilingRegion.

This is enabled by default if KOKKOS_ENABLE_PROFILING is defined. You mau control this at run time via the TPETRA_USE_KOKKOS_PROFILING environment variable.

Definition at line 454 of file Tpetra_Details_Behavior.cpp.

The documentation for this class was generated from the following files:

Static Public Member Functions

Detailed Description

Member Function Documentation