Tpetra parallel linear algebra  Version of the Day
 All Classes Namespaces Files Functions Variables Typedefs Enumerations Enumerator Macros Pages
List of all members
Tpetra::Distributor Class Reference

Sets up and executes a communication plan for a Tpetra DistObject. More...

#include <Tpetra_Distributor.hpp>

Inherits Describable, and ParameterListAcceptorDefaultBase.

Public Member Functions

Constructors and destructor
 Distributor (const Teuchos::RCP< const Teuchos::Comm< int > > &comm)
 Construct using the specified communicator and default parameters. More...
 
 Distributor (const Teuchos::RCP< const Teuchos::Comm< int > > &comm, const Teuchos::RCP< Teuchos::FancyOStream > &out)
 Construct using the specified communicator and default parameters, with an output stream. More...
 
 Distributor (const Teuchos::RCP< const Teuchos::Comm< int > > &comm, const Teuchos::RCP< Teuchos::ParameterList > &plist)
 Construct using the specified communicator and ParameterList. More...
 
 Distributor (const Teuchos::RCP< const Teuchos::Comm< int > > &comm, const Teuchos::RCP< Teuchos::FancyOStream > &out, const Teuchos::RCP< Teuchos::ParameterList > &plist)
 Construct using the specified communicator and ParameterList, with an output stream. More...
 
 Distributor (const Distributor &distributor)
 Copy constructor. More...
 
virtual ~Distributor ()=default
 Destructor (virtual for memory safety). More...
 
void swap (Distributor &rhs)
 Swap the contents of rhs with those of *this. More...
 
Implementation of ParameterListAcceptorDefaultBase
void setParameterList (const Teuchos::RCP< Teuchos::ParameterList > &plist)
 Set Distributor parameters. More...
 
Teuchos::RCP< const
Teuchos::ParameterList > 
getValidParameters () const
 List of valid Distributor parameters. More...
 
Gather / scatter "constructors"
size_t createFromSends (const Teuchos::ArrayView< const int > &exportProcIDs)
 Set up Distributor using list of process ranks to which this process will send. More...
 
template<class Ordinal >
void createFromRecvs (const Teuchos::ArrayView< const Ordinal > &remoteIDs, const Teuchos::ArrayView< const int > &remoteProcIDs, Teuchos::Array< Ordinal > &exportIDs, Teuchos::Array< int > &exportProcIDs)
 Set up Distributor using list of process ranks from which to receive. More...
 
void createFromSendsAndRecvs (const Teuchos::ArrayView< const int > &exportProcIDs, const Teuchos::ArrayView< const int > &remoteProcIDs)
 Set up Distributor using list of process ranks to which to send, and list of process ranks from which to receive. More...
 
Attribute accessor methods
size_t getNumReceives () const
 The number of processes from which we will receive data. More...
 
size_t getNumSends () const
 The number of processes to which we will send data. More...
 
bool hasSelfMessage () const
 Whether the calling process will send or receive messages to itself. More...
 
size_t getMaxSendLength () const
 Maximum number of values this process will send to another single process. More...
 
size_t getTotalReceiveLength () const
 Total number of values this process will receive from other processes. More...
 
Teuchos::ArrayView< const int > getProcsFrom () const
 Ranks of the processes sending values to this process. More...
 
Teuchos::ArrayView< const int > getProcsTo () const
 Ranks of the processes to which this process will send values. More...
 
Teuchos::ArrayView< const size_t > getLengthsFrom () const
 Number of values this process will receive from each process. More...
 
Teuchos::ArrayView< const size_t > getLengthsTo () const
 Number of values this process will send to each process. More...
 
Details::EDistributorHowInitialized howInitialized () const
 Return an enum indicating whether and how a Distributor was initialized. More...
 
Reverse communication methods
Teuchos::RCP< DistributorgetReverse () const
 A reverse communication plan Distributor. More...
 
Methods for executing a communication plan
template<class Packet >
void doPostsAndWaits (const Teuchos::ArrayView< const Packet > &exports, size_t numPackets, const Teuchos::ArrayView< Packet > &imports)
 Execute the (forward) communication plan. More...
 
template<class Packet >
void doPostsAndWaits (const Teuchos::ArrayView< const Packet > &exports, const Teuchos::ArrayView< const size_t > &numExportPacketsPerLID, const Teuchos::ArrayView< Packet > &imports, const Teuchos::ArrayView< const size_t > &numImportPacketsPerLID)
 Execute the (forward) communication plan. More...
 
template<class Packet >
void doPosts (const Teuchos::ArrayRCP< const Packet > &exports, size_t numPackets, const Teuchos::ArrayRCP< Packet > &imports)
 Post the data for a forward plan, but do not execute the waits yet. More...
 
template<class Packet >
void doPosts (const Teuchos::ArrayRCP< const Packet > &exports, const Teuchos::ArrayView< const size_t > &numExportPacketsPerLID, const Teuchos::ArrayRCP< Packet > &imports, const Teuchos::ArrayView< const size_t > &numImportPacketsPerLID)
 Post the data for a forward plan, but do not execute the waits yet. More...
 
void doWaits ()
 
template<class Packet >
void doReversePostsAndWaits (const Teuchos::ArrayView< const Packet > &exports, size_t numPackets, const Teuchos::ArrayView< Packet > &imports)
 Execute the reverse communication plan. More...
 
template<class Packet >
void doReversePostsAndWaits (const Teuchos::ArrayView< const Packet > &exports, const Teuchos::ArrayView< const size_t > &numExportPacketsPerLID, const Teuchos::ArrayView< Packet > &imports, const Teuchos::ArrayView< const size_t > &numImportPacketsPerLID)
 Execute the reverse communication plan. More...
 
template<class Packet >
void doReversePosts (const Teuchos::ArrayRCP< const Packet > &exports, size_t numPackets, const Teuchos::ArrayRCP< Packet > &imports)
 Post the data for a reverse plan, but do not execute the waits yet. More...
 
template<class Packet >
void doReversePosts (const Teuchos::ArrayRCP< const Packet > &exports, const Teuchos::ArrayView< const size_t > &numExportPacketsPerLID, const Teuchos::ArrayRCP< Packet > &imports, const Teuchos::ArrayView< const size_t > &numImportPacketsPerLID)
 Post the data for a reverse plan, but do not execute the waits yet. More...
 
void doReverseWaits ()
 
template<class ExpView , class ImpView >
std::enable_if
<(Kokkos::Impl::is_view
< ExpView >::value
&&Kokkos::Impl::is_view
< ImpView >::value)>::type 
doPostsAndWaits (const ExpView &exports, size_t numPackets, const ImpView &imports)
 Execute the (forward) communication plan. More...
 
template<class ExpView , class ImpView >
std::enable_if
<(Kokkos::Impl::is_view
< ExpView >::value
&&Kokkos::Impl::is_view
< ImpView >::value)>::type 
doPostsAndWaits (const ExpView &exports, const Teuchos::ArrayView< const size_t > &numExportPacketsPerLID, const ImpView &imports, const Teuchos::ArrayView< const size_t > &numImportPacketsPerLID)
 Execute the (forward) communication plan. More...
 
template<class ExpView , class ImpView >
std::enable_if
<(Kokkos::Impl::is_view
< ExpView >::value
&&Kokkos::Impl::is_view
< ImpView >::value)>::type 
doPosts (const ExpView &exports, size_t numPackets, const ImpView &imports)
 Post the data for a forward plan, but do not execute the waits yet. More...
 
template<class ExpView , class ImpView >
std::enable_if
<(Kokkos::Impl::is_view
< ExpView >::value
&&Kokkos::Impl::is_view
< ImpView >::value)>::type 
doPosts (const ExpView &exports, const Teuchos::ArrayView< const size_t > &numExportPacketsPerLID, const ImpView &imports, const Teuchos::ArrayView< const size_t > &numImportPacketsPerLID)
 Post the data for a forward plan, but do not execute the waits yet. More...
 
template<class ExpView , class ImpView >
std::enable_if
<(Kokkos::Impl::is_view
< ExpView >::value
&&Kokkos::Impl::is_view
< ImpView >::value)>::type 
doReversePostsAndWaits (const ExpView &exports, size_t numPackets, const ImpView &imports)
 Execute the reverse communication plan. More...
 
template<class ExpView , class ImpView >
std::enable_if
<(Kokkos::Impl::is_view
< ExpView >::value
&&Kokkos::Impl::is_view
< ImpView >::value)>::type 
doReversePostsAndWaits (const ExpView &exports, const Teuchos::ArrayView< const size_t > &numExportPacketsPerLID, const ImpView &imports, const Teuchos::ArrayView< const size_t > &numImportPacketsPerLID)
 Execute the reverse communication plan. More...
 
template<class ExpView , class ImpView >
std::enable_if
<(Kokkos::Impl::is_view
< ExpView >::value
&&Kokkos::Impl::is_view
< ImpView >::value)>::type 
doReversePosts (const ExpView &exports, size_t numPackets, const ImpView &imports)
 Post the data for a reverse plan, but do not execute the waits yet. More...
 
template<class ExpView , class ImpView >
std::enable_if
<(Kokkos::Impl::is_view
< ExpView >::value
&&Kokkos::Impl::is_view
< ImpView >::value)>::type 
doReversePosts (const ExpView &exports, const Teuchos::ArrayView< const size_t > &numExportPacketsPerLID, const ImpView &imports, const Teuchos::ArrayView< const size_t > &numImportPacketsPerLID)
 Post the data for a reverse plan, but do not execute the waits yet. More...
 
void getLastDoStatistics (size_t &bytes_sent, size_t &bytes_recvd) const
 Information on the last call to do/doReverse. More...
 
Implementation of Teuchos::Describable
std::string description () const
 Return a one-line description of this object. More...
 
void describe (Teuchos::FancyOStream &out, const Teuchos::EVerbosityLevel verbLevel=Teuchos::Describable::verbLevel_default) const
 Describe this object in a human-readable way to the given output stream. More...
 

Detailed Description

Sets up and executes a communication plan for a Tpetra DistObject.

Note
Most Tpetra users do not need to know about this class.

This class encapsulates the general information and communication services needed for subclasses of DistObject (such as CrsMatrix and MultiVector) to do data redistribution (Import and Export) operations. It is an implementation detail of Import and Export; in particular; it actually does the communication.

Here is the typical way to use this class:

  1. Create a Distributor. (The constructor is inexpensive.)
  2. Set up the Distributor once, using one of the two "plan creation" methods: either createFromSends(), or createFromRecvs(). This may be more expensive and communication-intensive than Step 3.
  3. Communicate the data by calling doPostsAndWaits() (forward mode), or doReversePostsAndWaits() (reverse mode). You may do this multiple times with the same Distributor instance.

Step 2 is expensive, but you can amortize its cost over multiple uses of the Distributor for communication (Step 3). You may also separate out "posts" (invoking nonblocking communication) and "waits" (waiting for that communication to complete), by calling doPosts() (resp. doReversePosts()), then doWaits() (resp. doReverseWaits()). This is useful if you have local work to do between the posts and waits, because it may overlap communication with computation. Whether it actually does overlap, depends on both the MPI implementation and your choice of parameters for the Distributor.

Instances of Distributor take the following parameters that control communication and debug output:

Definition at line 190 of file Tpetra_Distributor.hpp.

Constructor & Destructor Documentation

Tpetra::Distributor::Distributor ( const Teuchos::RCP< const Teuchos::Comm< int > > &  comm)
explicit

Construct using the specified communicator and default parameters.

Parameters
comm[in] Communicator used by the Distributor. MUST be nonnull.

The constructor doesn't actually set up the distribution pattern. You need to call one of the "gather / scatter 'constructors'" to do that.

Definition at line 174 of file Tpetra_Distributor.cpp.

Tpetra::Distributor::Distributor ( const Teuchos::RCP< const Teuchos::Comm< int > > &  comm,
const Teuchos::RCP< Teuchos::FancyOStream > &  out 
)

Construct using the specified communicator and default parameters, with an output stream.

Parameters
comm[in] Communicator used by the Distributor. MUST be nonnull.
out[in/out] Output stream (for debugging output). If null, the default is std::cerr.

The constructor doesn't actually set up the distribution pattern. You need to call one of the "gather / scatter 'constructors'" to do that.

Definition at line 179 of file Tpetra_Distributor.cpp.

Tpetra::Distributor::Distributor ( const Teuchos::RCP< const Teuchos::Comm< int > > &  comm,
const Teuchos::RCP< Teuchos::ParameterList > &  plist 
)

Construct using the specified communicator and ParameterList.

Parameters
comm[in] Communicator used by the Distributor. MUST be nonnull.
plist[in/out] List of parameters controlling how the Distributor performs communication, and specifying debug options. If null, all parameters take their default values. Please see the class documentation for a list of all accepted parameters and their default values.

The constructor doesn't actually set up the distribution pattern. You need to call one of the "gather / scatter 'constructors'" to do that.

Definition at line 185 of file Tpetra_Distributor.cpp.

Tpetra::Distributor::Distributor ( const Teuchos::RCP< const Teuchos::Comm< int > > &  comm,
const Teuchos::RCP< Teuchos::FancyOStream > &  out,
const Teuchos::RCP< Teuchos::ParameterList > &  plist 
)

Construct using the specified communicator and ParameterList, with an output stream.

Parameters
comm[in] Communicator used by the Distributor. MUST be nonnull.
out[in/out] Output stream (for debugging output). If null, the default is std::cerr.
plist[in/out] List of parameters controlling how the Distributor performs communication, and specifying debug options. If null, all parameters take their default values. Please see the class documentation for a list of all accepted parameters and their default values.

The constructor doesn't actually set up the distribution pattern. You need to call one of the "gather / scatter 'constructors'" to do that.

Definition at line 147 of file Tpetra_Distributor.cpp.

Tpetra::Distributor::Distributor ( const Distributor distributor)

Copy constructor.

Definition at line 191 of file Tpetra_Distributor.cpp.

virtual Tpetra::Distributor::~Distributor ( )
virtualdefault

Destructor (virtual for memory safety).

Precondition
No outstanding communication requests. (We could check, but see GitHub Issue #1303.)

Member Function Documentation

void Tpetra::Distributor::swap ( Distributor rhs)

Swap the contents of rhs with those of *this.

This is useful in Import's setUnion() method. It avoids the overhead of copying arrays, since it can use std::swap on the arrays.

Definition at line 232 of file Tpetra_Distributor.cpp.

void Tpetra::Distributor::setParameterList ( const Teuchos::RCP< Teuchos::ParameterList > &  plist)

Set Distributor parameters.

Please see the class documentation for a list of all accepted parameters and their default values.

Definition at line 281 of file Tpetra_Distributor.cpp.

Teuchos::RCP< const Teuchos::ParameterList > Tpetra::Distributor::getValidParameters ( ) const

List of valid Distributor parameters.

Please see the class documentation for a list of all accepted parameters and their default values.

Definition at line 352 of file Tpetra_Distributor.cpp.

size_t Tpetra::Distributor::createFromSends ( const Teuchos::ArrayView< const int > &  exportProcIDs)

Set up Distributor using list of process ranks to which this process will send.

Take a list of process ranks and construct a plan for efficiently scattering to those processes. Return the number of processes which will send me (the calling process) data.

Parameters
exportProcIDs[in] List of ranks of the processes that will get the exported data. If there is a process rank greater than or equal to the number of processes, all processes will throw an std::runtime_error exception. Process ranks less than zero are ignored; their placement corresponds to null sends in any future exports. That is, if exportProcIDs[0] == -1, then the corresponding position in the export array is ignored during a call to doPosts() or doPostsAndWaits(). For this reason, a negative entry is sufficient to break contiguity.
Returns
Number of imports this process will be receiving.

Definition at line 1041 of file Tpetra_Distributor.cpp.

template<class Ordinal >
void Tpetra::Distributor::createFromRecvs ( const Teuchos::ArrayView< const Ordinal > &  remoteIDs,
const Teuchos::ArrayView< const int > &  remoteProcIDs,
Teuchos::Array< Ordinal > &  exportIDs,
Teuchos::Array< int > &  exportProcIDs 
)

Set up Distributor using list of process ranks from which to receive.

Take a list of process ranks and construct a plan for efficiently scattering to those processes. Return the number and list of IDs being sent by me (the calling process).

Import invokes this method in order to create a Distributor from a list of receive neighbors and IDs. A common use case for this process is setting up sends and receives for the remote entries of the source vector in a distributed sparse matrix-vector multiply. The Mantevo HPCCG miniapp shows an annotated and simplified version of this process for that special case.

Parameters
remoteIDs[in] List of remote IDs wanted.
remoteProcIDs[in] The ranks of the process that will send the remote IDs listed in remoteIDs. Process ranks less than zero are ignored; their placement corresponds to null sends in any future exports. If there is a process rank greater than or equal to the number of processes, all processes will throw an std::runtime_error exception.
exportIDs[out] List of IDs that need to be sent from this process.
exportProcIDs[out] The ranks of the processes that will get the exported IDs in exportIDs.

The exportGIDs and exportProcIDs arrays are resized by the Distributor, which is why they are passed in as a nonconst Teuchos::Array reference.

void Tpetra::Distributor::createFromSendsAndRecvs ( const Teuchos::ArrayView< const int > &  exportProcIDs,
const Teuchos::ArrayView< const int > &  remoteProcIDs 
)

Set up Distributor using list of process ranks to which to send, and list of process ranks from which to receive.

Parameters
exportProcIDs[in] List of process ranks to which this process must send a message.
remoteProcIDs[in] List of process ranks from which this process must receive a message.

Definition at line 1377 of file Tpetra_Distributor.cpp.

size_t Tpetra::Distributor::getNumReceives ( ) const

The number of processes from which we will receive data.

The count does not include the calling process.

Definition at line 407 of file Tpetra_Distributor.cpp.

size_t Tpetra::Distributor::getNumSends ( ) const

The number of processes to which we will send data.

The count does not include the calling process.

Definition at line 413 of file Tpetra_Distributor.cpp.

bool Tpetra::Distributor::hasSelfMessage ( ) const

Whether the calling process will send or receive messages to itself.

Definition at line 410 of file Tpetra_Distributor.cpp.

size_t Tpetra::Distributor::getMaxSendLength ( ) const

Maximum number of values this process will send to another single process.

Definition at line 416 of file Tpetra_Distributor.cpp.

size_t Tpetra::Distributor::getTotalReceiveLength ( ) const

Total number of values this process will receive from other processes.

Definition at line 404 of file Tpetra_Distributor.cpp.

Teuchos::ArrayView< const int > Tpetra::Distributor::getProcsFrom ( ) const

Ranks of the processes sending values to this process.

This is a nonpersisting view. It will last only as long as this Distributor instance does.

Definition at line 419 of file Tpetra_Distributor.cpp.

Teuchos::ArrayView< const int > Tpetra::Distributor::getProcsTo ( ) const

Ranks of the processes to which this process will send values.

This is a nonpersisting view. It will last only as long as this Distributor instance does.

Definition at line 425 of file Tpetra_Distributor.cpp.

Teuchos::ArrayView< const size_t > Tpetra::Distributor::getLengthsFrom ( ) const

Number of values this process will receive from each process.

This process will receive getLengthsFrom[i] values from process getProcsFrom[i].

This is a nonpersisting view. It will last only as long as this Distributor instance does.

Definition at line 422 of file Tpetra_Distributor.cpp.

Teuchos::ArrayView< const size_t > Tpetra::Distributor::getLengthsTo ( ) const

Number of values this process will send to each process.

This process will send getLengthsTo[i] values to process getProcsTo[i].

This is a nonpersisting view. It will last only as long as this Distributor instance does.

Definition at line 428 of file Tpetra_Distributor.cpp.

Details::EDistributorHowInitialized Tpetra::Distributor::howInitialized ( ) const
inline

Return an enum indicating whether and how a Distributor was initialized.

This is an implementation detail of Tpetra. Please do not call this method or rely on it existing in your code.

Definition at line 422 of file Tpetra_Distributor.hpp.

Teuchos::RCP< Distributor > Tpetra::Distributor::getReverse ( ) const

A reverse communication plan Distributor.

The first time this method is called, it creates a Distributor with the reverse communication plan of *this. On subsequent calls, it returns the cached reverse Distributor.

Most users do not need to call this method. If you invoke doReversePosts() or doReversePostsAndWaits(), the reverse Distributor will be created automatically if it does not yet exist.

Definition at line 432 of file Tpetra_Distributor.cpp.

template<class Packet >
void Tpetra::Distributor::doPostsAndWaits ( const Teuchos::ArrayView< const Packet > &  exports,
size_t  numPackets,
const Teuchos::ArrayView< Packet > &  imports 
)

Execute the (forward) communication plan.

Call this version of the method when you have the same number of Packets for each LID (local ID) to send or receive.

Template Parameters
PacketThe type of data to send and receive.
Parameters
exports[in] Contains the values to be sent by this process. On exit from this method, it's OK to modify the entries of this buffer.
numPackets[in] The number of Packets per export / import. This version of the routine assumes that each LID has the same number of Packets associated with it. (MultiVector is an example of a DistObject subclass satisfying this property.)
imports[out] On entry, buffer must be large enough to accomodate the data exported (sent) to us. On exit, contains the values exported to us.

Definition at line 1037 of file Tpetra_Distributor.hpp.

template<class Packet >
void Tpetra::Distributor::doPostsAndWaits ( const Teuchos::ArrayView< const Packet > &  exports,
const Teuchos::ArrayView< const size_t > &  numExportPacketsPerLID,
const Teuchos::ArrayView< Packet > &  imports,
const Teuchos::ArrayView< const size_t > &  numImportPacketsPerLID 
)

Execute the (forward) communication plan.

Call this version of the method when you have possibly different numbers of Packets for each LID (local ID) to send or receive.

Template Parameters
PacketThe type of data to send and receive.
Parameters
exports[in] Contains the values to be sent by this process. On exit from this method, it's OK to modify the entries of this buffer.
numExportPacketsPerLID[in] The number of packets for each export LID (i.e., each LID to be sent).
imports[out] On entry, buffer must be large enough to accomodate the data exported (sent) to us. On exit, contains the values exported to us.
numImportPacketsPerLID[in] The number of packets for each import LID (i.e., each LID to be received).

Definition at line 1087 of file Tpetra_Distributor.hpp.

template<class Packet >
void Tpetra::Distributor::doPosts ( const Teuchos::ArrayRCP< const Packet > &  exports,
size_t  numPackets,
const Teuchos::ArrayRCP< Packet > &  imports 
)

Post the data for a forward plan, but do not execute the waits yet.

Call this overload when you have the same number of Packets for each LID to send or receive.

Template Parameters
PacketThe type of data to send and receive.
Parameters
exports[in] Contains the values to be sent by this process. This is an ArrayRCP and not an ArrayView so that we have the freedom to use nonblocking sends if we wish. Do not modify the data in this array until doWaits() has completed.
numPackets[in] The number of Packets per export / import. (Same as the three-argument version of doPostsAndWaits().)
imports[out] On entry, buffer must be large enough to accomodate the data exported (sent) to us. This is an ArrayRCP and not an ArrayView so that we have the freedom to use nonblocking sends if we wish. Do not modify the data in this array until doWaits() has completed. Upon completion of doWaits(), this buffer contains the values exported to us.

Definition at line 1133 of file Tpetra_Distributor.hpp.

template<class Packet >
void Tpetra::Distributor::doPosts ( const Teuchos::ArrayRCP< const Packet > &  exports,
const Teuchos::ArrayView< const size_t > &  numExportPacketsPerLID,
const Teuchos::ArrayRCP< Packet > &  imports,
const Teuchos::ArrayView< const size_t > &  numImportPacketsPerLID 
)

Post the data for a forward plan, but do not execute the waits yet.

Call this overload when you have possibly different numbers of Packets for each LID to send or receive.

Template Parameters
PacketThe type of data to send and receive.
Parameters
exports[in] Same as in the three-argument version of doPosts().
numExportPacketsPerLID[in] Same as in the four-argument version of doPostsAndWaits().
imports[out] Same as in the three-argument version of doPosts().
numImportPacketsPerLID[in] Same as in the four-argument version of doPostsAndWaits().

Definition at line 1508 of file Tpetra_Distributor.hpp.

void Tpetra::Distributor::doWaits ( )

Wait on any outstanding nonblocking message requests to complete.

This method is for forward mode communication only, that is, after calling doPosts(). For reverse mode communication (after calling doReversePosts()), call doReverseWaits() instead.

Definition at line 518 of file Tpetra_Distributor.cpp.

template<class Packet >
void Tpetra::Distributor::doReversePostsAndWaits ( const Teuchos::ArrayView< const Packet > &  exports,
size_t  numPackets,
const Teuchos::ArrayView< Packet > &  imports 
)

Execute the reverse communication plan.

This method takes the same arguments as the three-argument version of doPostsAndWaits().

Definition at line 1876 of file Tpetra_Distributor.hpp.

template<class Packet >
void Tpetra::Distributor::doReversePostsAndWaits ( const Teuchos::ArrayView< const Packet > &  exports,
const Teuchos::ArrayView< const size_t > &  numExportPacketsPerLID,
const Teuchos::ArrayView< Packet > &  imports,
const Teuchos::ArrayView< const size_t > &  numImportPacketsPerLID 
)

Execute the reverse communication plan.

This method takes the same arguments as the four-argument version of doPostsAndWaits().

Definition at line 1912 of file Tpetra_Distributor.hpp.

template<class Packet >
void Tpetra::Distributor::doReversePosts ( const Teuchos::ArrayRCP< const Packet > &  exports,
size_t  numPackets,
const Teuchos::ArrayRCP< Packet > &  imports 
)

Post the data for a reverse plan, but do not execute the waits yet.

This method takes the same arguments as the three-argument version of doPosts().

Definition at line 1953 of file Tpetra_Distributor.hpp.

template<class Packet >
void Tpetra::Distributor::doReversePosts ( const Teuchos::ArrayRCP< const Packet > &  exports,
const Teuchos::ArrayView< const size_t > &  numExportPacketsPerLID,
const Teuchos::ArrayRCP< Packet > &  imports,
const Teuchos::ArrayView< const size_t > &  numImportPacketsPerLID 
)

Post the data for a reverse plan, but do not execute the waits yet.

This method takes the same arguments as the four-argument version of doPosts().

Definition at line 1970 of file Tpetra_Distributor.hpp.

void Tpetra::Distributor::doReverseWaits ( )

Wait on any outstanding nonblocking message requests to complete.

This method is for reverse mode communication only, that is, after calling doReversePosts(). For forward mode communication (after calling doPosts()), call doWaits() instead.

Definition at line 585 of file Tpetra_Distributor.cpp.

template<class ExpView , class ImpView >
std::enable_if<(Kokkos::Impl::is_view< ExpView >::value &&Kokkos::Impl::is_view< ImpView >::value)>::type Tpetra::Distributor::doPostsAndWaits ( const ExpView &  exports,
size_t  numPackets,
const ImpView &  imports 
)

Execute the (forward) communication plan.

Call this version of the method when you have the same number of Packets for each LID (local ID) to send or receive.

Template Parameters
PacketThe type of data to send and receive.
Parameters
exports[in] Contains the values to be sent by this process. On exit from this method, it's OK to modify the entries of this buffer.
numPackets[in] The number of Packets per export / import. This version of the routine assumes that each LID has the same number of Packets associated with it. (MultiVector is an example of a DistObject subclass satisfying this property.)
imports[out] On entry, buffer must be large enough to accomodate the data exported (sent) to us. On exit, contains the values exported to us.

Definition at line 1990 of file Tpetra_Distributor.hpp.

template<class ExpView , class ImpView >
std::enable_if<(Kokkos::Impl::is_view< ExpView >::value &&Kokkos::Impl::is_view< ImpView >::value)>::type Tpetra::Distributor::doPostsAndWaits ( const ExpView &  exports,
const Teuchos::ArrayView< const size_t > &  numExportPacketsPerLID,
const ImpView &  imports,
const Teuchos::ArrayView< const size_t > &  numImportPacketsPerLID 
)

Execute the (forward) communication plan.

Call this version of the method when you have possibly different numbers of Packets for each LID (local ID) to send or receive.

Template Parameters
PacketThe type of data to send and receive.
Parameters
exports[in] Contains the values to be sent by this process. On exit from this method, it's OK to modify the entries of this buffer.
numExportPacketsPerLID[in] The number of packets for each export LID (i.e., each LID to be sent).
imports[out] On entry, buffer must be large enough to accomodate the data exported (sent) to us. On exit, contains the values exported to us.
numImportPacketsPerLID[in] The number of packets for each import LID (i.e., each LID to be received).

Definition at line 2038 of file Tpetra_Distributor.hpp.

template<class ExpView , class ImpView >
std::enable_if<(Kokkos::Impl::is_view< ExpView >::value &&Kokkos::Impl::is_view< ImpView >::value)>::type Tpetra::Distributor::doPosts ( const ExpView &  exports,
size_t  numPackets,
const ImpView &  imports 
)

Post the data for a forward plan, but do not execute the waits yet.

Call this overload when you have the same number of Packets for each LID to send or receive.

Template Parameters
PacketThe type of data to send and receive.
Parameters
exports[in] Contains the values to be sent by this process. This is an ArrayRCP and not an ArrayView so that we have the freedom to use nonblocking sends if we wish. Do not modify the data in this array until doWaits() has completed.
numPackets[in] The number of Packets per export / import. (Same as the three-argument version of doPostsAndWaits().)
imports[out] On entry, buffer must be large enough to accomodate the data exported (sent) to us. This is an ArrayRCP and not an ArrayView so that we have the freedom to use nonblocking sends if we wish. Do not modify the data in this array until doWaits() has completed. Upon completion of doWaits(), this buffer contains the values exported to us.

Definition at line 2057 of file Tpetra_Distributor.hpp.

template<class ExpView , class ImpView >
std::enable_if<(Kokkos::Impl::is_view< ExpView >::value &&Kokkos::Impl::is_view< ImpView >::value)>::type Tpetra::Distributor::doPosts ( const ExpView &  exports,
const Teuchos::ArrayView< const size_t > &  numExportPacketsPerLID,
const ImpView &  imports,
const Teuchos::ArrayView< const size_t > &  numImportPacketsPerLID 
)

Post the data for a forward plan, but do not execute the waits yet.

Call this overload when you have possibly different numbers of Packets for each LID to send or receive.

Template Parameters
PacketThe type of data to send and receive.
Parameters
exports[in] Same as in the three-argument version of doPosts().
numExportPacketsPerLID[in] Same as in the four-argument version of doPostsAndWaits().
imports[out] Same as in the three-argument version of doPosts().
numImportPacketsPerLID[in] Same as in the four-argument version of doPostsAndWaits().

Definition at line 2503 of file Tpetra_Distributor.hpp.

template<class ExpView , class ImpView >
std::enable_if<(Kokkos::Impl::is_view< ExpView >::value &&Kokkos::Impl::is_view< ImpView >::value)>::type Tpetra::Distributor::doReversePostsAndWaits ( const ExpView &  exports,
size_t  numPackets,
const ImpView &  imports 
)

Execute the reverse communication plan.

This method takes the same arguments as the three-argument version of doPostsAndWaits().

Definition at line 2877 of file Tpetra_Distributor.hpp.

template<class ExpView , class ImpView >
std::enable_if<(Kokkos::Impl::is_view< ExpView >::value &&Kokkos::Impl::is_view< ImpView >::value)>::type Tpetra::Distributor::doReversePostsAndWaits ( const ExpView &  exports,
const Teuchos::ArrayView< const size_t > &  numExportPacketsPerLID,
const ImpView &  imports,
const Teuchos::ArrayView< const size_t > &  numImportPacketsPerLID 
)

Execute the reverse communication plan.

This method takes the same arguments as the four-argument version of doPostsAndWaits().

Definition at line 2888 of file Tpetra_Distributor.hpp.

template<class ExpView , class ImpView >
std::enable_if<(Kokkos::Impl::is_view< ExpView >::value &&Kokkos::Impl::is_view< ImpView >::value)>::type Tpetra::Distributor::doReversePosts ( const ExpView &  exports,
size_t  numPackets,
const ImpView &  imports 
)

Post the data for a reverse plan, but do not execute the waits yet.

This method takes the same arguments as the three-argument version of doPosts().

Definition at line 2906 of file Tpetra_Distributor.hpp.

template<class ExpView , class ImpView >
std::enable_if<(Kokkos::Impl::is_view< ExpView >::value &&Kokkos::Impl::is_view< ImpView >::value)>::type Tpetra::Distributor::doReversePosts ( const ExpView &  exports,
const Teuchos::ArrayView< const size_t > &  numExportPacketsPerLID,
const ImpView &  imports,
const Teuchos::ArrayView< const size_t > &  numImportPacketsPerLID 
)

Post the data for a reverse plan, but do not execute the waits yet.

This method takes the same arguments as the four-argument version of doPosts().

Definition at line 2924 of file Tpetra_Distributor.hpp.

void Tpetra::Distributor::getLastDoStatistics ( size_t &  bytes_sent,
size_t &  bytes_recvd 
) const
inline

Information on the last call to do/doReverse.

Returns the amount of data sent & recv'd on this processor in bytes

Definition at line 768 of file Tpetra_Distributor.hpp.

std::string Tpetra::Distributor::description ( ) const

Return a one-line description of this object.

Definition at line 592 of file Tpetra_Distributor.cpp.

void Tpetra::Distributor::describe ( Teuchos::FancyOStream &  out,
const Teuchos::EVerbosityLevel  verbLevel = Teuchos::Describable::verbLevel_default 
) const

Describe this object in a human-readable way to the given output stream.

You must call this method as a collective over all processes in this object's communicator.

Parameters
out[out] Output stream to which to write. Only Process 0 in this object's communicator may write to the output stream.
verbLevel[in] Verbosity level. This also controls whether this method does any communication. At verbosity levels higher (greater) than Teuchos::VERB_LOW, this method behaves as a collective over the object's communicator.

Teuchos::FancyOStream wraps std::ostream. It adds features like tab levels. If you just want to wrap std::cout, try this:

auto out = Teuchos::getFancyOStream (Teuchos::rcpFromRef (std::out));

Definition at line 662 of file Tpetra_Distributor.cpp.


The documentation for this class was generated from the following files: