|
Tpetra parallel linear algebra
Version of the Day
|
All Classes Namespaces Files Functions Variables Typedefs Enumerations Enumerator Friends Macros Pages
|
Tpetra parallel linear algebra
Version of the Day
|
Base class for distributed Tpetra objects that support data redistribution. More...
#include <Tpetra_DistObject_decl.hpp>
Public Types | |
| typedef Kokkos::Device < typename device_type::execution_space, buffer_memory_space > | buffer_device_type |
| Kokkos::Device specialization for communication buffers. More... | |
Public Member Functions | |
Constructors and destructor | |
| DistObject (const Teuchos::RCP< const map_type > &map) | |
| Constructor. More... | |
| DistObject (const DistObject< Packet, LocalOrdinal, GlobalOrdinal, Node > &rhs) | |
| Copy constructor. More... | |
| virtual | ~DistObject () |
| Destructor (virtual for memory safety of derived classes). More... | |
Public methods for redistributing data | |
| void | doImport (const SrcDistObject &source, const Import< LocalOrdinal, GlobalOrdinal, Node > &importer, CombineMode CM) |
| Import data into this object using an Import object ("forward mode"). More... | |
| void | doExport (const SrcDistObject &source, const Export< LocalOrdinal, GlobalOrdinal, Node > &exporter, CombineMode CM) |
| Export data into this object using an Export object ("forward mode"). More... | |
| void | doImport (const SrcDistObject &source, const Export< LocalOrdinal, GlobalOrdinal, Node > &exporter, CombineMode CM) |
| Import data into this object using an Export object ("reverse mode"). More... | |
| void | doExport (const SrcDistObject &source, const Import< LocalOrdinal, GlobalOrdinal, Node > &importer, CombineMode CM) |
| Export data into this object using an Import object ("reverse mode"). More... | |
Attribute accessor methods | |
| bool | isDistributed () const |
| Whether this is a globally distributed object. More... | |
| virtual Teuchos::RCP< const map_type > | getMap () const |
| The Map describing the parallel distribution of this object. More... | |
I/O methods | |
| void | print (std::ostream &os) const |
| Print this object to the given output stream. More... | |
Implementation of Teuchos::Describable | |
| virtual std::string | description () const |
| One-line descriptiion of this object. More... | |
| virtual void | describe (Teuchos::FancyOStream &out, const Teuchos::EVerbosityLevel verbLevel=Teuchos::Describable::verbLevel_default) const |
| Print a descriptiion of this object to the given output stream. More... | |
Methods for use only by experts | |
| virtual void | removeEmptyProcessesInPlace (const Teuchos::RCP< const map_type > &newMap) |
| Remove processes which contain no entries in this object's Map. More... | |
Protected Types | |
| enum | ReverseOption |
| Whether the data transfer should be performed in forward or reverse mode. More... | |
| typedef device_type::memory_space | buffer_memory_space |
| Kokkos memory space for communication buffers. More... | |
Protected Member Functions | |
| virtual size_t | constantNumberOfPackets () const |
| Whether the implementation's instance promises always to have a constant number of packets per LID (local index), and if so, how many packets per LID there are. More... | |
| virtual void | doTransfer (const SrcDistObject &src, const ::Tpetra::Details::Transfer< local_ordinal_type, global_ordinal_type, node_type > &transfer, const char modeString[], const ReverseOption revOp, const CombineMode CM) |
| Redistribute data across memory images. More... | |
| virtual bool | reallocArraysForNumPacketsPerLid (const size_t numExportLIDs, const size_t numImportLIDs) |
| Reallocate numExportPacketsPerLID_ and/or numImportPacketsPerLID_, if necessary. More... | |
| virtual void | createViews () const |
| Hook for creating a const view. More... | |
| virtual void | createViewsNonConst (KokkosClassic::ReadWriteOption rwo) |
| Hook for creating a nonconst view. More... | |
| virtual void | releaseViews () const |
| Hook for releasing views. More... | |
| bool | reallocImportsIfNeeded (const size_t newSize, const bool debug=false) |
| Reallocate imports_ if needed. More... | |
Methods implemented by subclasses and used by doTransfer(). | |
The doTransfer() method uses the subclass' implementations of these methods to implement data transfer. Subclasses of DistObject must implement these methods. This is an instance of the Template Method Pattern. ("Template" here doesn't mean "C++ template"; it means "pattern with holes that are filled in by the subclass' method implementations.") | |
| virtual bool | checkSizes (const SrcDistObject &source)=0 |
| Compare the source and target (this) objects for compatibility. More... | |
| virtual bool | useNewInterface () |
| Whether the subclass implements the "old" or "new" (Kokkos-friendly) interface. More... | |
| virtual void | copyAndPermute (const SrcDistObject &source, size_t numSameIDs, const Teuchos::ArrayView< const local_ordinal_type > &permuteToLIDs, const Teuchos::ArrayView< const local_ordinal_type > &permuteFromLIDs) |
| Perform copies and permutations that are local to this process. More... | |
| virtual void | copyAndPermuteNew (const SrcDistObject &source, const size_t numSameIDs, const Kokkos::DualView< const local_ordinal_type *, device_type > &permuteToLIDs, const Kokkos::DualView< const local_ordinal_type *, device_type > &permuteFromLIDs) |
| virtual void | packAndPrepare (const SrcDistObject &source, const Teuchos::ArrayView< const local_ordinal_type > &exportLIDs, Teuchos::Array< packet_type > &exports, const Teuchos::ArrayView< size_t > &numPacketsPerLID, size_t &constantNumPackets, Distributor &distor) |
| Perform any packing or preparation required for communication. More... | |
| virtual void | packAndPrepareNew (const SrcDistObject &source, const Kokkos::DualView< const local_ordinal_type *, device_type > &exportLIDs, Kokkos::DualView< packet_type *, buffer_device_type > &exports, const Kokkos::DualView< size_t *, buffer_device_type > &numPacketsPerLID, size_t &constantNumPackets, Distributor &distor) |
| virtual void | unpackAndCombine (const Teuchos::ArrayView< const local_ordinal_type > &importLIDs, const Teuchos::ArrayView< const packet_type > &imports, const Teuchos::ArrayView< size_t > &numPacketsPerLID, size_t constantNumPackets, Distributor &distor, CombineMode CM) |
| Perform any unpacking and combining after communication (old version that uses Teuchos memory management classes to hold data). More... | |
| virtual void | unpackAndCombineNew (const Kokkos::DualView< const local_ordinal_type *, device_type > &importLIDs, const Kokkos::DualView< const packet_type *, buffer_device_type > &imports, const Kokkos::DualView< const size_t *, buffer_device_type > &numPacketsPerLID, const size_t constantNumPackets, Distributor &distor, const CombineMode CM) |
| Perform any unpacking and combining after communication (new version that uses Kokkos data structures to hold data). More... | |
Protected Attributes | |
| Teuchos::RCP< const map_type > | map_ |
| The Map over which this object is distributed. More... | |
| Kokkos::DualView< packet_type *, buffer_device_type > | imports_ |
| Buffer into which packed data are imported (received from other processes). More... | |
| Kokkos::DualView< size_t *, buffer_device_type > | numImportPacketsPerLID_ |
| Number of packets to receive for each receive operation. More... | |
| Kokkos::DualView< packet_type *, buffer_device_type > | exports_ |
| Buffer from which packed data are exported (sent to other processes). More... | |
| Kokkos::DualView< size_t *, buffer_device_type > | numExportPacketsPerLID_ |
| Number of packets to send for each send operation. More... | |
Typedefs | |
| typedef ::Kokkos::Details::ArithTraits < Packet >::val_type | packet_type |
| The type of each datum being sent or received in an Import or Export. More... | |
| typedef LocalOrdinal | local_ordinal_type |
| The type of local indices. More... | |
| typedef GlobalOrdinal | global_ordinal_type |
| The type of global indices. More... | |
| typedef Node | node_type |
| The Kokkos Node type. More... | |
| typedef Node::device_type | device_type |
| The Kokkos Device type. More... | |
| typedef device_type::execution_space | execution_space |
| The Kokkos execution space. More... | |
| typedef Map < local_ordinal_type, global_ordinal_type, node_type > | map_type |
| The type of the Map specialization to use with this class. More... | |
Base class for distributed Tpetra objects that support data redistribution.
DistObject is a base class for all Tpetra distributed global objects, including CrsGraph, CrsMatrix, MultiVector, and Vector. Classes that inherit from DistObject work as either the source or the target of an Export or Import (parallel redistribution) operation. If you want to write a class that can work as the source or target of an Export or Import operation, that class must inherit from DistObject.
| LocalOrdinal | The type of local indices. Same as Map's LocalOrdinal template parameter. See Map's documentation for a discussion of the types that are valid to use here. |
| GlobalOrdinal | The type of global indices. Same as Map's GlobalOrdinal template parameter. See Map's documentation for a discussion of the types that are valid to use here. |
| Node | Same as Map's Node template parameter. See Map's documentation for a discussion of the types that are valid to use here. |
Most Tpetra users will only use this class' methods to perform data redistribution for subclasses such as CrsGraph, CrsMatrix, MultiVector, and Vector. DistObject provides four methods for redistributing data: two versions of doImport(), and two versions of doExport(). Import operations redistribute data from a nonoverlapping (one-to-one) distribution to a possibly overlapping distribution. Export operations redistribute data from a possibly overlapping distribution to a nonoverlapping (one-to-one) distribution. Once you have precomputed a data redistribution plan (an Import or Export object), you may use the plan to redistribute an input object's data into this object, by calling one of these methods. The input object of doImport() or doExport() is always the "source" of the redistribution operation, which sends the data. The *this object is the target, which receives and combines the data. It has the distribution given by this->getMap().
Both Import and Export operations occur in two modes: forward and reverse. Forward mode is the usual case, where you are calling a method with its matching plan type (doImport() for an Import plan, or doExport() for an Export plan). In that case, the input DistObject must have the same Map as the source Map of the plan, and the target DistObject must have the same Map as the target Map of the plan. Reverse mode is also possible, where you call a method with the opposite plan type (doImport() for an Export plan, or doExport() for an Import plan). In that case, the source DistObject's Map must be the same as the target Map of the plan, and the target DistObject's Map must be the same as the source Map of the plan. If you call doImport(), we still call this an Import operation, even if you are using an Export plan in reverse. Similarly, if you call doExport(), we call this an Export operation.
Most users will want to use forward mode. However, reverse mode is useful for some applications. For example, suppose you are solving a nonlinear partial differential equation using the finite element method, with Newton's method for the nonlinear equation. When assembling into a vector, it is convenient and efficient to do local assembly first into a vector with an overlapping distribution, then do global assembly via forward mode Export into a vector with a nonoverlapping distribution. After the linear solve, you may want to bring the resulting nonoverlapping distribution vector back to the overlapping distribution for another update phase. This would be a reverse mode Import, using the precomputed Export object.
Another use case for reverse mode is in CrsMatrix, for the transpose version of distributed sparse matrix-vector multiply ("mat-vec"). Non-transpose mat-vec (a function from the domain Map to the range Map) does an Import to bring in the source vector's data from the domain Map to the column Map of the sparse matrix, and an Export (if necessary) to bring the results from the row Map of the sparse matrix to the range Map. Transpose mat-vec (a function from the range Map to the domain Map) uses these precomputed Import and Export objects in reverse mode: first the Export in reverse mode to Import the source vector's data to the row Map, and then the Import in reverse mode to Export the results to the domain Map. Reverse mode lets us reuse the precomputed data redistribution plans for the transpose case.
If you want to implement your own DistObject subclass, you have two choices of interface to implement: "old" (using Teuchos memory management classes, like Teuchos::ArrayRCP and Teuchos::ArrayView) or "new" (using Kokkos memory management classes, like Kokkos::View and Kokkos::DualView). Prefer new to old. The new interface gives you more options for thread parallelism and use of the GPU.
If you intend to implement the new interface, you must override useNewInterface() to return true. In that case, your class must override the following methods:
Comments in the implementation of doTransferNew() explain how DistObject uses these methods to pack and unpack data for redistribution.
If you choose to implement the old interface (not recommended), you should override the following methods instead:
In this case, you may also wish to implement createViews(), createViewsNonConst(), and releaseViews(). Comments in the implementation of doTransfer() explain how DistObject uses all these methods to pack and unpack data for redistribution.
DistObject implements SrcDistObject, because we presume that if an object can be the target of an Import or Export, it can also be the source of an Import or Export.
Definition at line 349 of file Tpetra_DistObject_decl.hpp.
| typedef ::Kokkos::Details::ArithTraits<Packet>::val_type Tpetra::Classes::DistObject< Packet, LocalOrdinal, GlobalOrdinal, Node >::packet_type |
The type of each datum being sent or received in an Import or Export.
Note that this type does not always correspond to the Scalar template parameter of subclasses.
Definition at line 361 of file Tpetra_DistObject_decl.hpp.
| typedef LocalOrdinal Tpetra::Classes::DistObject< Packet, LocalOrdinal, GlobalOrdinal, Node >::local_ordinal_type |
The type of local indices.
Definition at line 363 of file Tpetra_DistObject_decl.hpp.
| typedef GlobalOrdinal Tpetra::Classes::DistObject< Packet, LocalOrdinal, GlobalOrdinal, Node >::global_ordinal_type |
The type of global indices.
Definition at line 365 of file Tpetra_DistObject_decl.hpp.
| typedef Node Tpetra::Classes::DistObject< Packet, LocalOrdinal, GlobalOrdinal, Node >::node_type |
The Kokkos Node type.
Definition at line 367 of file Tpetra_DistObject_decl.hpp.
| typedef Node::device_type Tpetra::Classes::DistObject< Packet, LocalOrdinal, GlobalOrdinal, Node >::device_type |
The Kokkos Device type.
Definition at line 370 of file Tpetra_DistObject_decl.hpp.
| typedef device_type::execution_space Tpetra::Classes::DistObject< Packet, LocalOrdinal, GlobalOrdinal, Node >::execution_space |
The Kokkos execution space.
Definition at line 372 of file Tpetra_DistObject_decl.hpp.
| typedef Map<local_ordinal_type, global_ordinal_type, node_type> Tpetra::Classes::DistObject< Packet, LocalOrdinal, GlobalOrdinal, Node >::map_type |
The type of the Map specialization to use with this class.
Definition at line 380 of file Tpetra_DistObject_decl.hpp.
|
protected |
Kokkos memory space for communication buffers.
See #1088 for why this is not just device_type::memory_space.
Definition at line 693 of file Tpetra_DistObject_decl.hpp.
| Tpetra::Classes::DistObject< Packet, LocalOrdinal, GlobalOrdinal, Node >::buffer_device_type |
Kokkos::Device specialization for communication buffers.
See #1088 for why this is not just device_type::device_type.
This needs to be public so that I can declare functions like packAndPrepareWithOwningPIDs.
Definition at line 710 of file Tpetra_DistObject_decl.hpp.
|
protected |
Whether the data transfer should be performed in forward or reverse mode.
"Reverse mode" means calling doExport() with an Import object, or calling doImport() with an Export object. "Forward mode" means calling doExport() with an Export object, or calling doImport() with an Import object.
Definition at line 605 of file Tpetra_DistObject_decl.hpp.
|
explicit |
Constructor.
Definition at line 63 of file Tpetra_DistObject_def.hpp.
| Tpetra::Classes::DistObject< Packet, LocalOrdinal, GlobalOrdinal, Node >::DistObject | ( | const DistObject< Packet, LocalOrdinal, GlobalOrdinal, Node > & | rhs | ) |
Copy constructor.
Definition at line 115 of file Tpetra_DistObject_def.hpp.
|
virtual |
Destructor (virtual for memory safety of derived classes).
Definition at line 121 of file Tpetra_DistObject_def.hpp.
| void Tpetra::Classes::DistObject< Packet, LocalOrdinal, GlobalOrdinal, Node >::doImport | ( | const SrcDistObject< Packet, LocalOrdinal, GlobalOrdinal, Node > & | source, |
| const Import< LocalOrdinal, GlobalOrdinal, Node > & | importer, | ||
| CombineMode | CM | ||
| ) |
Import data into this object using an Import object ("forward mode").
The input DistObject is always the source of the data redistribution operation, and the *this object is always the target.
If you don't know the difference between forward and reverse mode, then you probably want forward mode. Use this method with your precomputed Import object if you want to do an Import, else use doExport() with a precomputed Export object.
| source | [in] The "source" object for redistribution. |
| importer | [in] Precomputed data redistribution plan. Its source Map must be the same as the input DistObject's Map, and its target Map must be the same as this->getMap(). |
| CM | [in] How to combine incoming data with the same global index. |
Definition at line 252 of file Tpetra_DistObject_def.hpp.
| void Tpetra::Classes::DistObject< Packet, LocalOrdinal, GlobalOrdinal, Node >::doExport | ( | const SrcDistObject< Packet, LocalOrdinal, GlobalOrdinal, Node > & | source, |
| const Export< LocalOrdinal, GlobalOrdinal, Node > & | exporter, | ||
| CombineMode | CM | ||
| ) |
Export data into this object using an Export object ("forward mode").
The input DistObject is always the source of the data redistribution operation, and the *this object is always the target.
If you don't know the difference between forward and reverse mode, then you probably want forward mode. Use this method with your precomputed Export object if you want to do an Export, else use doImport() with a precomputed Import object.
| source | [in] The "source" object for redistribution. |
| exporter | [in] Precomputed data redistribution plan. Its source Map must be the same as the input DistObject's Map, and its target Map must be the same as this->getMap(). |
| CM | [in] How to combine incoming data with the same global index. |
Definition at line 294 of file Tpetra_DistObject_def.hpp.
| void Tpetra::Classes::DistObject< Packet, LocalOrdinal, GlobalOrdinal, Node >::doImport | ( | const SrcDistObject< Packet, LocalOrdinal, GlobalOrdinal, Node > & | source, |
| const Export< LocalOrdinal, GlobalOrdinal, Node > & | exporter, | ||
| CombineMode | CM | ||
| ) |
Import data into this object using an Export object ("reverse mode").
The input DistObject is always the source of the data redistribution operation, and the *this object is always the target.
If you don't know the difference between forward and reverse mode, then you probably want forward mode. Use the version of doImport() that takes a precomputed Import object in that case.
| source | [in] The "source" object for redistribution. |
| exporter | [in] Precomputed data redistribution plan. Its target Map must be the same as the input DistObject's Map, and its source Map must be the same as this->getMap(). (Note the difference from forward mode.) |
| CM | [in] How to combine incoming data with the same global index. |
Definition at line 336 of file Tpetra_DistObject_def.hpp.
| void Tpetra::Classes::DistObject< Packet, LocalOrdinal, GlobalOrdinal, Node >::doExport | ( | const SrcDistObject< Packet, LocalOrdinal, GlobalOrdinal, Node > & | source, |
| const Import< LocalOrdinal, GlobalOrdinal, Node > & | importer, | ||
| CombineMode | CM | ||
| ) |
Export data into this object using an Import object ("reverse mode").
The input DistObject is always the source of the data redistribution operation, and the *this object is always the target.
If you don't know the difference between forward and reverse mode, then you probably want forward mode. Use the version of doExport() that takes a precomputed Export object in that case.
| source | [in] The "source" object for redistribution. |
| importer | [in] Precomputed data redistribution plan. Its target Map must be the same as the input DistObject's Map, and its source Map must be the same as this->getMap(). (Note the difference from forward mode.) |
| CM | [in] How to combine incoming data with the same global index. |
Definition at line 378 of file Tpetra_DistObject_def.hpp.
| bool Tpetra::Classes::DistObject< Packet, LocalOrdinal, GlobalOrdinal, Node >::isDistributed | ( | ) | const |
Whether this is a globally distributed object.
For a definition of "globally distributed" (and its opposite, "locally replicated"), see the documentation of Map's isDistributed() method.
Definition at line 420 of file Tpetra_DistObject_def.hpp.
|
inlinevirtual |
The Map describing the parallel distribution of this object.
Note that some Tpetra objects might be distributed using multiple Map objects. For example, CrsMatrix has both a row Map and a column Map. It is up to the subclass to decide which Map to use when invoking the DistObject constructor.
Definition at line 510 of file Tpetra_DistObject_decl.hpp.
| void Tpetra::Classes::DistObject< Packet, LocalOrdinal, GlobalOrdinal, Node >::print | ( | std::ostream & | os | ) | const |
Print this object to the given output stream.
We generally assume that all MPI processes can print to the given stream.
Definition at line 1609 of file Tpetra_DistObject_def.hpp.
|
virtual |
One-line descriptiion of this object.
We declare this method virtual so that subclasses of DistObject may override it.
Reimplemented in Tpetra::Classes::CrsMatrix< Scalar, LocalOrdinal, GlobalOrdinal, Node >, Tpetra::Classes::MultiVector< Scalar, LocalOrdinal, GlobalOrdinal, Node >, Tpetra::Classes::MultiVector< Scalar, LO, GO, Node >, Tpetra::Classes::CrsGraph< LocalOrdinal, GlobalOrdinal, Node >, Tpetra::Classes::CrsGraph< LO, GO, node_type >, Tpetra::Details::CooMatrix< SC, LO, GO, NT >, Tpetra::Classes::Vector< Scalar, LocalOrdinal, GlobalOrdinal, Node >, and Tpetra::Experimental::Classes::BlockCrsMatrix< Scalar, LO, GO, Node >.
Definition at line 127 of file Tpetra_DistObject_def.hpp.
|
virtual |
Print a descriptiion of this object to the given output stream.
We declare this method virtual so that subclasses of Distobject may override it.
Reimplemented in Tpetra::Classes::CrsMatrix< Scalar, LocalOrdinal, GlobalOrdinal, Node >, Tpetra::Classes::MultiVector< Scalar, LocalOrdinal, GlobalOrdinal, Node >, Tpetra::Classes::MultiVector< Scalar, LO, GO, Node >, Tpetra::Classes::CrsGraph< LocalOrdinal, GlobalOrdinal, Node >, Tpetra::Classes::CrsGraph< LO, GO, node_type >, Tpetra::Details::CooMatrix< SC, LO, GO, NT >, Tpetra::Classes::Vector< Scalar, LocalOrdinal, GlobalOrdinal, Node >, and Tpetra::Experimental::Classes::BlockCrsMatrix< Scalar, LO, GO, Node >.
Definition at line 147 of file Tpetra_DistObject_def.hpp.
|
virtual |
Remove processes which contain no entries in this object's Map.
On input, this object is distributed over the Map returned by getMap() (the "original Map," with its communicator, the "original communicator"). The input newMap of this method must be the same as the result of calling getMap()->removeEmptyProcesses(). On processes in the original communicator which contain zero entries ("excluded processes," as opposed to "included processes"), the input newMap must be Teuchos::null (which is what getMap()->removeEmptyProcesses() returns anyway).
On included processes, reassign this object's Map (that would be returned by getMap()) to the input newMap, and do any work that needs to be done to restore correct semantics. On excluded processes, free any data that needs freeing, and do any other work that needs to be done to restore correct semantics.
This method has collective semantics over the original communicator. On exit, the only method of this object which is safe to call on excluded processes is the destructor. This implies that subclasses' destructors must not contain communication operations.
Reimplemented in Tpetra::Classes::MultiVector< Scalar, LO, GO, Node >.
Definition at line 214 of file Tpetra_DistObject_def.hpp.
|
protectedvirtual |
Whether the implementation's instance promises always to have a constant number of packets per LID (local index), and if so, how many packets per LID there are.
If this method returns zero, the instance says that it might possibly have a different number of packets for each LID (local index) to send or receive. If it returns nonzero, the instance promises that the number of packets is the same for all LIDs, and that the return value is this number of packets per LID.
The default implementation of this method returns zero. This does not affect the behavior of doTransfer() in any way. If a nondefault implementation returns nonzero, doTransfer() will use this information to avoid unnecessary allocation and / or resizing of arrays.
Reimplemented in Tpetra::Classes::MultiVector< Scalar, LocalOrdinal, GlobalOrdinal, Node >, Tpetra::Classes::MultiVector< Scalar, LO, GO, Node >, and Tpetra::Details::CooMatrix< SC, LO, GO, NT >.
Definition at line 427 of file Tpetra_DistObject_def.hpp.
|
protectedvirtual |
Redistribute data across memory images.
| src | [in] The source object, to redistribute into the target object, which is *this object. |
| transfer | [in] The Export or Import object representing the communication pattern. (Details::Transfer is the common base class of these two objects.) |
| modeString | [in] Human-readable string, for verbose debugging output and error output, explaining what function called this method. Example: "doImport (forward)", "doExport (reverse)". |
| revOp | [in] Whether to do a forward or reverse mode redistribution. |
| CM | [in] The combine mode that describes how to combine values that map to the same global ID on the same process. |
Definition at line 434 of file Tpetra_DistObject_def.hpp.
|
protectedvirtual |
Reallocate numExportPacketsPerLID_ and/or numImportPacketsPerLID_, if necessary.
| numExportLIDs | [in] Number of entries in the exportLIDs input array argument of doTransfer(). |
| numImportLIDs | [in] Number of entries in the remoteLIDs input array argument of doTransfer(). |
Definition at line 617 of file Tpetra_DistObject_def.hpp.
|
protectedpure virtual |
Compare the source and target (this) objects for compatibility.
Implemented in Tpetra::Classes::CrsMatrix< Scalar, LocalOrdinal, GlobalOrdinal, Node >, Tpetra::Classes::MultiVector< Scalar, LocalOrdinal, GlobalOrdinal, Node >, Tpetra::Classes::MultiVector< Scalar, LO, GO, Node >, Tpetra::Classes::CrsGraph< LocalOrdinal, GlobalOrdinal, Node >, Tpetra::Classes::CrsGraph< LO, GO, node_type >, and Tpetra::Experimental::Classes::BlockMultiVector< Scalar, LO, GO, Node >.
|
inlineprotectedvirtual |
Whether the subclass implements the "old" or "new" (Kokkos-friendly) interface.
The "old" interface consists of copyAndPermute, packAndPrepare, and unpackAndCombine. The "new" interface consists of copyAndPermuteNew, packAndPrepareNew, and unpackAndCombineNew. We prefer the new interface, because it facilitates thread parallelization using Kokkos data structures.
At some point, we will remove the old interface, and rename the "new" interface (by removing "New" from the methods' names), so that it becomes the only interface.
Reimplemented in Tpetra::Classes::CrsMatrix< Scalar, LocalOrdinal, GlobalOrdinal, Node >, Tpetra::Classes::MultiVector< Scalar, LocalOrdinal, GlobalOrdinal, Node >, Tpetra::Classes::MultiVector< Scalar, LO, GO, Node >, and Tpetra::Details::CooMatrix< SC, LO, GO, NT >.
Definition at line 761 of file Tpetra_DistObject_decl.hpp.
|
inlineprotectedvirtual |
Perform copies and permutations that are local to this process.
| source | [in] On entry, the source object, from which we are distributing. We distribute to the destination object, which is *this object. |
| numSameIDs | [in] The umber of elements that are the same on the source and destination (this) objects. These elements are owned by the same process in both the source and destination objects. No permutation occurs. |
| numPermuteIDs | [in] The number of elements that are locally permuted between the source and destination objects. |
| permuteToLIDs | [in] List of the elements that are permuted. They are listed by their LID in the destination object. |
| permuteFromLIDs | [in] List of the elements that are permuted. They are listed by their LID in the source object. |
Reimplemented in Tpetra::Classes::CrsGraph< LocalOrdinal, GlobalOrdinal, Node >, and Tpetra::Classes::CrsGraph< LO, GO, node_type >.
Definition at line 781 of file Tpetra_DistObject_decl.hpp.
|
inlineprotectedvirtual |
Perform any packing or preparation required for communication.
| source | [in] Source object for the redistribution. |
| exportLIDs | [in] List of the entries (as local IDs in the source object) we will be sending to other images. |
| exports | [out] On exit, the buffer for data to send. |
| numPacketsPerLID | [out] On exit, the implementation of this method must do one of two things: set numPacketsPerLID[i] to contain the number of packets to be exported for exportLIDs[i] and set constantNumPackets to zero, or set constantNumPackets to a nonzero value. If the latter, the implementation need not fill numPacketsPerLID. |
| constantNumPackets | [out] On exit, 0 if numPacketsPerLID has variable contents (different size for each LID). If nonzero, then it is expected that the number of packets per LID is constant, and that constantNumPackets is that value. |
| distor | [in] The Distributor object we are using. |
Definition at line 816 of file Tpetra_DistObject_decl.hpp.
|
inlineprotectedvirtual |
Perform any unpacking and combining after communication (old version that uses Teuchos memory management classes to hold data).
| importLIDs | [in] List of the entries (as LIDs in the destination object) we received from other images. |
| imports | [in] Buffer containing data we received. |
| numPacketsPerLID | [in] If constantNumPackets is zero, then numPacketsPerLID[i] contains the number of packets imported for importLIDs[i]. |
| constantNumPackets | [in] If nonzero, then numPacketsPerLID is constant (same value in all entries) and constantNumPackets is that value. If zero, then numPacketsPerLID[i] is the number of packets imported for importLIDs[i]. |
| distor | [in] The Distributor object we are using. |
| CM | [in] The combine mode to use when combining the imported entries with existing entries. |
Definition at line 857 of file Tpetra_DistObject_decl.hpp.
|
inlineprotectedvirtual |
Perform any unpacking and combining after communication (new version that uses Kokkos data structures to hold data).
The imports input argument controls whether this method should unpack on host or unpack on device.
| importLIDs | [in] List of the entries (as LIDs in the destination object) we received from other images. |
| imports | [in] Buffer containing data we received. |
| numPacketsPerLID | [in] If constantNumPackets is zero, then numPacketsPerLID[i] contains the number of packets imported for importLIDs[i]. |
| constantNumPackets | [in] If nonzero, then numPacketsPerLID is constant (same value in all entries) and constantNumPackets is that value. If zero, then numPacketsPerLID[i] is the number of packets imported for importLIDs[i]. |
| distor | [in] The Distributor object we are using. |
| CM | [in] The combine mode to use when combining the imported entries with existing entries. |
Definition at line 891 of file Tpetra_DistObject_decl.hpp.
|
protectedvirtual |
Hook for creating a const view.
doTransfer() calls this on the source object. By default, it does nothing, but the source object can use this as a hint to fetch data from a compute buffer on an off-CPU device (such as a GPU) into host memory.
Definition at line 1624 of file Tpetra_DistObject_def.hpp.
|
protectedvirtual |
Hook for creating a nonconst view.
doTransfer() calls this on the destination (*this) object. By default, it does nothing, but the destination object can use this as a hint to fetch data from a compute buffer on an off-CPU device (such as a GPU) into host memory.
| rwo | [in] Whether to create a write-only or a read-and-write view. For Kokkos Node types where compute buffers live in a separate memory space (e.g., in the device memory of a discrete accelerator like a GPU), a write-only view only requires copying from host memory to the compute buffer, whereas a read-and-write view requires copying both ways (once to read, from the compute buffer to host memory, and once to write, back to the compute buffer). |
Definition at line 1630 of file Tpetra_DistObject_def.hpp.
|
protectedvirtual |
Hook for releasing views.
true.doTransfer() calls this on both the source and destination objects, once it no longer needs to access that object's data. By default, this method does nothing. Implementations may use this as a hint to free host memory which is a view of a compute buffer, once the host memory view is no longer needed. Some implementations may prefer to mirror compute buffers in host memory; for these implementations, releaseViews() may do nothing.
Definition at line 1636 of file Tpetra_DistObject_def.hpp.
|
protected |
Reallocate imports_ if needed.
This unfortunately must be declared protected, for the same reason that imports_ is declared protected.
| newSize | [in] New size of imports_. |
| debug | [in] Whether to print (copious) debug output to stderr. |
We don't need a "reallocExportsIfNeeded" method, because exports_ always gets passed into packAndPrepareNew() by nonconst reference. Thus, that method can resize the DualView without needing to call other DistObject methods.
Definition at line 592 of file Tpetra_DistObject_def.hpp.
|
protected |
The Map over which this object is distributed.
Definition at line 942 of file Tpetra_DistObject_decl.hpp.
|
protected |
Buffer into which packed data are imported (received from other processes).
Unfortunately, I had to declare these protected, because CrsMatrix uses them at one point. Please, nobody else use them.
Definition at line 951 of file Tpetra_DistObject_decl.hpp.
|
protected |
Number of packets to receive for each receive operation.
This array is used in Distributor::doPosts() (and doReversePosts()) when starting the ireceive operation.
This may be ignored in doTransfer() if constantNumPackets is nonzero, indicating a constant number of packets per LID. (For example, MultiVector sets the constantNumPackets output argument of packAndPrepare() to the number of columns in the multivector.)
Unfortunately, I had to declare this protected, because CrsMatrix uses it at one point. Please, nobody else use it.
Definition at line 983 of file Tpetra_DistObject_decl.hpp.
|
protected |
Buffer from which packed data are exported (sent to other processes).
Unfortunately, I had to declare this protected, because CrsMatrix uses it at one point. Please, nobody else use it.
Definition at line 990 of file Tpetra_DistObject_decl.hpp.
|
protected |
Number of packets to send for each send operation.
This array is used in Distributor::doPosts() (and doReversePosts()) for preparing for the send operation.
This may be ignored in doTransfer() if constantNumPackets is nonzero, indicating a constant number of packets per LID. (For example, MultiVector sets the constantNumPackets output argument of packAndPrepare() to the number of columns in the multivector.)
Unfortunately, I had to declare this protected, because CrsMatrix uses them at one point. Please, nobody else use it.
Definition at line 1005 of file Tpetra_DistObject_decl.hpp.
1.8.6