MOOCHO (Single Doxygen Collection)  Version of the Day
 All Classes Namespaces Files Functions Variables Typedefs Enumerations Enumerator Friends Macros Groups Pages
Functions
Helper fuctions for applying

Functions

void RTOp_MPI_type_signature (const int num_values, const int num_indexes, const int num_chars, int *num_entries, int block_lengths[], MPI_Aint displacements[], MPI_Datatype datatypes[])
 
int RTOp_extract_reduct_obj_ext_state (const struct RTOp_RTOp *op, RTOp_ReductTarget reduct_obj, int num_values, int num_indexes, int num_chars, void *reduct_obj_ext)
 
int RTOp_load_reduct_obj_ext_state (const struct RTOp_RTOp *op, const void *reduct_obj_ext, RTOp_ReductTarget reduct_obj)
 
int RTOp_MPI_apply_op (MPI_Comm comm, const struct RTOp_RTOp *op, int root_rank, const int num_cols, const int num_vecs, const struct RTOp_SubVector sub_vecs[], const int num_targ_vecs, const struct RTOp_MutableSubVector sub_targ_vecs[], RTOp_ReductTarget reduct_objs[])
 

Detailed Description

reduction/transformation operations in C with MPI.

Function Documentation

void RTOp_MPI_type_signature ( const int  num_values,
const int  num_indexes,
const int  num_chars,
int *  num_entries,
int  block_lengths[],
MPI_Aint  displacements[],
MPI_Datatype  datatypes[] 
)

Initialize MPI compatible type signature arrays for reduction/transformation operator object instance data and reduction target object data.

Parameters
num_values[in] Number of value members
num_indexes[in] Number of index members
num_chars[in] Number of character members
num_entries[out] Number of entries in output arrays set
block_lengths[out] array (length >= RTOp_NUM_DATA_TYPES)
displacements[out] array (length >= RTOp_NUM_DATA_TYPES)
datatypes[out] array (length >= RTOp_NUM_DATA_TYPES)

See the MPI function MPI_Type_struct(...) for a discription of these arrays.

Definition at line 53 of file RTOpToMPI.c.

int RTOp_extract_reduct_obj_ext_state ( const struct RTOp_RTOp op,
RTOp_ReductTarget  reduct_obj,
int  num_values,
int  num_indexes,
int  num_chars,
void reduct_obj_ext 
)

Fill a compacted representation for a reduction object.

Parameters
op[in] RTOp operator object
reduct_obj[in] Reduction object
num_values[in] Number of value members
num_indexes[in] Number of index members
num_chars[in] Number of character members
reduct_obj_ext[out] (size = sizeof(RTOp_value_type)*(3 + num_values)
  • sizeof(RTOp_index_type)*num_indexes + sizeof(RTOp_char_type)*num_chars). On output, this memory will be set according to as described in RTOp_reduct_op_func_ptr_t.
Returns
Returns 0 if successful otherwise returns an error code.

Definition at line 88 of file RTOpToMPI.c.

int RTOp_load_reduct_obj_ext_state ( const struct RTOp_RTOp op,
const void reduct_obj_ext,
RTOp_ReductTarget  reduct_obj 
)

Copy from a compacted representation for a reduction object.

Parameters
op[in] RTOp operator object
reduct_obj_ext[in] (size = sizeof(RTOp_value_type)*(3 + num_values)
  • sizeof(RTOp_index_type)*num_indexes + sizeof(RTOp_char_type)*num_chars) On input, this memory should be set according to as described in #RTOp_reduct_op_func_ptr_t.
reduct_obj [in/out] Reduction object set to the values in reduct_obj_ext (must have been previously constructed)
Returns
Returns 0 if successful otherwise returns an error code.

Definition at line 117 of file RTOpToMPI.c.

int RTOp_MPI_apply_op ( MPI_Comm  comm,
const struct RTOp_RTOp op,
int  root_rank,
const int  num_cols,
const int  num_vecs,
const struct RTOp_SubVector  sub_vecs[],
const int  num_targ_vecs,
const struct RTOp_MutableSubVector  sub_targ_vecs[],
RTOp_ReductTarget  reduct_objs[] 
)

Apply a reduction operation over a set of local sub-vectors using MPI.

Parameters
comm[in] MPI communicator
op[in] reduction/transformation operator object
root_rank[in] See below
num_cols[in] The number of columns for each vector
num_vecs[in] See RTOpPack::RTOp::apply_op()
sub_vecs[in] Array (size num_targ_vecs * num_cols) of nonmutable subvectors. The vectors for each column kc begin at sub_vecs+kc*num_vecs where kc=0...num_cols-1 Can be NULL if there are no local vector elements.
num_targ_vecs[in] See RTOpPack::RTOp::apply_op()
sub_targ_vecs[in] Array (size num_targ_vecs * num_cols) of mutable subvectors. The vectors for each column kc begin at sub_vecs+kc*num_targ_vecs where kc=0...num_cols-1 Can be NULL if there are no local vector elements.
reduct_objs[in/out] Array (size num_cols) See below. If reduct_objs != NULL then on output, reduct_objs[i] will contain the reduction target over all the processes along with the reduction on input if reduct_objs[i] has already been through one or more reductions already and not reinitialized.
Returns
Returns 0 if successful otherwise returns an error code. This function encapsulates a lot of the details of using MPI to perform reduction operations. For this function to work properly, each processor must contain a single sub-vector for each vector argument and every process (in the communicator) must call this function. Each process's sub-vector for each vector argument may be a different size and even dense on some processes and sparse in others. It is also allowed for a process to have empty sub-vectors, but still needs to participate in the collective operation and may want the value of the reduction object returned. The main responsibly of the client is to setup the sub_vecs[] and sub_targ_vecs[] arguments and to deside what type of reduction operation to perform (MPI_Allreduce() or MPI_Reduce()).

Let rank be the value returned from MPI_Comm_rank(comm,&rank) in this process. The expected arguments to this function depend on the argument root_rank and rank as follows:

  • root_rank < 0 : In this case we are performing a MPI_Allreduce(...) reduction operation over all of the processes with the results collected in all of the processes. In this case reduct_obj!=RTOp_REDUCT_OBJ_NULL must be true in all of the processes.
    • root_rank < 0 in all processes
    • reduct_obj != RTOp_REDUCT_OBJ_NULL in all processes if a reduction is being performed.
  • root_rank >= 0 : In this case we are performing a MPI_Reduce(...) reduction operation over all of the processes with the result only collected in the process with rank root_rank. Here all of the processes must pass the same value for root_rank. The reduction target object is only passed in for the processes with rank == root_rank.
    • root_rank same in all of the processes
    • If rank == root_rank then reduct_obj != RTOp_REDUCT_OBJ_NULL
    • If rank != root_rank then reduct_obj == RTOp_REDUCT_OBJ_NULL

Note that if a reduction operation is not performed then no synchronization of the the processes are performed. This is the fastest behavior and should be fine for a single thread but may cause problems with multiple threads.

Definition at line 144 of file RTOpToMPI.c.