Helper fuctions for applying

Functions
void	RTOp_MPI_type_signature (const int num_values, const int num_indexes, const int num_chars, int *num_entries, int block_lengths[], MPI_Aint displacements[], MPI_Datatype datatypes[])
int	RTOp_extract_reduct_obj_ext_state (const struct RTOp_RTOp *op, RTOp_ReductTarget reduct_obj, int num_values, int num_indexes, int num_chars, void *reduct_obj_ext)
int	RTOp_load_reduct_obj_ext_state (const struct RTOp_RTOp *op, const void *reduct_obj_ext, RTOp_ReductTarget reduct_obj)
int	RTOp_MPI_apply_op (MPI_Comm comm, const struct RTOp_RTOp *op, int root_rank, const int num_cols, const int num_vecs, const struct RTOp_SubVector sub_vecs[], const int num_targ_vecs, const struct RTOp_MutableSubVector sub_targ_vecs[], RTOp_ReductTarget reduct_objs[])

Detailed Description

reduction/transformation operations in C with MPI.

Function Documentation

void RTOp_MPI_type_signature	(	const int	num_values,
		const int	num_indexes,
		const int	num_chars,
		int *	num_entries,
		int	block_lengths[],
		MPI_Aint	displacements[],
		MPI_Datatype	datatypes[]
	)

Initialize MPI compatible type signature arrays for reduction/transformation operator object instance data and reduction target object data.

Parameters

num_values	[in] Number of value members
num_indexes	[in] Number of index members
num_chars	[in] Number of character members
num_entries	[out] Number of entries in output arrays set
block_lengths	[out] array (length >= RTOp_NUM_DATA_TYPES)
displacements	[out] array (length >= RTOp_NUM_DATA_TYPES)
datatypes	[out] array (length >= RTOp_NUM_DATA_TYPES)

See the MPI function MPI_Type_struct(...) for a discription of these arrays.

Definition at line 53 of file RTOpToMPI.c.

int RTOp_extract_reduct_obj_ext_state	(	const struct RTOp_RTOp *	op,
		RTOp_ReductTarget	reduct_obj,
		int	num_values,
		int	num_indexes,
		int	num_chars,
		void *	reduct_obj_ext
	)

Fill a compacted representation for a reduction object.

Parameters

op	[in] RTOp operator object
reduct_obj	[in] Reduction object
num_values	[in] Number of value members
num_indexes	[in] Number of index members
num_chars	[in] Number of character members
reduct_obj_ext	[out] (size = sizeof(RTOp_value_type)*(3 + num_values) sizeof(RTOp_index_type)*num_indexes + sizeof(RTOp_char_type)*num_chars). On output, this memory will be set according to as described in RTOp_reduct_op_func_ptr_t.

Returns

Returns 0 if successful otherwise returns an error code.

Definition at line 88 of file RTOpToMPI.c.

int RTOp_load_reduct_obj_ext_state	(	const struct RTOp_RTOp *	op,
		const void *	reduct_obj_ext,
		RTOp_ReductTarget	reduct_obj
	)

Copy from a compacted representation for a reduction object.

Parameters

op	[in] RTOp operator object
reduct_obj_ext	[in] (size = sizeof(RTOp_value_type)*(3 + num_values) sizeof(RTOp_index_type)*num_indexes + sizeof(RTOp_char_type)*num_chars) On input, this memory should be set according to as described in #RTOp_reduct_op_func_ptr_t.
reduct_obj	[in/out] Reduction object set to the values in reduct_obj_ext (must have been previously constructed)

Returns

Returns 0 if successful otherwise returns an error code.

Definition at line 117 of file RTOpToMPI.c.

int RTOp_MPI_apply_op	(	MPI_Comm	comm,
		const struct RTOp_RTOp *	op,
		int	root_rank,
		const int	num_cols,
		const int	num_vecs,
		const struct RTOp_SubVector	sub_vecs[],
		const int	num_targ_vecs,
		const struct RTOp_MutableSubVector	sub_targ_vecs[],
		RTOp_ReductTarget	reduct_objs[]
	)

Apply a reduction operation over a set of local sub-vectors using MPI.

Parameters

comm	[in] MPI communicator
op	[in] reduction/transformation operator object
root_rank	[in] See below
num_cols	[in] The number of columns for each vector
num_vecs	[in] See RTOpPack::RTOp::apply_op()
sub_vecs	[in] Array (size num_targ_vecs * num_cols) of nonmutable subvectors. The vectors for each column kc begin at sub_vecs+kc*num_vecs where kc=0...num_cols-1 Can be NULL if there are no local vector elements.
num_targ_vecs	[in] See RTOpPack::RTOp::apply_op()
sub_targ_vecs	[in] Array (size num_targ_vecs * num_cols) of mutable subvectors. The vectors for each column kc begin at sub_vecs+kc*num_targ_vecs where kc=0...num_cols-1 Can be NULL if there are no local vector elements.
reduct_objs	[in/out] Array (size num_cols) See below. If reduct_objs != NULL then on output, reduct_objs[i] will contain the reduction target over all the processes along with the reduction on input if reduct_objs[i] has already been through one or more reductions already and not reinitialized.

Returns

Returns 0 if successful otherwise returns an error code. This function encapsulates a lot of the details of using MPI to perform reduction operations. For this function to work properly, each processor must contain a single sub-vector for each vector argument and every process (in the communicator) must call this function. Each process's sub-vector for each vector argument may be a different size and even dense on some processes and sparse in others. It is also allowed for a process to have empty sub-vectors, but still needs to participate in the collective operation and may want the value of the reduction object returned. The main responsibly of the client is to setup the sub_vecs[] and sub_targ_vecs[] arguments and to deside what type of reduction operation to perform (MPI_Allreduce() or MPI_Reduce()).

Let rank be the value returned from MPI_Comm_rank(comm,&rank) in this process. The expected arguments to this function depend on the argument root_rank and rank as follows:

• root_rank < 0 : In this case we are performing a MPI_Allreduce(...) reduction operation over all of the processes with the results collected in all of the processes. In this case reduct_obj!=RTOp_REDUCT_OBJ_NULL must be true in all of the processes.
  • root_rank < 0 in all processes
  • reduct_obj != RTOp_REDUCT_OBJ_NULL in all processes if a reduction is being performed.
• root_rank >= 0 : In this case we are performing a MPI_Reduce(...) reduction operation over all of the processes with the result only collected in the process with rank root_rank. Here all of the processes must pass the same value for root_rank. The reduction target object is only passed in for the processes with rank == root_rank.
  • root_rank same in all of the processes
  • If rank == root_rank then reduct_obj != RTOp_REDUCT_OBJ_NULL
  • If rank != root_rank then reduct_obj == RTOp_REDUCT_OBJ_NULL

Note that if a reduction operation is not performed then no synchronization of the the processes are performed. This is the fastest behavior and should be fine for a single thread but may cause problems with multiple threads.

Definition at line 144 of file RTOpToMPI.c.