Epetra Lesson 03: Power method

Use Epetra sparse matrix and dense vector objects to implement a simple iteration (the power method)

Lesson topics

This lesson demonstrates the following:

1. How to construct a Epetra_CrsMatrix (a distributed sparse matrix)
2. How to modify the entries of a previously constructed CrsMatrix
3. How to use CrsMatrix and Vector to implement a simple iterative eigensolver (the power method)

Relation to other lessons

Before starting this lesson, it helps to have finished Epetra Lesson 01: Initialization (how to initialize Epetra) and Epetra_Vector (how to create distributions and distributed vectors). After completing this lesson, you might want to learn about more efficient ways to add or modify entries in a Epetra sparse matrix.

Creating and filling a sparse matrix

This is the first lesson in the usual sequence which covers adding entries to ("filling") a Epetra sparse matrix, and modifying the values of those entries after creating the matrix. Creating and filling a Epetra sparse matrix involves the following steps:

1. Create the CrsMatrix (by calling one of its constructors)
2. Call methods to add entries to the sparse matrix
3. Call the matrix's FillComplete() method

We will explain each of these steps in turn.

Creating the CrsMatrix

Epetra's sparse matrices are distributed over one or more parallel processes, just like vectors or other distributed objects. Also just like vectors, you have to tell the sparse matrix its distribution on construction. Unlike vectors, though, sparse matrices have two dimensions over which to be distributed: rows and columns.

Many users are perfectly happy ignoring the column distribution and just distributing the matrix in "one-dimensional" fashion over rows. In that case, you need only supply the "row Map" to the constructor. This implies that for any row which a process owns, that process may insert entries in any column in that row.

Some users want to use the full flexibility of distributing both the rows and columns of the matrix over processes. This "two-dimensional" distribution, if chosen optimally, can significantly reduce the amount of communication needed for distributed-memory parallel sparse matrix-vector multiply. Trilinos packages like Zoltan can help you compute this distribution. In that case, you may give both the "row Map" and the "column Map" to the constructor. This implies that for any row which a process owns, that process may insert entries in any column in that row which that process owns in its column Map.

Finally, other users already know the structure of the sparse matrix, and just want to fill in values. These users should first create the graph (as an Epetra_CrsGraph), call FillComplete() on the graph, and then give the graph to the constructor of CrsMatrix. The graph may have either a "1-D" or "2-D" distribution, as mentioned above.

Adding entries to the sparse matrix

Methods of CrsMatrix that start with "Insert" actually change the structure of the sparse matrix. Methods that start with "Replace" or "SumInto" only modify existing values.

Calling \c FillComplete()

Calling FillComplete() signals that you are done changing the structure (if allowed) or values of the sparse matrix. This is an expensive operation, because it both rearranges local data, and communicates in order to build reusable communication patterns for sparse matrix-vector multiply. You should try to amortize the cost of this operation whenever possible over many sparse matrix-vector multiplies.

FillComplete() takes two arguments:

• the domain Map (the distribution of the input vector x in a sparse matrix-vector multiply y = A*x)
• the range Map (the distribution of the output vector y in a sparse matrix-vector multiply y = A*x)

Both the domain and range Maps must be one-to-one: that is, each global index in the Map must be uniquely owned by one and only one process. You will need to supply these two arguments to FillComplete() under any of the following conditions:

• When the row Map is not the same as the range Map (it can't be if the row Map is not one to one)
• When the domain and range Maps are not equal (e.g., if the matrix is not square)
• When the domain or range Map as not the same as the row Map

If the domain and range Maps equal the row Map and the row Map is one-to-one, then you may call FillComplete() with no arguments.

Once you have called FillComplete(), you may modify the values in the sparse matrix, but you may not modify its graph structure.

Code example

The following code example shows how to fill and compute with a Epetra sparse matrix, using the procedure discussed in the text above.