[Trilinos-Users] corrections to Epetra_FECrsMatrix for GlobalAssemble

Riccardo Rossi rrossi at cimne.upc.edu
Tue Feb 8 07:04:04 MST 2011


Dear list,
     following Alan's suggestion, here it is a correction for the 
Epetra_FECrsMatrix and a little "test"

the combine mode does not allow the AbsMax (apparently it is not
supported by CrsMatrix)

i attach the "corrected" version of the Epetra Files, where i tried to
use the same style, and i also updated the doxygen as well as a little
test file.

on my computer all of the tests run correctly apart of 2, that are
however independent on the Epetra_FEcrs matrix

please let me know if somethign more is needed. also ...when is the next
release due?

thx
Riccardo

-- 
________________________________________________________________
Riccardo Rossi, Ph.D, Civil Engineer
member of the Kratos Group: kratos.cimne.upc.es
Centro Internacional de Métodos Numéricos en Ingeniería (CIMNE)
Universidad Politécnica de Cataluña (UPC)
Edificio C-1, campus Norte UPC
Gran Capitan, s/n
08034 Barcelona, España
Tel. (+34) 93 401 73 99
________________________________________________________________

AVISO IMPORTANTE

Los datos de carácter personal contenidos en el mensaje, se registrarán
en un fichero para facilitar la gestión de las comunicaciones de CIMNE.
Se pueden ejercitar los derechos de acceso, rectificación, cancelación y
oposición por escrito, dirigiéndose a nuestras oficinas de CIMNE, Gran
Capitán s/n,  Edificio C1 - Campus Norte UPC, 08034 Barcelona, España.

AVÍS IMPORTANT

Les dades de caràcter personal contingudes en aquest missatge es
registraran en un fitxer per facilitar la gestió de les comunicacions
del CIMNE. Es poden exercir els drets d'accés, rectificació,
cancel·lació i oposició, per escrit a les nostres oficines del CIMNE,
Gran Capità s/n, Edifici C1, Campus Nord UPC, 08034 Barcelona, Espanya.

IMPORTANT NOTICE

All personal data contained in this mail will be processed
confidentially and stored in a file property of CIMNE in order to manage
corporate communications. You may exercise the right of access,
rectification, deletion and objection by letter sent to CIMNE, Gran
Capitán, Edificio C1 - Campus Norte UPC, 08034 Barcelona, Spain.





-------------- next part --------------

//@HEADER
// ************************************************************************
// 
//               Epetra: Linear Algebra Services Package 
//                 Copyright (2001) Sandia Corporation
// 
// Under terms of Contract DE-AC04-94AL85000, there is a non-exclusive
// license for use of this work by or on behalf of the U.S. Government.
// 
// This library is free software; you can redistribute it and/or modify
// it under the terms of the GNU Lesser General Public License as
// published by the Free Software Foundation; either version 2.1 of the
// License, or (at your option) any later version.
//  
// This library is distributed in the hope that it will be useful, but
// WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
// Lesser General Public License for more details.
//  
// You should have received a copy of the GNU Lesser General Public
// License along with this library; if not, write to the Free Software
// Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307
// USA
// Questions? Contact Michael A. Heroux (maherou at sandia.gov) 
// 
// ************************************************************************
//@HEADER

#include <Epetra_FECrsMatrix.h>
#include <Epetra_IntSerialDenseVector.h>
#include <Epetra_SerialDenseMatrix.h>
#include <Epetra_Export.h>
#include <Epetra_Comm.h>
#include <Epetra_Map.h>
#include <Epetra_Util.h>

//----------------------------------------------------------------------------
Epetra_FECrsMatrix::Epetra_FECrsMatrix(Epetra_DataAccess CV,
				       const Epetra_Map& RowMap,
				       int* NumEntriesPerRow,
				       bool ignoreNonLocalEntries)
  : Epetra_CrsMatrix(CV, RowMap, NumEntriesPerRow),
    myFirstRow_(0),
    myNumRows_(0),
    ignoreNonLocalEntries_(ignoreNonLocalEntries),
    numNonlocalRows_(0),
    nonlocalRows_(NULL),
    nonlocalRowLengths_(NULL),
    nonlocalRowAllocLengths_(NULL),
    nonlocalCols_(NULL),
    nonlocalCoefs_(NULL),
    workData_(NULL),
    workDataLength_(0)
{
  myFirstRow_ = RowMap.MinMyGID();
  myNumRows_ = RowMap.NumMyElements();

  workData_ = new double[128];
  workDataLength_ = 128;
}

//----------------------------------------------------------------------------
Epetra_FECrsMatrix::Epetra_FECrsMatrix(Epetra_DataAccess CV,
				       const Epetra_Map& RowMap,
				       int NumEntriesPerRow,
				       bool ignoreNonLocalEntries)
  : Epetra_CrsMatrix(CV, RowMap, NumEntriesPerRow),
    myFirstRow_(0),
    myNumRows_(0),
    ignoreNonLocalEntries_(ignoreNonLocalEntries),
    numNonlocalRows_(0),
    nonlocalRows_(NULL),
    nonlocalRowLengths_(NULL),
    nonlocalRowAllocLengths_(NULL),
    nonlocalCols_(NULL),
    nonlocalCoefs_(NULL),
    workData_(NULL),
    workDataLength_(0)
{
  myFirstRow_ = RowMap.MinMyGID();
  myNumRows_ = RowMap.NumMyElements();

  workData_ = new double[128];
  workDataLength_ = 128;
}

//----------------------------------------------------------------------------
Epetra_FECrsMatrix::Epetra_FECrsMatrix(Epetra_DataAccess CV,
				       const Epetra_Map& RowMap,
				       const Epetra_Map& ColMap,
				       int* NumEntriesPerRow,
				       bool ignoreNonLocalEntries)
  : Epetra_CrsMatrix(CV, RowMap, ColMap, NumEntriesPerRow),
    myFirstRow_(0),
    myNumRows_(0),
    ignoreNonLocalEntries_(ignoreNonLocalEntries),
    numNonlocalRows_(0),
    nonlocalRows_(NULL),
    nonlocalRowLengths_(NULL),
    nonlocalRowAllocLengths_(NULL),
    nonlocalCols_(NULL),
    nonlocalCoefs_(NULL),
    workData_(NULL),
    workDataLength_(0)
{
  myFirstRow_ = RowMap.MinMyGID();
  myNumRows_ = RowMap.NumMyElements();

  workData_ = new double[128];
  workDataLength_ = 128;
}

//----------------------------------------------------------------------------
Epetra_FECrsMatrix::Epetra_FECrsMatrix(Epetra_DataAccess CV,
				       const Epetra_Map& RowMap,
				       const Epetra_Map& ColMap,
				       int NumEntriesPerRow,
				       bool ignoreNonLocalEntries)
  : Epetra_CrsMatrix(CV, RowMap, ColMap, NumEntriesPerRow),
    myFirstRow_(0),
    myNumRows_(0),
    ignoreNonLocalEntries_(ignoreNonLocalEntries),
    numNonlocalRows_(0),
    nonlocalRows_(NULL),
    nonlocalRowLengths_(NULL),
    nonlocalRowAllocLengths_(NULL),
    nonlocalCols_(NULL),
    nonlocalCoefs_(NULL),
    workData_(NULL),
    workDataLength_(0)
{
  myFirstRow_ = RowMap.MinMyGID();
  myNumRows_ = RowMap.NumMyElements();

  workData_ = new double[128];
  workDataLength_ = 128;
}

//----------------------------------------------------------------------------
Epetra_FECrsMatrix::Epetra_FECrsMatrix(Epetra_DataAccess CV,
				       const Epetra_CrsGraph& Graph,
				       bool ignoreNonLocalEntries)
  : Epetra_CrsMatrix(CV, Graph),
    myFirstRow_(0),
    myNumRows_(0),
    ignoreNonLocalEntries_(ignoreNonLocalEntries),
    numNonlocalRows_(0),
    nonlocalRows_(NULL),
    nonlocalRowLengths_(NULL),
    nonlocalRowAllocLengths_(NULL),
    nonlocalCols_(NULL),
    nonlocalCoefs_(NULL),
    workData_(NULL),
    workDataLength_(0)
{
  myFirstRow_ = RowMap().MinMyGID();
  myNumRows_ = RowMap().NumMyElements();

  workData_ = new double[128];
  workDataLength_ = 128;
}
   
//----------------------------------------------------------------------------
Epetra_FECrsMatrix::Epetra_FECrsMatrix(const Epetra_FECrsMatrix& src)
 : Epetra_CrsMatrix(src),
   myFirstRow_(0),
   myNumRows_(0),
   ignoreNonLocalEntries_(false),
   numNonlocalRows_(0),
   nonlocalRows_(NULL),
   nonlocalRowLengths_(NULL),
   nonlocalRowAllocLengths_(NULL),
   nonlocalCols_(NULL),
   nonlocalCoefs_(NULL),
   workData_(NULL),
   workDataLength_(0)
{
  operator=(src);
}

//----------------------------------------------------------------------------
Epetra_FECrsMatrix& Epetra_FECrsMatrix::operator=(const Epetra_FECrsMatrix& src)
{
  if (this == &src) {
    return( *this );
  }

  DeleteMemory();

  Epetra_CrsMatrix::operator=(src);

  myFirstRow_ = src.myFirstRow_;
  myNumRows_ = src.myNumRows_;
  ignoreNonLocalEntries_ = src.ignoreNonLocalEntries_;
  numNonlocalRows_ = src.numNonlocalRows_;

  workDataLength_ = 128;
  workData_ = new double[workDataLength_];

  if (numNonlocalRows_ < 1) {
    return( *this );
  }

  nonlocalRows_ = new int[numNonlocalRows_];
  nonlocalRowLengths_ = new int[numNonlocalRows_];
  nonlocalRowAllocLengths_ = new int[numNonlocalRows_];
  nonlocalCols_ = new int*[numNonlocalRows_];
  nonlocalCoefs_= new double*[numNonlocalRows_];

  for(int i=0; i<numNonlocalRows_; ++i) {
    nonlocalRows_[i] = src.nonlocalRows_[i];
    nonlocalRowLengths_[i] = src.nonlocalRowLengths_[i];
    nonlocalRowAllocLengths_[i] = src.nonlocalRowAllocLengths_[i];

    nonlocalCols_[i] = new int[nonlocalRowAllocLengths_[i]];
    nonlocalCoefs_[i] = new double[nonlocalRowAllocLengths_[i]];

    for(int j=0; j<nonlocalRowLengths_[i]; ++j) {
      nonlocalCols_[i][j] = src.nonlocalCols_[i][j];
      nonlocalCoefs_[i][j] = src.nonlocalCoefs_[i][j];
    }
  }

  return( *this );
}

//----------------------------------------------------------------------------
Epetra_FECrsMatrix::~Epetra_FECrsMatrix()
{
  DeleteMemory();
}

//----------------------------------------------------------------------------
void Epetra_FECrsMatrix::DeleteMemory()
{
  if (numNonlocalRows_ > 0) {
    for(int i=0; i<numNonlocalRows_; ++i) {
      delete [] nonlocalCols_[i];
      delete [] nonlocalCoefs_[i];
    }
    delete [] nonlocalCols_;
    delete [] nonlocalCoefs_;
    delete [] nonlocalRows_;
    delete [] nonlocalRowLengths_;
    delete [] nonlocalRowAllocLengths_;
    numNonlocalRows_ = 0;
  }

  delete [] workData_;
  workDataLength_ = 0;
}

//----------------------------------------------------------------------------
int Epetra_FECrsMatrix::SumIntoGlobalValues(int numIndices, const int* indices,
					    const double* const* values,
					    int format)
{
  return(InputGlobalValues(numIndices, indices,
                           numIndices, indices,
                           values, format, SUMINTO));
}

//----------------------------------------------------------------------------
int Epetra_FECrsMatrix::SumIntoGlobalValues(int numRows, const int* rows,
					    int numCols, const int* cols,
					    const double* const* values,
					    int format)
{
  return(InputGlobalValues(numRows, rows,
                           numCols, cols,
                           values, format, SUMINTO));
}

//----------------------------------------------------------------------------
int Epetra_FECrsMatrix::SumIntoGlobalValues(int numIndices, const int* indices,
					    const double* values,
					    int format)
{
  return(InputGlobalValues(numIndices, indices,
                           numIndices, indices,
                           values, format, SUMINTO));
}

//----------------------------------------------------------------------------
int Epetra_FECrsMatrix::SumIntoGlobalValues(int numRows, const int* rows,
					    int numCols, const int* cols,
					    const double* values,
					    int format)
{
  return(InputGlobalValues(numRows, rows,
                           numCols, cols,
                           values, format, SUMINTO));
}

//----------------------------------------------------------------------------
int Epetra_FECrsMatrix::SumIntoGlobalValues(const Epetra_IntSerialDenseVector& indices,
					    const Epetra_SerialDenseMatrix& values,
					    int format)
{
  if (indices.Length() != values.M() || indices.Length() != values.N()) {
    return(-1);
  }

  return( SumIntoGlobalValues(indices.Length(), indices.Values(),
			      values.A(), format) );
}

//----------------------------------------------------------------------------
int Epetra_FECrsMatrix::InsertGlobalValues(const Epetra_IntSerialDenseVector& indices,
					    const Epetra_SerialDenseMatrix& values,
					    int format)
{
  if (indices.Length() != values.M() || indices.Length() != values.N()) {
    return(-1);
  }

  return( InsertGlobalValues(indices.Length(), indices.Values(),
			      values.A(), format) );
}

//----------------------------------------------------------------------------
int Epetra_FECrsMatrix::ReplaceGlobalValues(const Epetra_IntSerialDenseVector& indices,
					    const Epetra_SerialDenseMatrix& values,
					    int format)
{
  if (indices.Length() != values.M() || indices.Length() != values.N()) {
    return(-1);
  }

  return( ReplaceGlobalValues(indices.Length(), indices.Values(),
			      values.A(), format) );
}

//----------------------------------------------------------------------------
int Epetra_FECrsMatrix::SumIntoGlobalValues(const Epetra_IntSerialDenseVector& rows,
					    const Epetra_IntSerialDenseVector& cols,
					    const Epetra_SerialDenseMatrix& values,
					    int format)
{
  if (rows.Length() != values.M() || cols.Length() != values.N()) {
    return(-1);
  }

  return( SumIntoGlobalValues(rows.Length(), rows.Values(),
			      cols.Length(), cols.Values(),
			      values.A(), format) );
}

//----------------------------------------------------------------------------
int Epetra_FECrsMatrix::InsertGlobalValues(const Epetra_IntSerialDenseVector& rows,
					   const Epetra_IntSerialDenseVector& cols,
					   const Epetra_SerialDenseMatrix& values,
					   int format)
{
  if (rows.Length() != values.M() || cols.Length() != values.N()) {
    return(-1);
  }

  return( InsertGlobalValues(rows.Length(), rows.Values(),
			     cols.Length(), cols.Values(),
			      values.A(), format) );
}

//----------------------------------------------------------------------------
int Epetra_FECrsMatrix::ReplaceGlobalValues(const Epetra_IntSerialDenseVector& rows,
					    const Epetra_IntSerialDenseVector& cols,
					    const Epetra_SerialDenseMatrix& values,
					    int format)
{
  if (rows.Length() != values.M() || cols.Length() != values.N()) {
    return(-1);
  }

  return( ReplaceGlobalValues(rows.Length(), rows.Values(),
			      cols.Length(), cols.Values(),
			      values.A(), format) );
}

//----------------------------------------------------------------------------
int Epetra_FECrsMatrix::InsertGlobalValues(int numIndices, const int* indices,
					    const double* const* values,
					    int format)
{
  return(InputGlobalValues(numIndices, indices,
                           numIndices, indices,
                           values, format, INSERT));
}

//----------------------------------------------------------------------------
int Epetra_FECrsMatrix::InsertGlobalValues(int numRows, const int* rows,
					    int numCols, const int* cols,
					    const double* const* values,
					    int format)
{
  return(InputGlobalValues(numRows, rows,
                           numCols, cols,
                           values, format, INSERT));
}

//----------------------------------------------------------------------------
int Epetra_FECrsMatrix::InsertGlobalValues(int numIndices, const int* indices,
					    const double* values,
					    int format)
{
  return(InputGlobalValues(numIndices, indices,
                           numIndices, indices,
                           values, format, INSERT));
}

//----------------------------------------------------------------------------
int Epetra_FECrsMatrix::InsertGlobalValues(int numRows, const int* rows,
					    int numCols, const int* cols,
					    const double* values,
					    int format)
{
  return(InputGlobalValues(numRows, rows,
                           numCols, cols,
                           values, format, INSERT));
}

//----------------------------------------------------------------------------
int Epetra_FECrsMatrix::SumIntoGlobalValues(int GlobalRow, int NumEntries,
                                            double* Values, int* Indices)
{
  return(InputGlobalValues(1, &GlobalRow,
                           NumEntries, Indices, Values,
                           Epetra_FECrsMatrix::ROW_MAJOR, SUMINTO));
}

//----------------------------------------------------------------------------
int Epetra_FECrsMatrix::InsertGlobalValues(int GlobalRow, int NumEntries,
                                            double* Values, int* Indices)
{
  return(InputGlobalValues(1, &GlobalRow,
                           NumEntries, Indices, Values,
                           Epetra_FECrsMatrix::ROW_MAJOR, INSERT));
}

//----------------------------------------------------------------------------
int Epetra_FECrsMatrix::ReplaceGlobalValues(int GlobalRow, int NumEntries,
                                            double* Values, int* Indices)
{
  return(InputGlobalValues(1, &GlobalRow,
                           NumEntries, Indices, Values,
                           Epetra_FECrsMatrix::ROW_MAJOR, REPLACE));
}

//----------------------------------------------------------------------------
int Epetra_FECrsMatrix::ReplaceGlobalValues(int numIndices, const int* indices,
					    const double* const* values,
					    int format)
{
  return(InputGlobalValues(numIndices, indices,
                           numIndices, indices,
                           values, format, REPLACE));
}

//----------------------------------------------------------------------------
int Epetra_FECrsMatrix::ReplaceGlobalValues(int numRows, const int* rows,
					    int numCols, const int* cols,
					    const double* const* values,
					    int format)
{
  return(InputGlobalValues(numRows, rows,
                           numCols, cols,
                           values, format, REPLACE));
}

//----------------------------------------------------------------------------
int Epetra_FECrsMatrix::ReplaceGlobalValues(int numIndices, const int* indices,
					    const double* values,
					    int format)
{
  return(InputGlobalValues(numIndices, indices,
                           numIndices, indices,
                           values, format, REPLACE));
}

//----------------------------------------------------------------------------
int Epetra_FECrsMatrix::ReplaceGlobalValues(int numRows, const int* rows,
					    int numCols, const int* cols,
					    const double* values,
					    int format)
{
  return(InputGlobalValues(numRows, rows,
                           numCols, cols,
                           values, format, REPLACE));
}

//----------------------------------------------------------------------------
int Epetra_FECrsMatrix::GlobalAssemble(bool callFillComplete,
				       Epetra_CombineMode CombineMode)
{
  if (Map().Comm().NumProc() < 2 || ignoreNonLocalEntries_) {
    if (callFillComplete) {
      EPETRA_CHK_ERR( FillComplete(DomainMap(), RangeMap()) );
    }
    return(0);
  }

  //In this method we need to gather all the non-local (overlapping) data
  //that's been input on each processor, into the
  //non-overlapping distribution defined by the map that 'this' matrix was
  //constructed with.

  //First build a map that describes our nonlocal data.
  //We'll use the arbitrary distribution constructor of Map.

  Epetra_Map sourceMap(-1, numNonlocalRows_, nonlocalRows_,
			 Map().IndexBase(), Map().Comm());

  //If sourceMap has global size 0, then no nonlocal data exists and we can
  //skip most of this function.
  if (sourceMap.NumGlobalElements() < 1) {
    if (callFillComplete) {
      EPETRA_CHK_ERR( FillComplete(DomainMap(), RangeMap()) );
    }
    return(0);
  }

  //We also need to build a column-map, containing the columns in our
  //nonlocal data. To do that, create a list of all column-indices that
  //occur in our nonlocal rows.

  int numCols = 0, allocLen = 0;
  int* cols = NULL;
  int insertPoint = -1;

  for(int i=0; i<numNonlocalRows_; ++i) {
    for(int j=0; j<nonlocalRowLengths_[i]; ++j) {
      int col = nonlocalCols_[i][j];
      int offset = Epetra_Util_binary_search(col, cols, numCols, insertPoint);
      if (offset < 0) {
	EPETRA_CHK_ERR( Epetra_Util_insert(col, insertPoint, cols,
					   numCols, allocLen) );
      }
    }
  }

  Epetra_Map colMap(-1, numCols, cols,
		      Map().IndexBase(), Map().Comm());

  delete [] cols;
  numCols = 0;
  allocLen = 0;

  //now we need to create a matrix with sourceMap and colMap, and fill it with
  //our nonlocal data so we can then export it to the correct owning processors.

  Epetra_CrsMatrix tempMat(Copy, sourceMap, colMap, nonlocalRowLengths_);

  //Next we need to make sure the 'indices-are-global' attribute of tempMat's
  //graph is set to true, in case this processor doesn't end up calling the
  //InsertGlobalValues method...

  const Epetra_CrsGraph& graph = tempMat.Graph();
  Epetra_CrsGraph& nonconst_graph = const_cast<Epetra_CrsGraph&>(graph);
  nonconst_graph.SetIndicesAreGlobal(true);

  for(int i=0; i<numNonlocalRows_; ++i) {
    EPETRA_CHK_ERR( tempMat.InsertGlobalValues(nonlocalRows_[i],
					       nonlocalRowLengths_[i],
					       nonlocalCoefs_[i],
					       nonlocalCols_[i]) );
  }

  //Now we need to call FillComplete on our temp matrix. We need to
  //pass a DomainMap and RangeMap, which are not the same as the RowMap
  //and ColMap that we constructed the matrix with.
  EPETRA_CHK_ERR(tempMat.FillComplete(DomainMap(), RangeMap()));

  Epetra_Export exporter(sourceMap, RowMap());

  EPETRA_CHK_ERR(Export(tempMat, exporter, CombineMode));

  if(callFillComplete) {
    EPETRA_CHK_ERR(FillComplete(DomainMap(), RangeMap()));
  }

  //now reset the values in our nonlocal data
  for(int i=0; i<numNonlocalRows_; ++i) {
    for(int j=0; j<nonlocalRowLengths_[i]; ++j) {
      nonlocalCols_[i][j] = 0;
      nonlocalCoefs_[i][j] = 0.0;
    }
    nonlocalRowLengths_[i] = 0;
  }

  return(0);
}

//----------------------------------------------------------------------------
int Epetra_FECrsMatrix::GlobalAssemble(const Epetra_Map& domain_map,
                                       const Epetra_Map& range_map,
                                       bool callFillComplete,
				       Epetra_CombineMode CombineMode)
{
  if (Map().Comm().NumProc() < 2 || ignoreNonLocalEntries_) {
    if (callFillComplete) {
      EPETRA_CHK_ERR( FillComplete(domain_map, range_map) );
    }
    return(0);
  }

  int i, j;

  //In this method we need to gather all the non-local (overlapping) data
  //that's been input on each processor, into the
  //non-overlapping distribution defined by the map that 'this' matrix was
  //constructed with.

  //First build a map that describes our nonlocal data.
  //We'll use the arbitrary distribution constructor of Map.

  Epetra_Map* sourceMap = new Epetra_Map(-1, numNonlocalRows_, nonlocalRows_,
					 Map().IndexBase(), Map().Comm());

  //If sourceMap has global size 0, then no nonlocal data exists and we can
  //skip most of this function.
  if (sourceMap->NumGlobalElements() < 1) {
    if (callFillComplete) {
      EPETRA_CHK_ERR( FillComplete(domain_map, range_map) );
    }
    delete sourceMap;
    return(0);
  }

  //We also need to build a column-map, containing the columns in our
  //nonlocal data. To do that, create a list of all column-indices that
  //occur in our nonlocal rows.

  int numCols = 0, allocLen = 0;
  int* cols = NULL;
  int insertPoint = -1;

  for(i=0; i<numNonlocalRows_; ++i) {
    for(j=0; j<nonlocalRowLengths_[i]; ++j) {
      int col = nonlocalCols_[i][j];
      int offset = Epetra_Util_binary_search(col, cols, numCols, insertPoint);
      if (offset < 0) {
	EPETRA_CHK_ERR( Epetra_Util_insert(col, insertPoint, cols,
					   numCols, allocLen) );
      }
    }
  }

  Epetra_Map* colMap = new Epetra_Map(-1, numCols, cols,
				      Map().IndexBase(), Map().Comm());

  delete [] cols;
  numCols = 0;
  allocLen = 0;

  //now we need to create a matrix with sourceMap and colMap, and fill it with
  //our nonlocal data so we can then export it to the correct owning processors.

  Epetra_CrsMatrix* tempMat = new Epetra_CrsMatrix(Copy, *sourceMap, *colMap,
						   nonlocalRowLengths_);


  //Next we need to make sure the 'indices-are-global' attribute of tempMat's
  //graph is set to true, in case this processor doesn't end up calling the
  //InsertGlobalValues method...

  const Epetra_CrsGraph& graph = tempMat->Graph();
  Epetra_CrsGraph& nonconst_graph = const_cast<Epetra_CrsGraph&>(graph);
  nonconst_graph.SetIndicesAreGlobal(true);

  for(i=0; i<numNonlocalRows_; ++i) {
    EPETRA_CHK_ERR( tempMat->InsertGlobalValues(nonlocalRows_[i],
					       nonlocalRowLengths_[i],
					       nonlocalCoefs_[i],
					       nonlocalCols_[i]) );
  }

  //Now we need to call FillComplete on our temp matrix. We need to
  //pass a DomainMap and RangeMap, which are not the same as the RowMap
  //and ColMap that we constructed the matrix with.
  EPETRA_CHK_ERR(tempMat->FillComplete(domain_map, range_map));

  Epetra_Export* exporter = new Epetra_Export(*sourceMap, RowMap());

  EPETRA_CHK_ERR(Export(*tempMat, *exporter, CombineMode));

  if(callFillComplete) {
    EPETRA_CHK_ERR(FillComplete(domain_map, range_map));
  }

  //now reset the values in our nonlocal data
  for(i=0; i<numNonlocalRows_; ++i) {
    for(j=0; j<nonlocalRowLengths_[i]; ++j) {
      nonlocalCols_[i][j] = 0;
      nonlocalCoefs_[i][j] = 0.0;
    }
    nonlocalRowLengths_[i] = 0;
  }

  delete exporter;
  delete tempMat;
  delete sourceMap;
  delete colMap;

  return(0);
}

//----------------------------------------------------------------------------
int Epetra_FECrsMatrix::InputGlobalValues(int numRows, const int* rows,
					  int numCols, const int* cols,
					  const double*const* values,
					  int format, int mode)
{
  if (format != Epetra_FECrsMatrix::ROW_MAJOR &&
      format != Epetra_FECrsMatrix::COLUMN_MAJOR) {
    cerr << "Epetra_FECrsMatrix: unrecognized format specifier."<<endl;
    return(-1);
  }

  if (format == Epetra_FECrsMatrix::COLUMN_MAJOR) {
    if (numCols > workDataLength_) {
      delete [] workData_;
      workDataLength_ = numCols*2;
      workData_ = new double[workDataLength_];
    }
  }

  int returncode = 0;
  int err = 0;

  for(int i=0; i<numRows; ++i) {
    double* valuesptr = (double*)values[i];

    if (format == Epetra_FECrsMatrix::COLUMN_MAJOR) {
      //if the data is in column-major order, then we need to copy the i-th
      //column of the values table into workData_, and that will be the i-th
      //row. ... Is that clear?

      for(int j=0; j<numCols; ++j) {
	workData_[j] = values[j][i];
      }
      valuesptr = workData_;
    }

    if (Map().MyGID(rows[i])) {
      switch(mode) {
      case Epetra_FECrsMatrix::SUMINTO:
        err = this->Epetra_CrsMatrix::SumIntoGlobalValues(rows[i], numCols,
					   valuesptr, (int*)cols);
	if (err<0) return(err);
	if (err>0) returncode = err;
	break;
      case Epetra_FECrsMatrix::REPLACE:
	err = this->Epetra_CrsMatrix::ReplaceGlobalValues(rows[i], numCols,
					   valuesptr, (int*)cols);
	if (err<0) return(err);
	if (err>0) returncode = err;
	break;
      case Epetra_FECrsMatrix::INSERT:
	err = this->Epetra_CrsMatrix::InsertGlobalValues(rows[i], numCols,
					  valuesptr, (int*)cols);
	if (err<0) return(err);
	if (err>0) returncode = err;
	break;
      default:
	cerr << "Epetra_FECrsMatrix: internal error, bad input mode."<<endl;
	return(-1);
      }
    }
    else {
      err = InputNonlocalGlobalValues(rows[i],
				      numCols, cols,
				      valuesptr, mode);
      if (err<0) return(err);
      if (err>0) returncode = err;
    }
  }

  return(returncode);
}

//----------------------------------------------------------------------------
int Epetra_FECrsMatrix::InputGlobalValues(int numRows, const int* rows,
					  int numCols, const int* cols,
					  const double* values,
					  int format, int mode)
{
  int first_dim = format==COLUMN_MAJOR ? numCols : numRows;
  int second_dim = format==COLUMN_MAJOR ? numRows : numCols;

  const double** values_2d = new const double*[first_dim];
  int offset = 0;
  for(int i=0; i<first_dim; ++i) {
    values_2d[i] = &(values[offset]);
    offset += second_dim;
  }

  int err = InputGlobalValues(numRows, rows, numCols, cols,
			      values_2d, format, mode);
  delete [] values_2d;

  return(err);
}

//----------------------------------------------------------------------------
int Epetra_FECrsMatrix::InputNonlocalGlobalValues(int row,
						  int numCols, const int* cols,
						  const double* values,
						  int mode)
{
  int insertPoint = -1;

  //find offset of this row in our list of nonlocal rows.
  int rowoffset = Epetra_Util_binary_search(row, nonlocalRows_,
                                          numNonlocalRows_, insertPoint);

  if (rowoffset < 0) {
    EPETRA_CHK_ERR( InsertNonlocalRow(row, insertPoint) );
    rowoffset = insertPoint;
  }

  for(int i=0; i<numCols; ++i) {
    EPETRA_CHK_ERR( InputNonlocalValue(rowoffset, cols[i], values[i],
				       mode) );
  }

  return(0);
}

//----------------------------------------------------------------------------
int Epetra_FECrsMatrix::InsertNonlocalRow(int row, int offset)
{
  int alloc_len = numNonlocalRows_;
  EPETRA_CHK_ERR( Epetra_Util_insert(row, offset, nonlocalRows_,
                                     numNonlocalRows_, alloc_len, 1) );

  int tmp1 = numNonlocalRows_-1;
  int tmp2 = alloc_len-1;

  EPETRA_CHK_ERR( Epetra_Util_insert(0, offset, nonlocalRowLengths_,
				     tmp1, tmp2, 1) );

  --tmp1;
  --tmp2;
  int initialAllocLen = 16;
  EPETRA_CHK_ERR( Epetra_Util_insert(initialAllocLen, offset,
				     nonlocalRowAllocLengths_,
				     tmp1, tmp2, 1) );

  int** newCols = new int*[numNonlocalRows_];
  double** newCoefs = new double*[numNonlocalRows_];

  if (newCols == NULL || newCoefs == NULL) {
    return(-1);
  }

  newCols[offset] = new int[initialAllocLen];
  newCoefs[offset] = new double[initialAllocLen];

  int index = 0;
  for(int i=0; i<numNonlocalRows_-1; ++i) {
    if (i == offset) {
      ++index;
    }

    newCols[index] = nonlocalCols_[i];
    newCoefs[index++] = nonlocalCoefs_[i];
  }

  delete [] nonlocalCols_;
  delete [] nonlocalCoefs_;

  nonlocalCols_ = newCols;
  nonlocalCoefs_ = newCoefs;

  return(0);
}

//----------------------------------------------------------------------------
int Epetra_FECrsMatrix::InputNonlocalValue(int rowoffset,
					   int col, double value,
					   int mode)
{
  int*& colIndices = nonlocalCols_[rowoffset];
  double*& coefs = nonlocalCoefs_[rowoffset];
  int len = nonlocalRowLengths_[rowoffset];

  int insertPoint = -1;
  int coloffset = Epetra_Util_binary_search(col, colIndices,
					    len, insertPoint);

  if (coloffset >= 0) {
    if (mode == SUMINTO || mode == INSERT) {
      coefs[coloffset] += value;
    }
    else {
      coefs[coloffset] = value;
    }
  }
  else {
    //else
    //  insert col in colIndices
    //  insert value in coefs

    int tmp1 = nonlocalRowLengths_[rowoffset];
    int tmp2 = nonlocalRowAllocLengths_[rowoffset];
    EPETRA_CHK_ERR( Epetra_Util_insert(col, insertPoint, colIndices, tmp1, tmp2));
    EPETRA_CHK_ERR( Epetra_Util_insert(value, insertPoint, coefs,
				       nonlocalRowLengths_[rowoffset],
				       nonlocalRowAllocLengths_[rowoffset]));
  }

  return(0);
}

-------------- next part --------------

//@HEADER
/*
************************************************************************

              Epetra: Linear Algebra Services Package 
                Copyright (2001) Sandia Corporation

Under terms of Contract DE-AC04-94AL85000, there is a non-exclusive
license for use of this work by or on behalf of the U.S. Government.

This library is free software; you can redistribute it and/or modify
it under the terms of the GNU Lesser General Public License as
published by the Free Software Foundation; either version 2.1 of the
License, or (at your option) any later version.
 
This library is distributed in the hope that it will be useful, but
WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
Lesser General Public License for more details.
 
You should have received a copy of the GNU Lesser General Public
License along with this library; if not, write to the Free Software
Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307
USA
Questions? Contact Michael A. Heroux (maherou at sandia.gov) 

************************************************************************
*/
//@HEADER

#ifndef EPETRA_FECRSMATRIX_H
#define EPETRA_FECRSMATRIX_H

#include <Epetra_CrsMatrix.h>
class Epetra_Map;
class Epetra_IntSerialDenseVector;
class Epetra_SerialDenseMatrix;

/** Epetra Finite-Element CrsMatrix. This class provides the ability to
    input finite-element style sub-matrix data, including sub-matrices with
    non-local rows (which could correspond to shared finite-element nodes for
    example). This class inherits Epetra_CrsMatrix, and so all Epetra_CrsMatrix
    functionality is also available.

    It is intended that this class will be used as follows:
    <ul>
    <li> Construct with either a map or graph that describes a (non-overlapping)
    data distribution.
    <li> Input data, including non-local data, using the methods
    InsertGlobalValues(), SumIntoGlobalValues() and/or ReplaceGlobalValues().
    <li> Call the method GlobalAssemble(), which gathers all non-local data
    onto the owning processors as determined by the map provided at
    construction. Users should note that the GlobalAssemble() method has an
    optional argument which determines whether GlobalAssemble() in turn calls
    FillComplete() after the data-exchange has occurred. If not explicitly
    supplied, this argument defaults to true.
    ***NOTE***: When GlobalAssemble() calls FillComplete(), it passes the
    arguments 'DomainMap()' and 'RangeMap()', which are the map attributes
    held by the base-class CrsMatrix and its graph. If a rectangular matrix
    is being assembled, the correct domain-map and range-map must be passed to
    GlobalAssemble (there are two overloadings of this method) -- otherwise, it
    has no way of knowing what these maps should really be.
    </ul>

    Sub-matrix data, which is assumed to be a rectangular 'table' of
    coefficients accompanied by 'scatter-indices', can be provided in three
    forms:
    <ul>
    <li>Fortran-style packed 1-D array.
    <li>C-style double-pointer, or list-of-rows.
    <li>Epetra_SerialDenseMatrix object.
    </ul>
    In all cases, a "format" parameter specifies whether the data is laid out
    in row-major or column-major order (i.e., whether coefficients for a row
    lie contiguously or whether coefficients for a column lie contiguously).
    See the documentation for the methods SumIntoGlobalValues() and
    ReplaceGlobalValues().

    Important notes:
    <ol>
    <li> Since Epetra_FECrsMatrix inherits Epetra_CrsMatrix, the semantics of
    the Insert/SumInto/Replace methods are the same as they are on
    Epetra_CrsMatrix, which is:
    <ul>
    <li>InsertGlobalValues() inserts values into the matrix only if the graph
    has not yet been finalized (FillComplete() has not yet been called). For
    non-local values, the call to InsertGlobalValues() may succeed but the
    GlobalAssemble() method may then fail because the non-local data is not
    actually inserted in the underlying matrix until GlobalAssemble() is called.
    <li>SumIntoGlobalValues() and ReplaceGlobalValues() only work for values
    that already exist in the matrix. In other words, these methods can not be
    used to put new values into the matrix.
    </ul>
    </ol>
*/
class EPETRA_LIB_DLL_EXPORT Epetra_FECrsMatrix : public Epetra_CrsMatrix {
  public:
  /** Constructor. */
   Epetra_FECrsMatrix(Epetra_DataAccess CV,
		      const Epetra_Map& RowMap,
		      int* NumEntriesPerRow,
		      bool ignoreNonLocalEntries=false);

   /** Constructor. */
   Epetra_FECrsMatrix(Epetra_DataAccess CV,
		      const Epetra_Map& RowMap,
		      int NumEntriesPerRow,
		      bool ignoreNonLocalEntries=false);

  /** Constructor. */
   Epetra_FECrsMatrix(Epetra_DataAccess CV,
		      const Epetra_Map& RowMap,
		      const Epetra_Map& ColMap,
		      int* NumEntriesPerRow,
		      bool ignoreNonLocalEntries=false);

   /** Constructor. */
   Epetra_FECrsMatrix(Epetra_DataAccess CV,
		      const Epetra_Map& RowMap,
		      const Epetra_Map& ColMap,
		      int NumEntriesPerRow,
		      bool ignoreNonLocalEntries=false);

   /** Constructor. */
   Epetra_FECrsMatrix(Epetra_DataAccess CV,
		      const Epetra_CrsGraph& Graph,
		      bool ignoreNonLocalEntries=false);

   /** Copy Constructor. */
   Epetra_FECrsMatrix(const Epetra_FECrsMatrix& src);

   /** Destructor. */
   virtual ~Epetra_FECrsMatrix();

   /** Assignment operator */
   Epetra_FECrsMatrix& operator=(const Epetra_FECrsMatrix& src);

   enum { ROW_MAJOR = 0, COLUMN_MAJOR = 3 };

   using Epetra_CrsMatrix::SumIntoGlobalValues;
   using Epetra_CrsMatrix::InsertGlobalValues;
   using Epetra_CrsMatrix::ReplaceGlobalValues;

   /** override base-class Epetra_CrsMatrix::SumIntoGlobalValues method */
   int SumIntoGlobalValues(int GlobalRow, int NumEntries,
                           double* Values, int* Indices);

   /** override base-class Epetra_CrsMatrix::InsertGlobalValues method */
   int InsertGlobalValues(int GlobalRow, int NumEntries,
                           double* Values, int* Indices);

   /** override base-class Epetra_CrsMatrix::ReplaceGlobalValues method */
   int ReplaceGlobalValues(int GlobalRow, int NumEntries,
                           double* Values, int* Indices);

   /** Sum a Fortran-style table (single-dimensional packed-list) of
       coefficients into the matrix, adding them to any coefficients that
       may already exist at the specified row/column locations.

       @param numIndices Number of rows (and columns) in the sub-matrix.
       @param indices List of scatter-indices (rows and columns) for the
       sub-matrix.
       @param values List, length numIndices*numIndices. Square sub-matrix of
       coefficients, packed in a 1-D array. Data is packed either contiguously
       by row or by column, specified by the final parameter 'format'.
       @param format Specifies whether the data in 'values' is packed in
       column-major or row-major order. Valid values are
       Epetra_FECrsMatrix::ROW_MAJOR or Epetra_FECrsMatrix::COLUMN_MAJOR. This
       is an optional parameter, default value is COLUMN_MAJOR.
   */
   int SumIntoGlobalValues(int numIndices, const int* indices,
                           const double* values,
                           int format=Epetra_FECrsMatrix::COLUMN_MAJOR);

   /** Sum a Fortran-style table (single-dimensional packed-list) of
       coefficients into the matrix, adding them to any coefficients that
       may already exist at the specified row/column locations.

       @param numRows Number of rows in the sub-matrix.
       @param rows List of row-numbers (scatter-indices) for the sub-matrix.
       @param numCols Number of columns in the sub-matrix.
       @param cols List of column-numbers (scatter-indices) for the sub-matrix.
       @param values List, length numRows*numCols. Rectangular sub-matrix of
       coefficients, packed in a 1-D array. Data is packed either contiguously
       by row or by column, specified by the final parameter 'format'.
       @param format Specifies whether the data in 'values' is packed in
       column-major or row-major order. Valid values are
       Epetra_FECrsMatrix::ROW_MAJOR or Epetra_FECrsMatrix::COLUMN_MAJOR. This
       is an optional parameter, default value is COLUMN_MAJOR.
   */
   int SumIntoGlobalValues(int numRows, const int* rows,
                           int numCols, const int* cols,
                           const double* values,
                           int format=Epetra_FECrsMatrix::COLUMN_MAJOR);

   /** Sum C-style table (double-pointer, or list of lists) of coefficients
       into the matrix, adding them to any coefficients that
       may already exist at the specified row/column locations.

       @param numIndices Number of rows (and columns) in the sub-matrix.
       @param indices List of scatter-indices (rows and columns) for the
       sub-matrix.
       @param values Square sub-matrix of coefficients, provided in a 2-D
       array, or double-pointer.
       @param format Specifies whether the data in 'values' is packed in
       column-major or row-major order. Valid values are
       Epetra_FECrsMatrix::ROW_MAJOR or Epetra_FECrsMatrix::COLUMN_MAJOR. This
       is an optional parameter, default value is ROW_MAJOR.
   */
   int SumIntoGlobalValues(int numIndices, const int* indices,
                           const double* const* values,
                           int format=Epetra_FECrsMatrix::ROW_MAJOR);

   /** Sum C-style table (double-pointer, or list of lists) of coefficients
       into the matrix, adding them to any coefficients that
       may already exist at the specified row/column locations.

       @param numRows Number of rows in the sub-matrix.
       @param rows List of row-numbers (scatter-indices) for the sub-matrix.
       @param numCols Number of columns in the sub-matrix.
       @param cols List of column-numbers (scatter-indices) for the sub-matrix.
       @param values Rectangular sub-matrix of coefficients, provided in a 2-D
       array, or double-pointer.
       @param format Specifies whether the data in 'values' is packed in
       column-major or row-major order. Valid values are
       Epetra_FECrsMatrix::ROW_MAJOR or Epetra_FECrsMatrix::COLUMN_MAJOR. This
       is an optional parameter, default value is ROW_MAJOR.
   */
   int SumIntoGlobalValues(int numRows, const int* rows,
	                   int numCols, const int* cols,
                           const double* const* values,
                           int format=Epetra_FECrsMatrix::ROW_MAJOR);

   /** Insert a Fortran-style table (single-dimensional packed-list) of
       coefficients into the matrix.

       @param numIndices Number of rows (and columns) in the sub-matrix.
       @param indices List of scatter-indices (rows and columns) for the
       sub-matrix.
       @param values List, length numIndices*numIndices. Square sub-matrix of
       coefficients, packed in a 1-D array. Data is packed either contiguously
       by row or by column, specified by the final parameter 'format'.
       @param format Specifies whether the data in 'values' is packed in
       column-major or row-major order. Valid values are
       Epetra_FECrsMatrix::ROW_MAJOR or Epetra_FECrsMatrix::COLUMN_MAJOR. This
       is an optional parameter, default value is COLUMN_MAJOR.
   */
   int InsertGlobalValues(int numIndices, const int* indices,
                           const double* values,
                           int format=Epetra_FECrsMatrix::COLUMN_MAJOR);

   /** Insert a Fortran-style table (single-dimensional packed-list) of
       coefficients into the matrix.

       @param numRows Number of rows in the sub-matrix.
       @param rows List of row-numbers (scatter-indices) for the sub-matrix.
       @param numCols Number of columns in the sub-matrix.
       @param cols List of column-numbers (scatter-indices) for the sub-matrix.
       @param values List, length numRows*numCols. Rectangular sub-matrix of
       coefficients, packed in a 1-D array. Data is packed either contiguously
       by row or by column, specified by the final parameter 'format'.
       @param format Specifies whether the data in 'values' is packed in
       column-major or row-major order. Valid values are
       Epetra_FECrsMatrix::ROW_MAJOR or Epetra_FECrsMatrix::COLUMN_MAJOR. This
       is an optional parameter, default value is COLUMN_MAJOR.
   */
   int InsertGlobalValues(int numRows, const int* rows,
                           int numCols, const int* cols,
                           const double* values,
                           int format=Epetra_FECrsMatrix::COLUMN_MAJOR);

   /** Insert a C-style table (double-pointer, or list of lists) of coefficients
       into the matrix.

       @param numIndices Number of rows (and columns) in the sub-matrix.
       @param indices List of scatter-indices (rows and columns) for the
       sub-matrix.
       @param values Square sub-matrix of coefficients, provided in a 2-D
       array, or double-pointer.
       @param format Specifies whether the data in 'values' is packed in
       column-major or row-major order. Valid values are
       Epetra_FECrsMatrix::ROW_MAJOR or Epetra_FECrsMatrix::COLUMN_MAJOR. This
       is an optional parameter, default value is ROW_MAJOR.
   */
   int InsertGlobalValues(int numIndices, const int* indices,
                           const double* const* values,
                           int format=Epetra_FECrsMatrix::ROW_MAJOR);

   /** Insert a C-style table (double-pointer, or list of lists) of coefficients
       into the matrix.

       @param numRows Number of rows in the sub-matrix.
       @param rows List of row-numbers (scatter-indices) for the sub-matrix.
       @param numCols Number of columns in the sub-matrix.
       @param cols List of column-numbers (scatter-indices) for the sub-matrix.
       @param values Rectangular sub-matrix of coefficients, provided in a 2-D
       array, or double-pointer.
       @param format Specifies whether the data in 'values' is packed in
       column-major or row-major order. Valid values are
       Epetra_FECrsMatrix::ROW_MAJOR or Epetra_FECrsMatrix::COLUMN_MAJOR. This
       is an optional parameter, default value is ROW_MAJOR.
   */
   int InsertGlobalValues(int numRows, const int* rows,
	                   int numCols, const int* cols,
                           const double* const* values,
                           int format=Epetra_FECrsMatrix::ROW_MAJOR);

   /** Copy a Fortran-style table (single-dimensional packed-list) of
       coefficients into the matrix, replacing any coefficients that
       may already exist at the specified row/column locations.

       @param numIndices Number of rows (and columns) in the sub-matrix.
       @param indices List of scatter-indices (rows and columns) for the
       sub-matrix.
       @param values List, length numIndices*numIndices. Square sub-matrix of
       coefficients, packed in a 1-D array. Data is packed either contiguously
       by row or by column, specified by the final parameter 'format'.
       @param format Specifies whether the data in 'values' is packed in
       column-major or row-major order. Valid values are
       Epetra_FECrsMatrix::ROW_MAJOR or Epetra_FECrsMatrix::COLUMN_MAJOR. This
       is an optional parameter, default value is COLUMN_MAJOR.
   */
   int ReplaceGlobalValues(int numIndices, const int* indices,
                           const double* values,
                           int format=Epetra_FECrsMatrix::COLUMN_MAJOR);

   /** Copy Fortran-style table (single-dimensional packed-list) of coefficients
       into the matrix, replacing any coefficients that
       may already exist at the specified row/column locations.

       @param numRows Number of rows in the sub-matrix.
       @param rows List of row-numbers (scatter-indices) for the sub-matrix.
       @param numCols Number of columns in the sub-matrix.
       @param cols List, of column-numbers 
       (scatter-indices) for the sub-matrix.
       @param values List, length numRows*numCols. Rectangular sub-matrix of
       coefficients, packed in a 1-D array. Data is packed either contiguously
       by row or by column, specified by the final parameter 'format'.
       @param format Specifies whether the data in 'values' is packed in
       column-major or row-major order. Valid values are
       Epetra_FECrsMatrix::ROW_MAJOR or Epetra_FECrsMatrix::COLUMN_MAJOR. This
       is an optional parameter, default value is COLUMN_MAJOR.
   */
   int ReplaceGlobalValues(int numRows, const int* rows,
                           int numCols, const int* cols,
                           const double* values,
                           int format=Epetra_FECrsMatrix::COLUMN_MAJOR);

   /** Copy C-style table (double-pointer, or list of lists) of coefficients
       into the matrix, replacing any coefficients that
       may already exist at the specified row/column locations.

       @param numIndices Number of rows (and columns) in the sub-matrix.
       @param indices List of scatter-indices (rows and columns) for the
       sub-matrix.
       @param values Square sub-matrix of coefficients, provided in a 2-D
       array, or double-pointer.
       @param format Specifies whether the data in 'values' is packed in
       column-major or row-major order. Valid values are
       Epetra_FECrsMatrix::ROW_MAJOR or Epetra_FECrsMatrix::COLUMN_MAJOR. This
       is an optional parameter, default value is ROW_MAJOR.
   */
   int ReplaceGlobalValues(int numIndices, const int* indices,
                           const double* const* values,
                           int format=Epetra_FECrsMatrix::ROW_MAJOR);

   /** Copy C-style table (double-pointer, or list of lists) of coefficients
       into the matrix, replacing any coefficients that
       may already exist at the specified row/column locations.

       @param numRows Number of rows in the sub-matrix.
       @param rows List of row-numbers (scatter-indices) for the sub-matrix.
       @param numCols Number of columns in the sub-matrix.
       @param cols List of column-numbers (scatter-indices) for the sub-matrix.
       @param values Rectangular sub-matrix of coefficients, provided in a 2-D
       array, or double-pointer.
       @param format Specifies whether the data in 'values' is packed in
       column-major or row-major order. Valid values are
       Epetra_FECrsMatrix::ROW_MAJOR or Epetra_FECrsMatrix::COLUMN_MAJOR. This
       is an optional parameter, default value is ROW_MAJOR.
   */
   int ReplaceGlobalValues(int numRows, const int* rows,
                           int numCols, const int* cols,
                           const double* const* values,
                           int format=Epetra_FECrsMatrix::ROW_MAJOR);

   /** Sum a square structurally-symmetric sub-matrix into the global matrix.
       For non-square sub-matrices, see the other overloading of this method.

       @param indices List of scatter-indices. indices.Length() must be the same
       as values.M() and values.N().

       @param values Sub-matrix of coefficients. Must be square.

       @param format Optional format specifier, defaults to COLUMN_MAJOR.
   */
   int SumIntoGlobalValues(const Epetra_IntSerialDenseVector& indices,
			   const Epetra_SerialDenseMatrix& values,
			   int format=Epetra_FECrsMatrix::COLUMN_MAJOR);

   /** Sum a general sub-matrix into the global matrix.
       For square structurally-symmetric sub-matrices, see the other
       overloading of this method.

       @param rows List of row-indices. rows.Length() must be the same
       as values.M().

       @param cols List of column-indices. cols.Length() must be the same
       as values.N().

       @param values Sub-matrix of coefficients.

       @param format Optional format specifier, defaults to COLUMN_MAJOR.
   */
   int SumIntoGlobalValues(const Epetra_IntSerialDenseVector& rows,
			   const Epetra_IntSerialDenseVector& cols,
			   const Epetra_SerialDenseMatrix& values,
			   int format=Epetra_FECrsMatrix::COLUMN_MAJOR);

   /** Insert a square structurally-symmetric sub-matrix into the global matrix.
       For non-square sub-matrices, see the other overloading of this method.

       @param indices List of scatter-indices. indices.Length() must be the same
       as values.M() and values.N().

       @param values Sub-matrix of coefficients. Must be square.

       @param format Optional format specifier, defaults to COLUMN_MAJOR.
   */
   int InsertGlobalValues(const Epetra_IntSerialDenseVector& indices,
			   const Epetra_SerialDenseMatrix& values,
			   int format=Epetra_FECrsMatrix::COLUMN_MAJOR);

   /** Insert a general sub-matrix into the global matrix.
       For square structurally-symmetric sub-matrices, see the other
       overloading of this method.

       @param rows List of row-indices. rows.Length() must be the same
       as values.M().

       @param cols List of column-indices. cols.Length() must be the same
       as values.N().

       @param values Sub-matrix of coefficients.

       @param format Optional format specifier, defaults to COLUMN_MAJOR.
   */
   int InsertGlobalValues(const Epetra_IntSerialDenseVector& rows,
			   const Epetra_IntSerialDenseVector& cols,
			   const Epetra_SerialDenseMatrix& values,
			   int format=Epetra_FECrsMatrix::COLUMN_MAJOR);

   /** Use a square structurally-symmetric sub-matrix to replace existing
       values in the global matrix.
       For non-square sub-matrices, see the other overloading of this method.

       @param indices List of scatter-indices. indices.Length() must be the same
       as values.M() and values.N().

       @param values Sub-matrix of coefficients. Must be square.

       @param format Optional format specifier, defaults to COLUMN_MAJOR.
   */
   int ReplaceGlobalValues(const Epetra_IntSerialDenseVector& indices,
			   const Epetra_SerialDenseMatrix& values,
			   int format=Epetra_FECrsMatrix::COLUMN_MAJOR);

   /** Use a general sub-matrix to replace existing values.
       For square structurally-symmetric sub-matrices, see the other
       overloading of this method.

       @param rows List of row-indices. rows.Length() must be the same
       as values.M().

       @param cols List of column-indices. cols.Length() must be the same
       as values.N().

       @param values Sub-matrix of coefficients.

       @param format Optional format specifier, defaults to COLUMN_MAJOR.
   */
   int ReplaceGlobalValues(const Epetra_IntSerialDenseVector& rows,
			   const Epetra_IntSerialDenseVector& cols,
			   const Epetra_SerialDenseMatrix& values,
			   int format=Epetra_FECrsMatrix::COLUMN_MAJOR);

   /** Gather any overlapping/shared data into the non-overlapping partitioning
      defined by the Map that was passed to this matrix at construction time.
      Data imported from other processors is stored on the owning processor
      with a "sumInto" or accumulate operation.
      This is a collective method -- every processor must enter it before any
      will complete it.

      ***NOTE***: When GlobalAssemble() calls FillComplete(), it passes the
      arguments 'DomainMap()' and 'RangeMap()', which are the map attributes
      held by the base-class CrsMatrix and its graph. If a rectangular matrix
      is being assembled, the domain-map and range-map must be specified by
      calling the other overloading of this method. Otherwise, GlobalAssemble()
      has no way of knowing what these maps should really be.


      @param callFillComplete option argument, defaults to true.
        Determines whether GlobalAssemble() internally calls the
        FillComplete() method on this matrix.
        
      @param CombineMode option argument, defaults to true. Determines the behaviour 
        when combining non local elements.

      @return error-code 0 if successful, non-zero if some error occurs
   */
   int GlobalAssemble(bool callFillComplete=true,
		      Epetra_CombineMode CombineMode = Add);

   /** Gather any overlapping/shared data into the non-overlapping partitioning
      defined by the Map that was passed to this matrix at construction time.
      Data imported from other processors is stored on the owning processor
      with a "sumInto" or accumulate operation.
      This is a collective method -- every processor must enter it before any
      will complete it.

      ***NOTE***: When GlobalAssemble() (the other overloading of this method)
      calls FillComplete(), it passes the arguments 'DomainMap()' and
      'RangeMap()', which are the map attributes already held by the base-class
      CrsMatrix and its graph. If a rectangular matrix is being assembled, the
      domain-map and range-map must be specified. Otherwise, GlobalAssemble()
      has no way of knowing what these maps should really be.


      @param domain_map user-supplied domain map for this matrix

      @param range_map user-supplied range map for this matrix

      @param callFillComplete option argument, defaults to true.
        Determines whether GlobalAssemble() internally calls the
        FillComplete() method on this matrix.
        
      @param CombineMode option argument, defaults to true. Determines the behaviour 
        when combining non local elements.

      @return error-code 0 if successful, non-zero if some error occurs
   */
   int GlobalAssemble(const Epetra_Map& domain_map,
                      const Epetra_Map& range_map,
                      bool callFillComplete=true,
		      Epetra_CombineMode CombineMode = Add);

   /** Set whether or not non-local data values should be ignored. By default,
       non-local data values are NOT ignored.
    */
   void setIgnoreNonLocalEntries(bool flag) {
     ignoreNonLocalEntries_ = flag;
   }

  private:
   void DeleteMemory();

   enum {SUMINTO = 0, REPLACE = 1, INSERT = 2};

   int InputGlobalValues(int numRows, const int* rows,
                         int numCols, const int* cols,
                         const double* const* values,
                         int format,
                         int mode);

   int InputGlobalValues(int numRows, const int* rows,
                         int numCols, const int* cols,
                         const double* values,
                         int format,
                         int mode);

   int InputNonlocalGlobalValues(int row,
				 int numCols, const int* cols,
				 const double* values,
				 int mode);

   int InsertNonlocalRow(int row, int offset);

   int InputNonlocalValue(int rowoffset,
			  int col, double value,
			  int mode);

   int myFirstRow_;
   int myNumRows_;

   bool ignoreNonLocalEntries_;

   int numNonlocalRows_;
   int* nonlocalRows_;
   int* nonlocalRowLengths_;
   int* nonlocalRowAllocLengths_;
   int** nonlocalCols_;
   double** nonlocalCoefs_;

   double* workData_;
   int workDataLength_;
};//class Epetra_FECrsMatrix

#endif /* EPETRA_FECRSMATRIX_H */
-------------- next part --------------
// @HEADER
// ***********************************************************************
// 
//                      Didasko Tutorial Package
//                 Copyright (2005) Sandia Corporation
// 
// Under terms of Contract DE-AC04-94AL85000, there is a non-exclusive
// license for use of this work by or on behalf of the U.S. Government.
// 
// This library is free software; you can redistribute it and/or modify
// it under the terms of the GNU Lesser General Public License as
// published by the Free Software Foundation; either version 2.1 of the
// License, or (at your option) any later version.
//  
// This library is distributed in the hope that it will be useful, but
// WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
// Lesser General Public License for more details.
//  
// You should have received a copy of the GNU Lesser General Public
// License along with this library; if not, write to the Free Software
// Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307
// USA
// Questions? Contact Michael A. Heroux (maherou at sandia.gov) 
// 
// ***********************************************************************
// @HEADER

// Show the usage of RowMap, ColumnMap, RangeMap and DomainMap
// This code must be run with two processes

#include "Didasko_ConfigDefs.h"
#if defined(HAVE_DIDASKO_EPETRA)

#include "Epetra_ConfigDefs.h"
#ifdef HAVE_MPI
#include "mpi.h"
#include "Epetra_MpiComm.h"
#else
#include "Epetra_SerialComm.h"
#endif
#include "Epetra_Vector.h"
#include "Epetra_Map.h"
#include "Epetra_SerialDenseMatrix.h"
#include "Epetra_FECrsGraph.h"
#include "Epetra_FECrsMatrix.h"
#include "Epetra_Import.h"
void check_err(int& ierr)
{
  if(ierr<0)
    cout << "++++++++++++++++++++++++++++++++++++++++++++++++++++++ trilinos error!! " << ierr << endl;
  else  if(ierr>0)
    cout << "performance warning!! " << ierr << endl;

}

int main(int argc, char *argv[])
{

#ifdef HAVE_MPI
  MPI_Init(&argc, &argv);
  Epetra_MpiComm Comm(MPI_COMM_WORLD);
#else
  Epetra_SerialComm Comm;
#endif

  int MyPID = Comm.MyPID();
  int NumProc = Comm.NumProc();

  if( NumProc != 2 ) {
#ifdef HAVE_MPI
    MPI_Finalize();
#endif
    return(EXIT_SUCCESS);
  }

  // I define two maps, with and without overlapping elements:
  int non_overlap_nelements = 0;
  
  
  switch( MyPID ) {
  case 0:
    non_overlap_nelements = 2;
    break;

  case 1:
    non_overlap_nelements = 2;
    break;
  }

  int* non_overlapping_indices = new int[non_overlap_nelements];

  //i assing the indices so that proc0 owns 0 1 and proc2 owns 2 3
  //1 and 2 are overlapping in the overlappign map
  switch( MyPID ) {
  case 0:
    non_overlapping_indices[0] = 0; non_overlapping_indices[1] = 1; 
    break;
  case 1:
    non_overlapping_indices[0] = 2; non_overlapping_indices[1] = 3; 
    break;
  }
 
  Epetra_Map NonOverlappingMap(-1, non_overlap_nelements, non_overlapping_indices, 0, Comm);  
  
  cout << "map defined correctly" << endl;
  
  //create the matrix graph
  int guess_row_size = 3;
  Epetra_FECrsGraph Agraph(Copy, NonOverlappingMap, guess_row_size);
  
  //fill the graph
  int* assembling_temp = new int[2];
  int ierr = 0;
  switch( MyPID ) {
  case 0:
    assembling_temp[0] = 0; assembling_temp[1] = 1;
    ierr = Agraph.InsertGlobalIndices(2,assembling_temp,2, assembling_temp);
    check_err(ierr);
    cout << "proc= " << MyPID << " ierr = " << ierr << endl;
    assembling_temp[0] = 1; assembling_temp[1] = 2;
    ierr = Agraph.InsertGlobalIndices(2,assembling_temp,2, assembling_temp);
    cout << "proc= " << MyPID << " ierr = " << ierr << endl;
    check_err(ierr);
    break;
  case 1:
    assembling_temp[0] = 2; assembling_temp[1] = 3;
    ierr = Agraph.InsertGlobalIndices(2,assembling_temp,2, assembling_temp);
    cout << "proc= " << MyPID << " ierr = " << ierr << endl;
    check_err(ierr);
    break;
  } 
  delete [] assembling_temp;
  cout << "ln136" << endl;

  int graph_assemble_ierr = Agraph.GlobalAssemble();
  check_err(graph_assemble_ierr);
  cout << "graph defined correctly" << endl;
  
  //create a FE matrix (using the precomputed graph) and set it to zero
  Epetra_FECrsMatrix A(Copy,Agraph);
  A.PutScalar(0.0);

  //now do some simple FE assembly of 3 elements of the type (1 -1; -1 1) 
  Epetra_IntSerialDenseVector indices(2);
  Epetra_SerialDenseMatrix values(2, 2);
  values(0,0) = 1.0; values(0,1) = -1.0; values(1,0) = -1.0; values(1,1) = 1.0;

  switch( MyPID ) {
  case 0:
    values(0,0) = 10; values(0,1) = 10; values(1,0) = 10; values(1,1) = 10;
    indices(0) = 0; indices(1) = 1;
    ierr = A.ReplaceGlobalValues(indices, values);
    check_err(ierr);

    indices(0) = 1; indices(1) = 2;
    ierr = A.ReplaceGlobalValues(indices, values);
    check_err(ierr);
    

    break;
  case 1:
    values(0,0) = 20; values(0,1) = 20; values(1,0) = 20; values(1,1) = 20;

    indices(0) = 2; indices(1) = 3;
    ierr = A.ReplaceGlobalValues(indices, values);
    check_err(ierr);
    
    break;
  } 
  cout << "before Global Assemble" << endl;
  ierr = A.GlobalAssemble(true,Insert);  
  check_err(ierr);
  
  cout << "after Global Assemble" << endl;
  
  cout << A;
  
  
#ifdef HAVE_MPI
  MPI_Finalize();
#endif

  return(EXIT_SUCCESS);
  
}


#else

#include <stdlib.h>
#include <stdio.h>

int main(int argc, char *argv[])
{
  puts("Please configure Didasko with:\n"
       "--enable-epetra");

  return 0;
}
#endif


More information about the Trilinos-Users mailing list