[Trilinos-Users] Problem with Epetra_FECrsGraph and 64-bit indices

[Ramsey, James J CIV (US)]

I seem to have a problem where my program segfaults when I run the GlobalAssemble() method on Epetra_FECrsGraph and 64-bit indices are used. This seems to happen in particular when I manually specify the rows used on each processor. I have not found it to occur for 32-bit indices, or when the number of processors is two or fewer. I have reproduced the crashing with both OpenMPI and IntelMPI. Here's a contrived test case that reproduces the problem for me:

#include <iostream>
#include <bitset>

#include "Epetra_ConfigDefs.h"

#include "mpi.h"
#include "Epetra_MpiComm.h"
#include "Epetra_Map.h"
#include "Epetra_FECrsGraph.h"

#ifdef DOF_LONG_LONG
typedef long long DOFnum;
typedef Epetra_LongLongSerialDenseVector Epetra_DOFVector;
#else
typedef int DOFnum;
typedef Epetra_IntSerialDenseVector Epetra_DOFVector;
#endif

int main(int argc, char *argv[]) {

  MPI_Init(&argc, &argv);  
  Epetra_MpiComm Comm(MPI_COMM_WORLD);

  int nProcs = Comm.NumProc();
  int currProc = Comm.MyPID();

  if (nProcs > 4) {
    std::cerr << "Must use at most four processes for this test case.!\n" << std::endl;
    MPI_Abort(MPI_COMM_WORLD, 1);
  }

  Epetra_DOFVector elemNodes, myDofVec;

  switch (currProc) {
  case 0:
    elemNodes.Resize(3);

    elemNodes(0) = 0;
    elemNodes(1) = 2;
    elemNodes(2) = 3;

    myDofVec.Resize(2);

    myDofVec(0) = 4;
    myDofVec(1) = 6;
    break;
  case 1:
    elemNodes.Resize(4);

    elemNodes(0) = 0;
    elemNodes(1) = 1;
    elemNodes(2) = 3;
    elemNodes(3) = 4;

    myDofVec.Resize(2);

    myDofVec(0) = 2;
    myDofVec(1) = 3;
    break;
  case 2:
    elemNodes.Resize(4);

    elemNodes(0) = 2;
    elemNodes(1) = 3;
    elemNodes(2) = 5;
    elemNodes(3) = 6;

    myDofVec.Resize(2);

    myDofVec(0) = 0;
    myDofVec(1) = 1;
    break;
  case 3:
    elemNodes.Resize(3);

    elemNodes(0) = 3;
    elemNodes(1) = 4;
    elemNodes(2) = 6;

    myDofVec.Resize(1);

    myDofVec(0) = 5; 
    break;
  }

  Epetra_Map Map(static_cast<DOFnum>(-1), myDofVec.Length(), myDofVec.Values(), 0, Comm);
  Epetra_FECrsGraph Graph(Copy, Map, 1);

  Graph.InsertGlobalIndices(elemNodes.Length(), elemNodes.Values(),
			    elemNodes.Length(), elemNodes.Values());

  Graph.GlobalAssemble();

  std::cout << Graph;

  MPI_Finalize();
  return 0;
}

Basically, whether the above program crashes on me depends on whether I define DOF_LONG_LONG. I'm not sure if this is a bug or me missing something at my end, though.