[Trilinos-Users] [ZOLTAN] Problem running Zoltan unit test

Kudo, Jun jun.kudo at aramcoservices.com
Fri Jun 9 14:02:15 EDT 2017


Hello!
I'm having some unexpected results when running the built-in Zoltan unit test "problemGRAPH.exe" with multiple processors and I was hoping someone could point to any areas of potential issues.

Here's my configuration statement :
../configure CC=/share/apps/openmpi-intel/2.0.0/bin/mpicc CXX=/share/apps/openmpi-intel/2.0.0/bin/mpic++ CFLAGS="-xAVX -fPIC" CXXFLAGS="-xAVX -fPIC" FC=mpif90 FFLAGS="-xAVX -fPIC" \
--with-id-type=ulong \
--enable-f90interface \
--enable-mpi \
--with-mpi=/share/apps/openmpi-intel/2.0.0 \
--with-mpi-compilers=/share/apps/openmpi-intel/2.0.0/bin \
--with-mpi-libdir="/share/apps/openmpi-intel/2.0.0/lib" \
--with-mpi-incdir="/share/apps/openmpi-intel/2.0.0/include" \
--with-parmetis \
--with-parmetis-libdir="/share/apps/parmetis/4.0.3_mod/lib" \
--with-parmetis-incdir="/share/apps/parmetis/4.0.3_mod/include" \
--with-ldflags="-L/share/apps/metis/5.1.0/lib" \
--with-incdirs="-I/share/apps/metis/5.1.0/include" \
2>&1 | tee CONFIGURE_fPIC.out


Here's my output :
$ mpirun -n 2 ./problemGRAPH.exe
ZOLTAN Load balancing method = 9 (GRAPH)
Build configuration:

  ZOLTAN_ID_TYPE: unsigned long (8 bytes)
  ZOLTAN_GNO_TYPE: ssize_t, (8 bytes)
  MPI_Datatype for ZOLTAN_ID_TYPE: MPI_UNSIGNED_LONG
  MPI_Datatype for ZOLTAN_GNO_TYPE: MPI_LONG
  Third party library: ParMetis version 4.0.3

ZOLTAN Parameter IMBALANCE_TOL[0] = 1.100000
ZOLTAN Parameter AUTO_MIGRATE = FALSE
ZOLTAN Parameter MIGRATE_ONLY_PROC_CHANGES = 1
ZOLTAN Parameter OBJ_WEIGHT_DIM = 0
ZOLTAN Parameter EDGE_WEIGHT_DIM = 1
ZOLTAN Parameter DEBUG_LEVEL = 1
ZOLTAN Parameter DEBUG_PROCESSOR = 0
ZOLTAN Parameter DETERMINISTIC = TRUE
ZOLTAN Parameter TIMER = 1 (wall)
ZOLTAN Parameter NUM_GID_ENTRIES = 1
ZOLTAN Parameter NUM_LID_ENTRIES = 1
ZOLTAN Parameter RETURN_LISTS = IMPORT AND EXPORT
ZOLTAN Parameter NUM_GLOBAL_PARTS = -1
ZOLTAN Parameter NUM_LOCAL_PARTS = -1
ZOLTAN Parameter REMAP = 1
ZOLTAN Parameter SEED = 123456789 (123456789)
ZOLTAN Parameter LB_APPROACH = PARTITION
ZOLTAN Parameter GRAPH_PACKAGE = PARMETIS
ZOLTAN Parameter PARMETIS_METHOD = PARTKWAY
ZOLTAN Parameter PARMETIS_OUTPUT_LEVEL = 0
ZOLTAN Parameter PARMETIS_SEED = 15
ZOLTAN Parameter PARMETIS_ITR = 0.000000
ZOLTAN Parameter PARMETIS_COARSE_ALG = 2
ZOLTAN Parameter PARMETIS_FOLD = 0
Third party library real type is 4-byte real number
Third party library index type is 4-byte integer
Third party library weight type is 4-byte integer
ZOLTAN Parameter CHECK_GRAPH = 2
ZOLTAN Parameter SCATTER_GRAPH = 1
ZOLTAN Parameter FINAL_OUTPUT = 0
ZOLTAN Parameter USE_TIMERS = 0
ZOLTAN Parameter ADD_OBJ_WEIGHT = NONE
ZOLTAN Parameter GRAPH_SYMMETRIZE = NONE
ZOLTAN Parameter GRAPH_SYM_WEIGHT = ADD
ZOLTAN Parameter GRAPH_BIPARTITE_TYPE = OBJ
ZOLTAN Parameter GRAPH_BUILD_TYPE = NORMAL
ZOLTAN Parameter GRAPH_FAST_BUILD_BASE = 0
PARMETIS ERROR: Poor initial vertex distribution. Processor 0 has no vertices assigned to it!
Zoltan_LB No changes to the decomposition due to partitioning; no migration is needed.

Graph partition before calling Zoltan
1--x--0--x--1--x--0--x--1
x     x     x     x     x
0--x--1--x--0--x--1--x--0
x     x     x     x     x
1--x--0--x--1--x--0--x--1
x     x     x     x     x
0--x--1--x--0--x--1--x--0
x     x     x     x     x
1--x--0--x--1--x--0--x--1

Object imbalance (1.0 perfect, larger numbers are worse): 1.040000
Total number of edge cuts: 40

Graph partition after calling Zoltan
1--x--0--x--1--x--0--x--1
x     x     x     x     x
0--x--1--x--0--x--1--x--0
x     x     x     x     x
1--x--0--x--1--x--0--x--1
x     x     x     x     x
0--x--1--x--0--x--1--x--0
x     x     x     x     x
1--x--0--x--1--x--0--x--1

Object imbalance (1.0 perfect, larger numbers are worse): 1.040000
Total number of edge cuts: 40


For some reason, I am seeing the error "PARMETIS ERROR: Poor initial vertex distribution. Processor 0 has no vertices assigned to it!", when it's pretty clear to me the input graph has vertices assigned to both ranks 0 and 1.
When I do the equivalent build with GCC libraries, I actually encounter a segmentation fault.
Let me know if there's anything obviously wrong.

Thanks in advance,
Jun
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://trilinos.org/pipermail/trilinos-users/attachments/20170609/fb07d8d9/attachment-0001.html>


More information about the Trilinos-Users mailing list