[Trilinos-Users] [EXTERNAL] Re: decomp tool Error

Thu Feb 18 18:28:01 EST 2016

Hi Sai,

Just a guess, but what happens if you add the command-line flag -64 ?

Andrew

________________________________
From: Trilinos-Users <trilinos-users-bounces at trilinos.org> on behalf of Sai P Uppati <uppatis at utexas.edu>
Sent: Thursday, February 18, 2016 4:23 PM
To: Sjaardema, Gregory D
Cc: trilinos-users at trilinos.org
Subject: Re: [Trilinos-Users] [EXTERNAL] Re: decomp tool Error

UPDATE:

I think all other tools may be working fine from my trilinos build. The tools I commonly use from trilinos include decomp, epu, exodiff and epu. All except decomp seem to working fine.

Even after rebuilding trilinos several times (varying some options each time) and Peridigm passing all the tests each time, decomp throws errors like these:

Executing:
   /usr/local/trilinos/bin/nem_slice -e -S  -l inertial -c -o HEGF-res-cylin.g.nem -m mesh=4 HEGF-res-cylin.g
   ...see HEGF-res-cylin.g.decomp.out for nem_slice status

Beginning nem_slice execution.
Input Mesh File = 'HEGF-res-cylin.g'
Using 32-bit integer mode for decomposition...
Exodus Library Warning/Error: [ex_put_cmap_params_cc]
Error: failed to add dimension for "ncnt_cmap" of size 6313656973 in file ID 65536
NetCDF: Invalid dimension size
================================messages================================
fatal: unable to output communication map parameters
fatal: could not output Nemesis file

ERROR:******************************************************************
ERROR:
ERROR     During nem_slice execution. Check error output above and rerun
ERROR:
ERROR:******************************************************************

Sai
[https://mailfoogae.appspot.com/t?sender=adXBwYXRpc0B1dGV4YXMuZWR1&type=zerocontent&guid=11e726bf-5482-49da-9c86-c27869c8737b]ᐧ

On Thu, Feb 18, 2016 at 11:03 AM, Sai P Uppati <uppatis at utexas.edu<mailto:uppatis at utexas.edu>> wrote:
Hi Greg,

An example mesh I'm trying to decompose contains 178320 elements, 189405 nodes and 1 block. I tried decomposing for 4, 6 and 8 processors. I haven't had problems with previous Trilinos versions I was using before. I think it was only since I rebased to the official version hosted on the GitHub page.

Anyways, getopt I was able to fix with John Foster's help. I just installed a gnu-getopt version from Homebrew and modified the PATH variable to look for it first before looking in /usr/bin.

Coming to Netcdf, I followed the instructions exactly as they stated in the following page: https://peridigm.sandia.gov/content/netcdf. So, I disabled netcdf-4 and dap, and installed it using the changed numbers in netcdf.h file as well. All the tests passed when I did 'make check'. So I didn't think there were any issues with the netcdf installation. Doing it this way, however, there was no referencing the HDF5 build I did in the previous step. Even the in summary of netcdf configuation, the HDF5 support seems to off. I left HDF5 installed though because I saw that it maybe needed for the SEACAS package in Trilinos.

But as I mentioned before, I didn't have issues like this with previous Trilinos versions (I also didn't follow the netcdf instructions given at the webpage before, I just installed whatever was default from unidata). Perhaps, the instructions on the page are not completely correct?

Sorry for the long email, but those are all the details.

Sai

On Thu, Feb 18, 2016 at 7:38 AM, Sjaardema, Gregory D <gdsjaar at sandia.gov<mailto:gdsjaar at sandia.gov>> wrote:
What size mesh are you decomposing (#elem, #block, #node) and how many processors are you decompsing it for?

Did you also install hdf5 and reference it in the netcdf build for netcdf-4 support, or is it a netcdf build only?

The current getopt that you have will work, but will give reduced functionality in regards to long options which you can see by entering -H and -h and seeing the difference.  I’m not sure if installing the gnu-getopt in parallel with the system getopt would cause issues or not, but on my and many other macs we have both installed and have not noticed any issues (However, I use port instead of brew).

..Greg
--
"A supercomputer is a device for turning compute-bound problems into I/O-bound problems”

From: Trilinos-Users <trilinos-users-bounces at trilinos.org<mailto:trilinos-users-bounces at trilinos.org>> on behalf of "John T. Foster" <jfoster at austin.utexas.edu<mailto:jfoster at austin.utexas.edu>>
Date: Wednesday, February 17, 2016 at 6:00 PM
To: Sai P Uppati <uppatis at utexas.edu<mailto:uppatis at utexas.edu>>
Cc: "trilinos-users at trilinos.org<mailto:trilinos-users at trilinos.org>" <trilinos-users at trilinos.org<mailto:trilinos-users at trilinos.org>>
Subject: [EXTERNAL] Re: [Trilinos-Users] decomp tool Error

Sai,

I believe your using homebrew as a package manager so use:

brew install getopt

To install the getopt command line utility.

JTF

On Wednesday, February 17, 2016, Sai P Uppati <uppatis at utexas.edu<mailto:uppatis at utexas.edu>> wrote:
Hi,

I installed Trilinos and Peridigm (official versions hosted on GitHub) on my Mac OS X 10.11.3, including the dependencies boost, hdf5 and netcdf. I followed the instructions on Sandia's Peridigm installation guide to the dot.

The Peridigm unit tests all passed, which is good. However, when I try to use the decomp tool from Trilinos, I get the following errors:

########################################################################
The "getopt" executable that is available on this system is an older
version that is not compatible with the needs of the "decomp" tool.
If possible, you should update your getopt to a newer version and make
sure that the new getopt is in your path.

Below are some options for getting the current getopt version:
* If on a Mac: "sudo port install getopt"
* Search the internet for "getopt-1.1.5" or "getopt-1.1.4"; download and build

Enter "-h" for the modified options that this version supports.
Enter "-H" for the options that the standard version supports.
########################################################################

Executing:
   /usr/local/trilinos/bin/nem_slice -e -S  -l inertial -c -o prism-precrack.g.nem -m mesh=8 prism-precrack.g
   ...see prism-precrack.g.decomp.out for nem_slice status

Beginning nem_slice execution.
Input Mesh File = 'prism-precrack.g'
Using 32-bit integer mode for decomposition...
Exodus Library Warning/Error: [ex_put_cmap_params_cc]
Error: unable to output variable in file ID 65536
NetCDF: Index exceeds dimension bound
================================messages================================
fatal: unable to output communication map parameters
fatal: could not output Nemesis file

ERROR:******************************************************************
ERROR:
ERROR     During nem_slice execution. Check error output above and rerun
ERROR:
ERROR:******************************************************************

There are multiple errors here.

1) I don't know how to update the getopt executable. It seems Mac OS X already comes with a built in version (which I checked and found to be in /usr/bin), but this version in not compatible with decomp. I checked Homebrew, and there is a key only option to install gnu-getopt, but they have a warning that installing different versions in parallel can cause trouble. I'm not able to find any other working way to install get opt with out causing errors.

2) NetCDF error about exceeding dimensions. I installed the latest version of netcdf-c, 4.4.0. I changed the numbers in netcdf.h as instructed in the Peridigm installation guide. I have a feeling that this may have something to do with the error, but I'm not quite sure. All tests passed, however, when I installed netcdf from source.

There may be other errors I'm not seeing. Please, I would appreciate if I can get some guidance on how to address these errors.

Sai

--
Sent from iPhone

[https://mailfoogae.appspot.com/t?sender=adXBwYXRpc0B1dGV4YXMuZWR1&type=zerocontent&guid=32df3964-c134-447d-a50e-3021bc915ab5]ᐧ

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://trilinos.org/pipermail/trilinos-users/attachments/20160218/a56866ff/attachment.html>