[Trilinos-Users] [EXTERNAL] Tpetra::MatrixMarket: Write and Read MultiVector

[Andrey Prokopenko]

It seems that Tpetra's multivector writer is fundamentally broken.

It does the writing in chunks, where each processor sends its part of 
multivector to processor zero.
The problem is that each chunk is written consequently. So if a chunk 
has, say, 5 entries and two columns, the writer outputs 5 entries in the 
first column, and then 5 entries in the second column. Globally, 
however, it means that it mixes first and second columns.

I think the problem started when it was decided that the writer is not 
scalable enough, and that we should not assemble the global vector on 
processor 0. It would have worked, except that MatrixMarket format 
requires column-major order. So no cigar.

Mark,

Is my understanding what's happening correct?

-Andrey
On 02/09/2015 07:21 AM, Sander Schaffner wrote:
> Hello again,
>
> the new version helpt in the sense that it reads now the file. But 
> still it behaves like described before:
>
> [1 2 3 ... 63] -> [0 1 64 65 2 3 ...]
> [64 65 66 ... 127] -> [32 33 96 97 34 35 ...]
>
> I wrote a small test program: First I create a 64x2 MV on 32 cores and 
> write into the first MV  [1 2 3 ... 63] and into the second [64 65 66 
> ... 127]. Then I write out the map and the MV. After that I read both 
> files to create a new MV to finally write it out again. Since I 
> changed nothing inbetween and only read and write a valid file i 
> expect two identical files. But this is not the case.
>
> Can someone tell me if I'm doing something wrong in my Code, or is it 
> maybe a bug?
>
> Thanks
>
> Sander
>
> Output first File (test_64_2):
>
> %%MatrixMarket matrix array real general
> 64 2
> 0
> 1
> 64
> 65
> 2
> 3
> 66
> 67
> 4
> 5
> 68
> 69
> 6
> 7
> ...
>
> Second (test_64):
>
> %%MatrixMarket matrix array real general
> 64 2
> 0
> 1
> 32
> 33
> 64
> 65
> 96
> 97
> 2
> 3
> 34
> 35
> 66
> 67
>
> Am 06.02.2015 um 19:04 schrieb Andrey Prokopenko:
>> Hi Sander,
>>
>> I believe that was fixed (see 
>> https://software.sandia.gov/bugzilla/show_bug.cgi?id=6264).
>> Which version of Trilinos do you use?
>>
>> -Andrey
>> On 02/06/2015 02:25 AM, Sander Schaffner wrote:
>>> Hi
>>>
>>> I'm trying to write a MultiVector with 
>>> Tpetra::MatrixMarket::Writer.writeDenseFile. The goal is to read the 
>>> same MultiVector afterwards with 
>>> Tpetra::MatrixMarket::Reader.readDenseFile. But there are some 
>>> reasons why this doesn't work:
>>>
>>> If i generate a MultiVector with two Vectors (dimension 128) where 
>>> the first holds the values 0 and the second 1 I get this output on 
>>> 32 cores:
>>>
>>> %%MatrixMarket matrix array real general
>>> 128 2
>>> 0
>>> 0
>>> 0
>>> 0
>>>
>>> 1
>>> 1
>>> 1
>>> 1
>>>
>>> 0
>>> 0
>>> 0
>>> 0
>>>
>>> 1
>>> 1
>>> 1
>>> 1
>>>
>>> and so on. So I get first the local part of processor 0 for both 
>>> Vectors and so on.
>>>
>>> The reader does not accept this file. It complains about the spaces. 
>>> If I manually get ridd of them it reads the file. But then it 
>>> handles the first half of the values as Vector 1 and the second part 
>>> as Vector 2. So in the End we have the following:
>>>
>>> [0 0 0 0 0 ... 0] -> [0 0 0 0 1 1 1 1 ...]
>>> [1 1 1 1 ... 1] -> [0 0 0 0 1 1 1 1 ...]
>>>
>>> Is there a way of directly read in a MatrixMarket file created by 
>>> the Writer? What other options are there to write/read MultiVectors?
>>>
>>> Any help is appreciated.
>>>
>>> Sander
>>> _______________________________________________
>>> Trilinos-Users mailing list
>>> Trilinos-Users at software.sandia.gov
>>> https://software.sandia.gov/mailman/listinfo/trilinos-users
>>
>
>
>
> Am 06.02.2015 um 19:04 schrieb Andrey Prokopenko:
>> Hi Sander,
>>
>> I believe that was fixed (see 
>> https://software.sandia.gov/bugzilla/show_bug.cgi?id=6264).
>> Which version of Trilinos do you use?
>>
>> -Andrey
>> On 02/06/2015 02:25 AM, Sander Schaffner wrote:
>>> Hi
>>>
>>> I'm trying to write a MultiVector with 
>>> Tpetra::MatrixMarket::Writer.writeDenseFile. The goal is to read the 
>>> same MultiVector afterwards with 
>>> Tpetra::MatrixMarket::Reader.readDenseFile. But there are some 
>>> reasons why this doesn't work:
>>>
>>> If i generate a MultiVector with two Vectors (dimension 128) where 
>>> the first holds the values 0 and the second 1 I get this output on 
>>> 32 cores:
>>>
>>> %%MatrixMarket matrix array real general
>>> 128 2
>>> 0
>>> 0
>>> 0
>>> 0
>>>
>>> 1
>>> 1
>>> 1
>>> 1
>>>
>>> 0
>>> 0
>>> 0
>>> 0
>>>
>>> 1
>>> 1
>>> 1
>>> 1
>>>
>>> and so on. So I get first the local part of processor 0 for both 
>>> Vectors and so on.
>>>
>>> The reader does not accept this file. It complains about the spaces. 
>>> If I manually get ridd of them it reads the file. But then it 
>>> handles the first half of the values as Vector 1 and the second part 
>>> as Vector 2. So in the End we have the following:
>>>
>>> [0 0 0 0 0 ... 0] -> [0 0 0 0 1 1 1 1 ...]
>>> [1 1 1 1 ... 1] -> [0 0 0 0 1 1 1 1 ...]
>>>
>>> Is there a way of directly read in a MatrixMarket file created by 
>>> the Writer? What other options are there to write/read MultiVectors?
>>>
>>> Any help is appreciated.
>>>
>>> Sander
>>> _______________________________________________
>>> Trilinos-Users mailing list
>>> Trilinos-Users at software.sandia.gov
>>> https://software.sandia.gov/mailman/listinfo/trilinos-users
>>
>