MVSFORUMS.com :: View topic - Syncsort

through

[^/[Print]\]

MVSFORUMS.com -> Utilities

#1: Syncsort - sortin/sortout Author: rintu,

Posted: Tue Feb 28, 2006 9:39 pm
—
Hi,

We have a syncsort which has the same dataset (sequential) as sortin and sortout (both with disp=shr) and control parms as OPTION COPY,STOPAFT=12345678

recently while handling over 10 million records, the result seems to be inaccurate. Any idea whether there is a buffer limit for this type of operation

#2: Author: rintu,

Posted: Wed Mar 01, 2006 10:20 pm
—
Shocked

no one seems to know the answer - I tried other forums also !!!

#3: Author: kolusu, Location: San Jose

Posted: Thu Mar 02, 2006 11:20 am
—

Quote:

We have a syncsort which has the same dataset (sequential) as sortin and sortout (both with disp=shr) and control parms as OPTION COPY,STOPAFT=12345678

You are using the same input as output. If for somereason your job fails you will loose the input dataset.

Quote:

recently while handling over 10 million records,the result seems to be inaccurate. Any idea whether there is a buffer limit for this type of operation

What kind of inaccurate results are you getting? If your file has only 10 million records and your control cards say to stop after copying 12 million 345 thousand and 678 records, which is greater than the no: of input records , your file will have all the 10 million records.

Hope this helps...

Cheers

Kolusu

#4: Author: rintu,

Posted: Thu Mar 02, 2006 7:57 pm
—
This is part of a commit/restart logic. The STOPAFT will have the count of committed rows. The SORTIN file will need to be cleaned up to get rid of the rows after the committed number of rows.

Inaccurate results, in the sense for eg:
SORTIN = 94543210 records (FB 596 bytes)
STOPAFT= 91543210
sortout = 5010900 (while we expect 91543210 records)

#5: Author: Phantom, Location: The Blue Planet

Posted: Fri Mar 03, 2006 12:39 am
—
Rintu,

If this was your Sort card

Code:

//SYSIN DD *
OPTION COPY,STOPAFT=91543210
/*

Then I don't think there will any problem. But if you have any other statements in your Control card, then those may result in confusion. Could you please post your complete Sort Step, so that we can see what might have gone wrong.

And one more point. I don't seem to understand the Commit/Restart logic that you were mentioning.

Code:

This is part of a commit/restart logic. The STOPAFT will have the count of committed rows. The SORTIN file will need to be cleaned up to get rid of the rows after the committed number of rows.

Normally, we discard all committed records from the input file and process only those which are not committed. In that case, you need to skip Committed records - meaning using OUTFIL STARTREC=xxxxxx instead of OPTION STOPAFT. can you pls explain your process a bit.

Thanks,
Phantom

#6: Author: kolusu, Location: San Jose

Posted: Fri Mar 03, 2006 9:22 am
—

Quote:

Inaccurate results, in the sense for eg:
SORTIN = 94543210 records (FB 596 bytes)
STOPAFT= 91543210
sortout = 5010900 (while we expect 91543210 records)

rintu,

Going with the no: of records I can say it is a multi volume dataset and that might be the reason for inaccurate results.

Anyway this approach for checkpoint-restart logic is very bad.

Kolusu

#7: Author: rintu,

Posted: Thu Mar 09, 2006 8:11 pm
—
Hi,

The problem was related to space and it happens only when there is large volume of data ( in millions). I know, this logic is bad. But this is the standard used throughout the shop.

The STOPAFT is used to cleanup the output files (not input files).

After much discussion, we decided to use different files for sortin and sortout. We could get hold of the author of the process and according to him, years ago when this was implemented, syncsort had okay'd this approach. But now the syncsort manual says this is a big NO-NO !!!

MVSFORUMS.com -> Utilities

output generated using printer-friendly topic mod. All times are GMT - 5 Hours

Page 1 of 1