MVSFORUMS.com Forum Index MVSFORUMS.com
A Community of and for MVS Professionals
 
 FAQFAQ   SearchSearch   Quick Manuals   RegisterRegister 
 ProfileProfile   Log in to check your private messagesLog in to check your private messages   Log inLog in 

Syncsort - sortin/sortout

 
Post new topic   Reply to topic   printer-friendly view    MVSFORUMS.com Forum Index -> Utilities
View previous topic :: View next topic  
Author Message
rintu
Beginner


Joined: 28 Feb 2006
Posts: 7
Topics: 3

PostPosted: Tue Feb 28, 2006 9:39 pm    Post subject: Syncsort - sortin/sortout Reply with quote

Hi,

We have a syncsort which has the same dataset (sequential) as sortin and sortout (both with disp=shr) and control parms as OPTION COPY,STOPAFT=12345678

recently while handling over 10 million records, the result seems to be inaccurate. Any idea whether there is a buffer limit for this type of operation
Back to top
View user's profile Send private message
rintu
Beginner


Joined: 28 Feb 2006
Posts: 7
Topics: 3

PostPosted: Wed Mar 01, 2006 10:20 pm    Post subject: Reply with quote

Shocked no one seems to know the answer - I tried other forums also !!!
Back to top
View user's profile Send private message
kolusu
Site Admin
Site Admin


Joined: 26 Nov 2002
Posts: 12357
Topics: 75
Location: San Jose

PostPosted: Thu Mar 02, 2006 11:20 am    Post subject: Reply with quote

Quote:

We have a syncsort which has the same dataset (sequential) as sortin and sortout (both with disp=shr) and control parms as OPTION COPY,STOPAFT=12345678


You are using the same input as output. If for somereason your job fails you will loose the input dataset.

Quote:

recently while handling over 10 million records,the result seems to be inaccurate. Any idea whether there is a buffer limit for this type of operation


What kind of inaccurate results are you getting? If your file has only 10 million records and your control cards say to stop after copying 12 million 345 thousand and 678 records, which is greater than the no: of input records , your file will have all the 10 million records.

Hope this helps...

Cheers

Kolusu
_________________
Kolusu - DFSORT Development Team (IBM)
DFSORT is on the Web at:
www.ibm.com/storage/dfsort

www.linkedin.com/in/kolusu
Back to top
View user's profile Send private message Send e-mail Visit poster's website
rintu
Beginner


Joined: 28 Feb 2006
Posts: 7
Topics: 3

PostPosted: Thu Mar 02, 2006 7:57 pm    Post subject: Reply with quote

This is part of a commit/restart logic. The STOPAFT will have the count of committed rows. The SORTIN file will need to be cleaned up to get rid of the rows after the committed number of rows.

Inaccurate results, in the sense for eg:
SORTIN = 94543210 records (FB 596 bytes)
STOPAFT= 91543210
sortout = 5010900 (while we expect 91543210 records)
Back to top
View user's profile Send private message
Phantom
Data Mgmt Moderator
Data Mgmt Moderator


Joined: 07 Jan 2003
Posts: 1056
Topics: 91
Location: The Blue Planet

PostPosted: Fri Mar 03, 2006 12:39 am    Post subject: Reply with quote

Rintu,

If this was your Sort card
Code:

//SYSIN   DD  *
  OPTION COPY,STOPAFT=91543210
/*


Then I don't think there will any problem. But if you have any other statements in your Control card, then those may result in confusion. Could you please post your complete Sort Step, so that we can see what might have gone wrong.

And one more point. I don't seem to understand the Commit/Restart logic that you were mentioning.

Code:

This is part of a commit/restart logic. The STOPAFT will have the count of committed rows. The SORTIN file will need to be cleaned up to get rid of the rows after the committed number of rows.


Normally, we discard all committed records from the input file and process only those which are not committed. In that case, you need to skip Committed records - meaning using OUTFIL STARTREC=xxxxxx instead of OPTION STOPAFT. can you pls explain your process a bit.

Thanks,
Phantom
Back to top
View user's profile Send private message
kolusu
Site Admin
Site Admin


Joined: 26 Nov 2002
Posts: 12357
Topics: 75
Location: San Jose

PostPosted: Fri Mar 03, 2006 9:22 am    Post subject: Reply with quote

Quote:

Inaccurate results, in the sense for eg:
SORTIN = 94543210 records (FB 596 bytes)
STOPAFT= 91543210
sortout = 5010900 (while we expect 91543210 records)

rintu,

Going with the no: of records I can say it is a multi volume dataset and that might be the reason for inaccurate results.

Anyway this approach for checkpoint-restart logic is very bad.

Kolusu
_________________
Kolusu - DFSORT Development Team (IBM)
DFSORT is on the Web at:
www.ibm.com/storage/dfsort

www.linkedin.com/in/kolusu
Back to top
View user's profile Send private message Send e-mail Visit poster's website
rintu
Beginner


Joined: 28 Feb 2006
Posts: 7
Topics: 3

PostPosted: Thu Mar 09, 2006 8:11 pm    Post subject: Reply with quote

Hi,

The problem was related to space and it happens only when there is large volume of data ( in millions). I know, this logic is bad. But this is the standard used throughout the shop.

The STOPAFT is used to cleanup the output files (not input files).

After much discussion, we decided to use different files for sortin and sortout. We could get hold of the author of the process and according to him, years ago when this was implemented, syncsort had okay'd this approach. But now the syncsort manual says this is a big NO-NO !!!
Back to top
View user's profile Send private message
Display posts from previous:   
Post new topic   Reply to topic   printer-friendly view    MVSFORUMS.com Forum Index -> Utilities All times are GMT - 5 Hours
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum


MVSFORUMS
Powered by phpBB © 2001, 2005 phpBB Group