MVSFORUMS.com Forum Index MVSFORUMS.com
A Community of and for MVS Professionals
 
 FAQFAQ   SearchSearch   Quick Manuals   RegisterRegister 
 ProfileProfile   Log in to check your private messagesLog in to check your private messages   Log inLog in 

Eliminate Dups & Update Trailer count using syncsort

 
Post new topic   Reply to topic   printer-friendly view    MVSFORUMS.com Forum Index -> Utilities
View previous topic :: View next topic  
Author Message
vaasavi
Beginner


Joined: 20 Jun 2005
Posts: 14
Topics: 5

PostPosted: Mon Jun 20, 2005 6:33 am    Post subject: Eliminate Dups & Update Trailer count using syncsort Reply with quote

Hi,
I am using Syncsort.
My requiremnt ,i have a file having header,detail lines and trailer.
Trailer contains the count of detail lines,and other info also which cannot be hardcoded,
My output should be a file with header,detail lines(remove duplicates) and trailer should contain count of detail lines(excluding duplicates)
Ex:


code:
--------------------------------------------------------------------------------

Header ABC Company
abc def xxxxxxxxx
abc def xxxxxxxxx
abc def aaaaaaaaa
abc efg bbbbbbbbb
Trailer 04052005 4(count of detail lines)

--------------------------------------------------------------------------------

My Output should be


code:
--------------------------------------------------------------------------------

Header ABC Company
abc def xxxxxxxxx
abc def aaaaaaaaa
abc efg bbbbbbbbb
Trailer 04052005 3(after removing duplicates)

--------------------------------------------------------------------------------

Thanks,
Vasavi
Back to top
View user's profile Send private message
kolusu
Site Admin
Site Admin


Joined: 26 Nov 2002
Posts: 12377
Topics: 75
Location: San Jose

PostPosted: Mon Jun 20, 2005 7:38 am    Post subject: Reply with quote

Vasavi,

The following JCL will give you the desired results. I assumed that your input file is of FB recfm and is has lrecl of 80. I assumed you have to remove duplicates considering 3 fields which start at pos 1 for 3 bytes and at pos 5 for 3 bytes and at pos 9 for 9 bytes. I also assumed that you header record is recongnized by 'header' in the first 6 bytes and similarly trailer by 'trailer' in the first 7 bytes. We first add a indicator at the end to make sure that header record always stays at the top and the trailer at the bottom.

Now we sort on the the key fields along with the indicator at the end and eliminate dups. But to get the updated count excluding the header record is a little bit tricky. We add a seqnum on OUTREC FIELDS which starts from zero. By doing so, the header record is not counted as part of the records. And we use this on the trailer for getting the total detail records after you are done with eliminating the dups.

Code:

//STEP0100 EXEC PGM=SORT                           
//SYSOUT   DD  SYSOUT=*                           
//SORTIN   DD  *
----+----1----+----2----+----3
HEADER ABC COMPANY                                 
ABC DEF XXXXXXXXX                                 
ABC DEF XXXXXXXXX                                 
ABC DEF AAAAAAAAA                                 
ABC EFG BBBBBBBBB                                 
TRAILER 04052005 4                                 
//SORTOUT  DD  SYSOUT=*                           
//SYSIN    DD  *                                   
  INREC FIELDS=(1,80,                             
                81:1,7,CHANGE=(1,C'HEADER ',C'0', 
                                 C'TRAILER',C'9'),
                               NOMATCH=(C'1'))     
  SORT FIELDS=(81,01,CH,A,                         
               01,03,CH,A,                         
               05,03,CH,A,                         
               09,09,CH,A)                         
  SUM FIELDS=NONE                                 
  OUTREC FIELDS=(1,80,SEQNUM,8,ZD,START=0)         
  OUTFIL REMOVECC,OMIT=(1,7,CH,EQ,C'TRAILER'),     
  OUTREC=(1,80),                                   
  TRAILER1=(C'TRAILER ',DATENS=(MD4),X,81,8)       
/*                                                 


Hope this helps...

Cheers

Kolusu
_________________
Kolusu
www.linkedin.com/in/kolusu
Back to top
View user's profile Send private message Send e-mail Visit poster's website
vaasavi
Beginner


Joined: 20 Jun 2005
Posts: 14
Topics: 5

PostPosted: Mon Jun 20, 2005 8:41 am    Post subject: Reply with quote

Hi Kolusu,

My trailer doesn't have any 'TRAILER' word to represent as trailer.
My Trailer looks like
<DATE> <COUNT>
Date is not current date,its date from file so i cna't even write it as constant.
Ex:
040505 003

040505 is value from a file which can't be tretaed as constant.

Hope so requirement is clear now?
Back to top
View user's profile Send private message
kolusu
Site Admin
Site Admin


Joined: 26 Nov 2002
Posts: 12377
Topics: 75
Location: San Jose

PostPosted: Mon Jun 20, 2005 11:54 am    Post subject: Reply with quote

vasavi,

The following JCL will give you the desired results. Since there is no indicator to reprenst the trailer record, we need an additional pass.

A brief explanation of the job. we first add a seqnum at the end of all records and sort on that seqnum descending. By doing so, we have the trailer record as the first record.

Now we split the file into 3 files, namely TRL, DTL and HDR files.

TRL file will be symbol file which will be used in the next step.

The HDR file will just have the header record and DTL file will have all the detail records.

The next step we concatenate the HDR and DTL file and sum sort on the key fields to eliminate the duplicates and use the symbol to update the trailer count.

Code:

//STEP0100 EXEC PGM=SORT                                 
//SYSOUT   DD SYSOUT=*                                   
//SORTIN   DD *                                           
HEADER ABC COMPANY                                       
ABC DEF XXXXXXXXX                                         
ABC DEF XXXXXXXXX                                         
ABC DEF AAAAAAAAA                                         
ABC EFG BBBBBBBBB                                         
04052005 4                                               
//HDR      DD DSN=&HDR,DISP=(,PASS),SPACE=(TRK,(1,1),RLSE)
//DTL      DD DSN=&DTL,DISP=(,PASS),SPACE=(CYL,(X,Y),RLSE)
//TRL      DD DSN=&TRL,DISP=(,PASS),SPACE=(TRK,(1,1),RLSE)
//SYSIN    DD *                                           
  OPTION EQUALS
  INREC FIELDS=(1,80,                                     
                SEQNUM,8,ZD)                             
  SORT FIELDS=(81,8,CH,D)                                 
  OUTREC FIELDS=(1,88,SEQNUM,8,ZD)                       
  OUTFIL FNAMES=TRL,REMOVECC,NODETAIL,                   
  OUTREC=(1,80),                                         
  HEADER1=('TRLDATE,''',1,8,C'''',80:X)                   
  OUTFIL FNAMES=HDR,INCLUDE=(1,6,CH,EQ,C'HEADER'),       
  OUTREC=(1,80,8C'0')                                     
  OUTFIL FNAMES=DTL,OMIT=(81,8,ZD,EQ,1,OR,89,8,ZD,EQ,1), 
  OUTREC=(1,80,7C'0',C'1')                               
/*   
//STEP0200 EXEC PGM=SORT                         
//SYSOUT   DD SYSOUT=*                           
//SYMNAMES DD DSN=&TRL,DISP=OLD                   
//SORTIN   DD DSN=&HDR,DISP=OLD                   
//         DD DSN=&DTL,DISP=OLD                   
//SORTOUT  DD SYSOUT=*                           
//SYSIN    DD *                                   
  OPTION EQUALS
  SORT FIELDS=(81,08,CH,A,                       
               01,03,CH,A,                       
               05,03,CH,A,                       
               09,09,CH,A)                       
  SUM FIELDS=NONE                                 
  OUTFIL REMOVECC,                               
  TRAILER1=(TRLDATE,TOT=(81,8,ZD,EDIT=(IIIIIIIT)))
/*


Hope this helps...

Cheers

kolusu
_________________
Kolusu
www.linkedin.com/in/kolusu
Back to top
View user's profile Send private message Send e-mail Visit poster's website
vaasavi
Beginner


Joined: 20 Jun 2005
Posts: 14
Topics: 5

PostPosted: Tue Jun 21, 2005 2:27 am    Post subject: Reply with quote

Thanks kolusu,its working
Back to top
View user's profile Send private message
vaasavi
Beginner


Joined: 20 Jun 2005
Posts: 14
Topics: 5

PostPosted: Wed Jun 22, 2005 1:35 am    Post subject: Reply with quote

Hi Kolusu,

Can you provide me material on Syncsort please
Back to top
View user's profile Send private message
kolusu
Site Admin
Site Admin


Joined: 26 Nov 2002
Posts: 12377
Topics: 75
Location: San Jose

PostPosted: Wed Jun 22, 2005 4:36 am    Post subject: Reply with quote

vaasavi,

Syncsort manuals are copyrighted and hence are not available online. If your shop has a valid licence you can contact your sys admin for a copy of the manual

Hope this helps...

Cheers

Kolusu
_________________
Kolusu
www.linkedin.com/in/kolusu
Back to top
View user's profile Send private message Send e-mail Visit poster's website
vaasavi
Beginner


Joined: 20 Jun 2005
Posts: 14
Topics: 5

PostPosted: Mon Jun 27, 2005 4:34 am    Post subject: Reply with quote

Hi Kolusu,
Your JCL is woring fine for datasets with lrecl=80,if i am trying to use same jcl for input file with lrecl=200,it's not working,it is getting abended.
jcl which i used is
Code:

//STEP0100 EXEC PGM=SORT                                             
//SYSOUT   DD SYSOUT=*                                               
//SORTIN   DD DSN=AP1T.abcdT.INPUT,DISP=SHR                       
//HDR      DD DSN=AP1T.HDR,DISP=(,PASS),SPACE=(TRK,(1,1),RLSE)       
//DTL      DD DSN=AP1T.DTL,DISP=(,PASS),SPACE=(CYL,(1,1),RLSE)       
//TRL      DD DSN=AP1T.TRL1,DISP=(,PASS),SPACE=(CYL,(1,1),RLSe)
//SYSIN    DD *                                                     
  OPTION EQUALS                                                     
  INREC FIELDS=(1,200,SEQNUM,8,ZD)                                   
  SORT FIELDS=(201,8,CH,A)                                           
  OUTFIL FNAMES=TRL,REMOVECC,NODETAIL,                               
  OUTREC=(1,200),                                                   
  TRAILER1=('TRLDATE,''',2,8,C'''',200:X)                           
  OUTFIL FNAMES=HDR,INCLUDE=(1,1,CH,EQ,C'H'),                       
  OUTREC=(1,200,8C'0')                                               
  OUTFIL FNAMES=DTL,INCLUDE=(1,1,CH,EQ,C'D'),                       
  OUTREC=(1,200,7C'0',C'1')   
//STEP0200 EXEC PGM=SORT                                       
//SYSOUT   DD SYSOUT=*                                         
//SYMNAMES DD DSN=AP1T.TRL1,DISP=old                           
//SORTIN   DD DSN=AP1T.HDR,DISP=OLD                             
//         DD DSN=AP1T.DTL,DISP=OLD                             
//SORTOUT  DD DSN=AP1T.SORT.PRU.OUTPUT,DISP=(NEW,CATLG,CATLG), 
//            DCB=(LRECL=208,RECFM=FB),                         
//            SPACE=(CYL,(5,3),RLSE),UNIT=SYSDA                 
//SYSIN    DD *                                                 
 OPTION EQUALS                                                 
   SORT FIELDS=(201,08,CH,A)                                   
   OUTFIL REMOVECC,                                             
     TRAILER1=(1:C'T',2:TRLDATE,TOT=(201,8,ZD,EDIT=(TTTTTTTT)))   

Its working perfectly for lrecl=80,why it is not working for lrecl=200
Will SYMNAMES work only for lrecl=80?
Back to top
View user's profile Send private message
kolusu
Site Admin
Site Admin


Joined: 26 Nov 2002
Posts: 12377
Topics: 75
Location: San Jose

PostPosted: Mon Jun 27, 2005 5:05 am    Post subject: Reply with quote

vaasavi,

Your JCL has many errors.You are adding a seqnum and sorting on it in ASCending sequence which will be the same as input.

Since you can identify the header(H) and detail(D) records you really don't need a seqnum.You are right about SYMNAMES DD . It must have the following attributes: RECFM=F or RECFM=FB and LRECL=80.

You are not eliminating any dups, then I wonder what is the purpose of your JCL?

It is always a good idea to understand the JCL and change it to your needs instead of simply copying it.

From your JCL I understood that you have a file with the following attributes

LRECL=200, RECFM=FB

The header record can be identified with 'h' in the first byte and the detail records can be identified with 'd' in the first byte.

Now where are your key fields that are considered to dups?

Kolusu
_________________
Kolusu
www.linkedin.com/in/kolusu
Back to top
View user's profile Send private message Send e-mail Visit poster's website
vaasavi
Beginner


Joined: 20 Jun 2005
Posts: 14
Topics: 5

PostPosted: Tue Jun 28, 2005 9:21 am    Post subject: Reply with quote

Hi kolusu,
i have not given the whole jcl.thanks for your reply..
Back to top
View user's profile Send private message
kolusu
Site Admin
Site Admin


Joined: 26 Nov 2002
Posts: 12377
Topics: 75
Location: San Jose

PostPosted: Tue Jun 28, 2005 10:08 am    Post subject: Reply with quote

Quote:

i have not given the whole jcl.thanks for your reply..


vaasavi,

hmm you have solved your problem?

Kolusu
_________________
Kolusu
www.linkedin.com/in/kolusu
Back to top
View user's profile Send private message Send e-mail Visit poster's website
vaasavi
Beginner


Joined: 20 Jun 2005
Posts: 14
Topics: 5

PostPosted: Wed Jun 29, 2005 3:21 am    Post subject: Reply with quote

ya kolusu ,my problem is solved.thanks for your help..
Back to top
View user's profile Send private message
Display posts from previous:   
Post new topic   Reply to topic   printer-friendly view    MVSFORUMS.com Forum Index -> Utilities All times are GMT - 5 Hours
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum


MVSFORUMS
Powered by phpBB © 2001, 2005 phpBB Group