MVSFORUMS.com Forum Index MVSFORUMS.com
A Community of and for MVS Professionals
 
 FAQFAQ   SearchSearch   Quick Manuals   RegisterRegister 
 ProfileProfile   Log in to check your private messagesLog in to check your private messages   Log inLog in 

Sorting of a file.

 
Post new topic   Reply to topic   printer-friendly view    MVSFORUMS.com Forum Index -> Job Control Language(JCL)
View previous topic :: View next topic  
Author Message
krk123
Beginner


Joined: 03 Jun 2003
Posts: 58
Topics: 19
PostPosted: Mon Jul 28, 2003 2:08 pm    Post subject: Sorting of a file. Reply with quote
Hi All,
I have a file with 200,000 records. Is there any way I can sort the file in to different files based on the data in positions 1:to 5:. i.e.

for ex:

11111......................
11111................
22221................
22221...........
33322...........
33322.........
33322.................
44444...........
44444...........
44444...........
44444...........
55555..........
55555................

I want all the 11111 records to go to file 1. 22221 records to file 2 etc...I am not sure of all the combinations that can occur in the positions 1: to 5:.

Thanks a lot.
Back to top
View user's profile Send private message
kolusu
Site Admin
Site Admin


Joined: 26 Nov 2002
Posts: 12378
Topics: 75
Location: San Jose
PostPosted: Mon Jul 28, 2003 5:17 pm    Post subject: Reply with quote
krk123,

Try this.

Code:

/STEP0100 EXEC PGM=SORT   
//*                       
//SYSOUT    DD SYSOUT=*   
//SORTIN    DD DSN=YOUR INPUT FILE,
//             DISP=SHR           
//SORTOF01  DD DSN=YOUR FIRST FILE,
//             DISP=(NEW,CATLG,DELETE),
//             UNIT=SYSDA,
//             SPACE=(CYL,(X,Y),RLSE)
//SORTOF02  DD DSN=YOUR SECOND FILE,
//             DISP=(NEW,CATLG,DELETE),
//             UNIT=SYSDA,
//             SPACE=(CYL,(X,Y),RLSE)
//SORTOF03  DD DSN=YOUR THIRD FILE,
//             DISP=(NEW,CATLG,DELETE),
//             UNIT=SYSDA,
//             SPACE=(CYL,(X,Y),RLSE)
//SORTOF04  DD DSN=YOUR FOURTH FILE,
//             DISP=(NEW,CATLG,DELETE),
//             UNIT=SYSDA,
//             SPACE=(CYL,(X,Y),RLSE)
//SORTOF05  DD DSN=YOUR FIFTH FILE,
//             DISP=(NEW,CATLG,DELETE),
//             UNIT=SYSDA,
//             SPACE=(CYL,(X,Y),RLSE)
//SYSIN    DD  *                                   
 SORT FIELDS=COPY                                   
 OUTFIL FILES=01,INCLUDE=(1,5,SS,EQ,C'11111')       
 OUTFIL FILES=02,INCLUDE=(1,5,SS,EQ,C'22222')       
 OUTFIL FILES=03,INCLUDE=(1,5,SS,EQ,C'33333')       
 OUTFIL FILES=04,INCLUDE=(1,5,SS,EQ,C'44444')       
 OUTFIL FILES=05,INCLUDE=(1,5,SS,EQ,C'55555')       
//*
Back to top
View user's profile Send private message Send e-mail Visit poster's website
Frank Yaeger
Sort Forum Moderator
Sort Forum Moderator


Joined: 02 Dec 2002
Posts: 1618
Topics: 31
Location: San Jose
PostPosted: Mon Jul 28, 2003 7:44 pm    Post subject: Reply with quote
Kolusu,

Since the field has 5 bytes and the constant has 5 bytes, you INCLUDE parameter of:

INCLUDE=(1,5,SS,EQ,C'11111')

is equivalent to:

INCLUDE=(1,5,CH,EQ,C'11111')

Not sure what you're trying to do here, but your INCLUDE parameters get you the '11111' records in file1, the '44444' records in file4 and the '55555' records in file5. It doesn't get the '22221' records in file2 or the '33322' records in file3.

Anyway, I think the idea is to switch to a new file every time the key changes. I don't know of any reasonable way to do that unless we know the actual values of the keys (which krk123 said wasn't the case), and we can hardcode the DDs (which krk123 may or may not want).
_________________
Frank Yaeger - DFSORT Development Team (IBM)
Specialties: JOINKEYS, FINDREP, WHEN=GROUP, ICETOOL, Symbols, Migration
DFSORT is on the Web at:
www.ibm.com/storage/dfsort
Back to top
View user's profile Send private message Send e-mail Visit poster's website
krk123
Beginner


Joined: 03 Jun 2003
Posts: 58
Topics: 19
PostPosted: Tue Jul 29, 2003 8:45 am    Post subject: Reply with quote
Thanks a lot Kolusu and Frank. As I dont know the different combinations that can occur in the key field, I dont think there is any other way of doing this.

Thanks a lot for your time and your help on this.
Back to top
View user's profile Send private message
krk123
Beginner


Joined: 03 Jun 2003
Posts: 58
Topics: 19
PostPosted: Tue Jul 29, 2003 10:20 am    Post subject: Reply with quote
Hi Kolusu, Frank,
what is the maximum outfiles we can use. I.e how many sortof files we can use...(SORTOF01, sortof02 ...how far we can go)
Is there a limit?
Thanks,
krk
Back to top
View user's profile Send private message
Frank Yaeger
Sort Forum Moderator
Sort Forum Moderator


Joined: 02 Dec 2002
Posts: 1618
Topics: 31
Location: San Jose
PostPosted: Tue Jul 29, 2003 11:05 am    Post subject: Reply with quote
For DFSORT, there's no practical limit to the number of OUTFIL data sets you can use as long as you give DFSORT enough storage for the buffers. I just did a test using 1000 DDs and OUTFIL statements as shown below, with REGION=256M, and it ran successfully (I didn't try more than 1000, so I don't know what the actual limit is):

Code:

//OUT0001 DD UNIT=SYSDA,SPACE=(CYL,(1,1)),DISP=(,PASS)
...
//OUT1000 DD UNIT=SYSDA,SPACE=(CYL,(1,1)),DISP=(,PASS)
//SYSIN    DD    *                                 
  OPTION COPY                                       
  OUTFIL FNAMES=OUT0001,INCLUDE=(1,5,CH,EQ,C'0001')
  ...
  OUTFIL FNAMES=OUT1000,INCLUDE=(1,5,CH,EQ,C'1000')
/*

_________________
Frank Yaeger - DFSORT Development Team (IBM)
Specialties: JOINKEYS, FINDREP, WHEN=GROUP, ICETOOL, Symbols, Migration
DFSORT is on the Web at:
www.ibm.com/storage/dfsort
Back to top
View user's profile Send private message Send e-mail Visit poster's website
krk123
Beginner


Joined: 03 Jun 2003
Posts: 58
Topics: 19
PostPosted: Tue Jul 29, 2003 11:48 am    Post subject: Reply with quote
Thanks Frank.
Back to top
View user's profile Send private message
kolusu
Site Admin
Site Admin


Joined: 26 Nov 2002
Posts: 12378
Topics: 75
Location: San Jose
PostPosted: Tue Jul 29, 2003 2:37 pm    Post subject: Reply with quote
Krk123,

sorry about my previous post. I over looked your input and suggested a solution which frank was kind enough to correct me. Thanks frank.I was in a hurry when I posted that solution. Here is another approach for your problem.

We first take the input file and sum sort it on the first 5 bytes to get the unique keys. From these unique keys we write dynamic control cards and file allocations and submit a job to the INTRDR.I assumed that you will have a max of 500 unique keys at any give time.If you have more than 500 , they will all be stored in a seperate dataset which you can process as if it is the input file.

Code:

//STEP0100 EXEC PGM=SORT 
//*                       
//SYSOUT    DD SYSOUT=*   
//SORTIN    DD DSN=YOUR INPUT FILE,
//             DISP=SHR
//CTL1      DD DSN=&C1,DISP=(,PASS),SPACE=(CYL,(3,1),RLSE)
//CTL2      DD DSN=&C2,DISP=(,PASS),SPACE=(CYL,(3,1),RLSE)
//CTL3      DD DSN=&C3,DISP=(,PASS),SPACE=(CYL,(3,1),RLSE)
//CTL4      DD DSN=&C4,DISP=(,PASS),SPACE=(CYL,(3,1),RLSE)
//CTL5      DD DSN=&C5,DISP=(,PASS),SPACE=(CYL,(3,1),RLSE)
//DD1       DD DSN=&D1,DISP=(,PASS),SPACE=(CYL,(3,1),RLSE)
//DD2       DD DSN=&D2,DISP=(,PASS),SPACE=(CYL,(3,1),RLSE)
//DD3       DD DSN=&D3,DISP=(,PASS),SPACE=(CYL,(3,1),RLSE)
//DD4       DD DSN=&D4,DISP=(,PASS),SPACE=(CYL,(3,1),RLSE)
//DD5       DD DSN=&D5,DISP=(,PASS),SPACE=(CYL,(3,1),RLSE)
//SAVE      DD DSN=YOUR SAVE KEY FILE FOR NEXT RUN,
//             DISP=(NEW,CATLG,DELETE),
//             UNIT=SYSDA,
//             SPACE=(CYL,(X,Y),RLSE)
//SYSIN    DD  *                                                     
 SORT FIELDS=(1,5,CH,A)                                               
 SUM FIELDS=NONE                                                     
 OUTFIL FNAMES=CTL1,STARTREC=1,ENDREC=100,                           
 OUTREC=(C' OUTFIL FNAMES=FILE',SEQNUM,3,ZD,C',INCLUDE=(1,5,CH,EQ,C',
         X'7D',1,5,X'7D',C')',80:X)                                   
 OUTFIL FNAMES=CTL2,STARTREC=101,ENDREC=200,                         
 OUTREC=(C' OUTFIL FNAMES=FILE',SEQNUM,3,ZD,START=101,               
         C',INCLUDE=(1,5,CH,EQ,C',X'7D',1,5,X'7D',C')',80:X)         
 OUTFIL FNAMES=CTL3,STARTREC=201,ENDREC=300,                         
 OUTREC=(C' OUTFIL FNAMES=FILE',SEQNUM,3,ZD,START=201,               
         C',INCLUDE=(1,5,CH,EQ,C',X'7D',1,5,X'7D',C')',80:X)         
 OUTFIL FNAMES=CTL4,STARTREC=301,ENDREC=400,                         
 OUTREC=(C' OUTFIL FNAMES=FILE',SEQNUM,3,ZD,START=301,               
         C',INCLUDE=(1,5,CH,EQ,C',X'7D',1,5,X'7D',C')',80:X)         
 OUTFIL FNAMES=CTL5,STARTREC=401,ENDREC=500,                         
 OUTREC=(C' OUTFIL FNAMES=FILE',SEQNUM,3,ZD,START=401,               
         C',INCLUDE=(1,5,CH,EQ,C',X'7D',1,5,X'7D',C')',80:X)         
 OUTFIL FNAMES=SAVE,STARTREC=501                                     
 OUTFIL FNAMES=DD1,STARTREC=1,ENDREC=100,                             
 OUTREC=(C'//FILE',SEQNUM,3,ZD,C' DD DSN=OUTPUT.FILE',               
         SEQNUM,3,ZD,C',',/,                                         
         C'//',12X,C'DISP=(NEW,CATLG,DELETE),',/,                     
         C'//',12X,C'UNIT=SYSDA,',/,                                 
         C'//',12X,C'SPACE=(CYL,(X,Y),RLSE)',80:X)                   
 OUTFIL FNAMES=DD2,STARTREC=101,ENDREC=200,                           
 OUTREC=(C'//FILE',SEQNUM,3,ZD,START=101,                             
         C' DD DSN=OUTPUT.FILE',SEQNUM,3,ZD,START=101,C',',/,         
         C'//',12X,C'DISP=(NEW,CATLG,DELETE),',/,                     
         C'//',12X,C'UNIT=SYSDA,',/,                                 
         C'//',12X,C'SPACE=(CYL,(X,Y),RLSE)',80:X)                   
 OUTFIL FNAMES=DD3,STARTREC=201,ENDREC=300,                           
 OUTREC=(C'//FILE',SEQNUM,3,ZD,START=201,                             
         C' DD DSN=OUTPUT.FILE',SEQNUM,3,ZD,START=201,C',',/,         
         C'//',12X,C'DISP=(NEW,CATLG,DELETE),',/,                     
         C'//',12X,C'UNIT=SYSDA,',/,                                 
         C'//',12X,C'SPACE=(CYL,(X,Y),RLSE)',80:X)                   
 OUTFIL FNAMES=DD4,STARTREC=301,ENDREC=400,                   
 OUTREC=(C'//FILE',SEQNUM,3,ZD,START=301,                     
         C' DD DSN=OUTPUT.FILE',SEQNUM,3,ZD,START=301,C',',/, 
         C'//',12X,C'DISP=(NEW,CATLG,DELETE),',/,             
         C'//',12X,C'UNIT=SYSDA,',/,                           
         C'//',12X,C'SPACE=(CYL,(X,Y),RLSE)',80:X)             
 OUTFIL FNAMES=DD5,STARTREC=401,ENDREC=500,                   
 OUTREC=(C'//FILE',SEQNUM,3,ZD,START=401,                     
         C' DD DSN=OUTPUT.FILE',SEQNUM,3,ZD,START=401,C',',/, 
         C'//',12X,C'DISP=(NEW,CATLG,DELETE),',/,             
         C'//',12X,C'UNIT=SYSDA,',/,                           
         C'//',12X,C'SPACE=(CYL,(X,Y),RLSE)',80:X)             
/*                       
//STEP0200 EXEC  PGM=SORT               
//SYSOUT   DD SYSOUT=*                 
//SYSIN    DD *                         
  SORT FIELDS=COPY                     
/*                                     
//SORTOUT  DD SYSOUT=(*,INTRDR)         
//SORTIN   DD DATA,DLM=$$               
//TZZZZAAB JOB 'COPY',                 
//             CLASS=A,                 
//             MSGCLASS=Y,             
//             MSGLEVEL=(1,1),         
//             NOTIFY=&SYSUID           
//*                                     
//STEP0100 EXEC  PGM=SORT               
//*                                     
//SYSOUT   DD SYSOUT=*                 
//SORTIN   DD DSN=YOUR INPUT FILE,     
//            DISP=SHR                 
//SYSIN    DD *                         
 SORT FIELDS=COPY                       
$$                                     
//         DD DSN=&C1,DISP=(OLD,PASS)   
//         DD DSN=&C2,DISP=(OLD,PASS)   
//         DD DSN=&C3,DISP=(OLD,PASS)   
//         DD DSN=&C4,DISP=(OLD,PASS)   
//         DD DSN=&C5,DISP=(OLD,PASS)   
//         DD DSN=&D1,DISP=(OLD,PASS)   
//         DD DSN=&D2,DISP=(OLD,PASS)   
//         DD DSN=&D3,DISP=(OLD,PASS)   
//         DD DSN=&D4,DISP=(OLD,PASS)   
//         DD DSN=&D5,DISP=(OLD,PASS)   
/*                   


The contents of C1 are as follows:

Code:

OUTFIL FNAMES=FILE001,INCLUDE=(1,5,CH,EQ,C'11111')
OUTFIL FNAMES=FILE002,INCLUDE=(1,5,CH,EQ,C'22221')
OUTFIL FNAMES=FILE003,INCLUDE=(1,5,CH,EQ,C'22222')
OUTFIL FNAMES=FILE004,INCLUDE=(1,5,CH,EQ,C'33332')
OUTFIL FNAMES=FILE005,INCLUDE=(1,5,CH,EQ,C'33333')
.....

OUTFIL FNAMES=FILE100,INCLUDE=(1,5,CH,EQ,C'99999')
                         


simlarly the c2 thru c5 will have control cards for file101 thru file500.

The contents of DD1 are as follows:

Code:

//FILE001 DD DSN=OUTPUT.FILE001,       
//            DISP=(NEW,CATLG,DELETE), 
//            UNIT=SYSDA,               
//            SPACE=(CYL,(X,Y),RLSE)   
//FILE002 DD DSN=OUTPUT.FILE002,       
//            DISP=(NEW,CATLG,DELETE), 
//            UNIT=SYSDA,               
//            SPACE=(CYL,(X,Y),RLSE)   
//FILE003 DD DSN=OUTPUT.FILE003,       
//            DISP=(NEW,CATLG,DELETE), 
//            UNIT=SYSDA,               
//            SPACE=(CYL,(X,Y),RLSE)   
//FILE004 DD DSN=OUTPUT.FILE004,       
//            DISP=(NEW,CATLG,DELETE), 
//            UNIT=SYSDA,               
//            SPACE=(CYL,(X,Y),RLSE)   
//FILE005 DD DSN=OUTPUT.FILE005,       
//            DISP=(NEW,CATLG,DELETE), 
//            UNIT=SYSDA,               
//            SPACE=(CYL,(X,Y),RLSE)   
...
//FILE100  DD DSN=OUTPUT.FILE100,       
//            DISP=(NEW,CATLG,DELETE), 
//            UNIT=SYSDA,               
//            SPACE=(CYL,(X,Y),RLSE)   



Similary files DD2 thru DD5 will have file allocations for files101 thru file500

Step0200 in the above jcl will merge all these control card and dynamic allocations files together and submits a job to the INTRDR which will actually split the records into various files.

Hope this helps....

cheers

kolusu
Back to top
View user's profile Send private message Send e-mail Visit poster's website
Frank Yaeger
Sort Forum Moderator
Sort Forum Moderator


Joined: 02 Dec 2002
Posts: 1618
Topics: 31
Location: San Jose
PostPosted: Tue Jul 29, 2003 3:50 pm    Post subject: Reply with quote
Kolusu,

Good idea. But I'm curious why you split the generated control statements into five files instead of just putting them all in one file? Likewise for the generated DD statements?
_________________
Frank Yaeger - DFSORT Development Team (IBM)
Specialties: JOINKEYS, FINDREP, WHEN=GROUP, ICETOOL, Symbols, Migration
DFSORT is on the Web at:
www.ibm.com/storage/dfsort
Back to top
View user's profile Send private message Send e-mail Visit poster's website
kolusu
Site Admin
Site Admin


Joined: 26 Nov 2002
Posts: 12378
Topics: 75
Location: San Jose
PostPosted: Tue Jul 29, 2003 5:12 pm    Post subject: Reply with quote
Frank,

Good catch! I was just afraid of "excess of control cards" error. I also was thinking about the max no: of DD names in a jcl. So splitting into 5 or 6 files can be easy as they can be used to submit multiple jobs at the same time. Ideally I would have to create 5 control cards datasets with 200 control statements and submit 5 jobs to the intrdr to execute simultaneously there by creating 1000 datasets in one go. If krk123 is willing to pursue this solution then I will post the updated solution.

Thanks

Kolusu
Back to top
View user's profile Send private message Send e-mail Visit poster's website
Frank Yaeger
Sort Forum Moderator
Sort Forum Moderator


Joined: 02 Dec 2002
Posts: 1618
Topics: 31
Location: San Jose
PostPosted: Tue Jul 29, 2003 5:49 pm    Post subject: Reply with quote
Kolusu,

I guess the "excess of control cards" error is a Syncsort problem that DFSORT doesn't have. I don't know what the max number of DD names in a JCL is. As I said, I had 1000 DDs in the DFSORT job I tested and that didn't cause a problem, but I didn't run it through the internal reader (maybe that makes a difference?). Anyway, just curious.
_________________
Frank Yaeger - DFSORT Development Team (IBM)
Specialties: JOINKEYS, FINDREP, WHEN=GROUP, ICETOOL, Symbols, Migration
DFSORT is on the Web at:
www.ibm.com/storage/dfsort
Back to top
View user's profile Send private message Send e-mail Visit poster's website
krk123
Beginner


Joined: 03 Jun 2003
Posts: 58
Topics: 19
PostPosted: Wed Jul 30, 2003 1:02 pm    Post subject: Reply with quote
Hi Kolusu & Frank,
here is the version I used. It worked great. To be frank initially it was above my head when kolusu posted. I did a lot of experimenting and reading to understand this. I still have some doubts, but not sure whether I can ask these here as they are kind of basic doubts..

Any ways, thanks a lot for kolusu and frank. I really appreciate your time and help on this.
Code:

//STEP0100 EXEC PGM=SORT                             
//SYSOUT    DD SYSOUT=*                             
//SORTIN    DD DSN=MY input file,     
//             DISP=SHR                             
//CTL1      DD DSN=sysuid.TEST.CTL11,               
//             DISP=(NEW,CATLG,DELETE),             
//             UNIT=SYSDA,                           
//             SPACE=(CYL,(1,1),RLSE)               
//DD1       DD DSN=sysuid.TEST.DD1,                 
//             DISP=(NEW,CATLG,DELETE),             
//             UNIT=SYSDA,                           
//             SPACE=(CYL,(1,1),RLSE)               
//SYSIN    DD  *                                     
    SORT FIELDS=(1,5,CH,A)                           
    SUM FIELDS=NONE                                 
        OUTFIL FNAMES=CTL1,STARTREC=1,ENDREC=100,                         
        OUTREC=(C' OUTFIL FNAMES=FILE',SEQNUM,3,ZD,                       
                 C',INCLUDE=(1,5,CH,EQ,C',X'7D',1,5,X'7D',C')',80:X)     
        OUTFIL FNAMES=DD1,STARTREC=1,ENDREC=100,                         
        OUTREC=(C'//FILE',SEQNUM,3,ZD,C' DD DSN=sysuid.FILE',           
                 SEQNUM,3,ZD,C',',/,                                     
                 C'//',12X,C'DISP=(NEW,CATLG,DELETE),',/,                 
                 C'//',12X,C'UNIT=SYSDA,',/,                             
                 C'//',12X,C'SPACE=(CYL,(1,1),RLSE)',80:X)               
    /*                                                                   
    /*                                                                   
    //STEP0200 EXEC  PGM=SORT                                             
    //SYSOUT   DD SYSOUT=*                                               
    //SYSIN    DD *                                                       
      SORT FIELDS=COPY                                                   
    /*                                                                   
    //SORTOUT  DD SYSOUT=(*,INTRDR)                                       
    //SORTIN   DD DATA,DLM=$$                                             
    //sysuidK JOB (xxx,#xxxxx),'xxx KO',CLASS=R,MSGCLASS=U,           
    //             NOTIFY=sysuid,                                       
    //             USER=sysuid,PASSWORD=                                 
//*                                                     
//STEP0100 EXEC  PGM=SORT                               
//*                                                     
//SYSOUT   DD SYSOUT=*                                   
//SORTIN   DD DSN=my input file,         
//            DISP=SHR                                   
//SYSIN    DD *                                         
  SORT FIELDS=COPY                                       
$$                                                       
//         DD DSN=output.TEST.CTL11,DISP=SHR           
//         DD DSN=output.TEST.DD1,DISP=SHR             
/*
Back to top
View user's profile Send private message
Display posts from previous:   
Post new topic   Reply to topic   printer-friendly view    MVSFORUMS.com Forum Index -> Job Control Language(JCL) All times are GMT - 5 Hours
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum


MVSFORUMS
Powered by phpBB © 2001, 2005 phpBB Group