MVSFORUMS.com Forum Index MVSFORUMS.com
A Community of and for MVS Professionals
 
 FAQFAQ   SearchSearch   Quick Manuals   RegisterRegister 
 ProfileProfile   Log in to check your private messagesLog in to check your private messages   Log inLog in 

Inserting a header records while merging two different files
Goto page 1, 2  Next
 
Post new topic   Reply to topic   printer-friendly view    MVSFORUMS.com Forum Index -> Utilities
View previous topic :: View next topic  
Author Message
Phantom
Data Mgmt Moderator
Data Mgmt Moderator


Joined: 07 Jan 2003
Posts: 1056
Topics: 91
Location: The Blue Planet

PostPosted: Thu Mar 18, 2004 2:05 am    Post subject: Inserting a header records while merging two different files Reply with quote

I have two or more files with me that needs to be merged into one. But before merging, I need to insert a header record before each file to differentiate the records from two different files. Please see below.

Code:

OUTPUT:
HEADER RECORD: FILE 1 STARTS HERE
<data of file 1>
HEADER RECORD: FILE 2 STARTS HERE
<data of file 2>
HEADER RECORD: FILE 3 STARTS HERE
<data of file 3>
....


Please let me know how I can accomplish this.....

Its an emergency....I didn't go through all the posts in the forum....so please forgive me if anyone has already posted question...
Back to top
View user's profile Send private message
Phantom
Data Mgmt Moderator
Data Mgmt Moderator


Joined: 07 Jan 2003
Posts: 1056
Topics: 91
Location: The Blue Planet

PostPosted: Thu Mar 18, 2004 6:09 am    Post subject: Reply with quote

Thanks Ravi, But the solution does not serve my purpose.

The output comes like this:
Code:

<data of file 1>
<data of file 2>
<data of file 3>
HEADER RECORD 1
HEADER RECORD 2
HEADER RECORD 3


But I need the output as shown below.
Code:

HEADER RECORD 1
<data of file 1>
HEADER RECORD 2
<data of file 2>
HEADER RECORD 3
<data of file 3>


Meaning, the header records should be used as separators between two different files.
Back to top
View user's profile Send private message
Phantom
Data Mgmt Moderator
Data Mgmt Moderator


Joined: 07 Jan 2003
Posts: 1056
Topics: 91
Location: The Blue Planet

PostPosted: Thu Mar 18, 2004 6:11 am    Post subject: Reply with quote

I tried to code something like the one shown below. Can someone help me in fine tuning the code. I would like to eliminate the Temporaray file (T2) from my code.

Code:

//R002     EXEC PGM=IEBDG                                               
//SYSPRINT DD  SYSOUT=*                                                 
//OUTDD    DD  DSN=&T2,DISP=(,PASS),                                   
//            SPACE=(80,(1,1),RLSE),                                   
//            DCB=(RECFM=FB,LRECL=80)                                   
//SYSIN    DD  *                                                       
 DSD     OUTPUT=(OUTDD)                                                 
 FD      NAME=FILLER1,LENGTH=80,FORMAT=AN                               
 CREATE  QUANTITY=1                                                     
 END                                                                   
/*                                                                     
//*                                                                     
//R010  EXEC PGM=SYNCTOOL                                               
//TOOLMSG  DD SYSOUT=*                                                 
//DFSMSG   DD SYSOUT=*                                                 
//IFILE1   DD DSN=TSONJAY.CRD.MERGE.TEST1,DISP=SHR                     
//IFILE2   DD DSN=TSONJAY.CRD.MERGE.TEST2,DISP=SHR                     
//IFILE3   DD DSN=TSONJAY.CRD.MERGE.TEST3,DISP=SHR                     
//T1       DD DSN=&T2,DISP=SHR                                         
//OFILE    DD DSN=TSONJAY.CRD.MERGE.OUTPUT,DISP=MOD                     
//TOOLIN   DD *                                             
  COPY FROM(T1) USING(CTLA)                                 
  COPY FROM(IFILE1) USING(CTL1)                             
  COPY FROM(T1) USING(CTLB)                                 
  COPY FROM(IFILE2) USING(CTL1)                             
  COPY FROM(T1) USING(CTLC)                                 
  COPY FROM(IFILE3) USING(CTL1)                             
//CTL1CNTL DD *                                             
  OUTFIL FNAMES=OFILE,OUTREC=(1,80)                         
//CTLACNTL DD *                                             
  OUTFIL FNAMES=OFILE,                                     
  OUTREC=(C'HEADER RECORD: FILE1 STARTS HERE',48C' ')       
//CTLBCNTL DD *                                             
  OUTFIL FNAMES=OFILE,                                     
  OUTREC=(C'HEADER RECORD: FILE2 STARTS HERE',48C' ')       
//CTLCCNTL DD *                                             
  OUTFIL FNAMES=OFILE,                                     
  OUTREC=(C'HEADER RECORD: FILE3 STARTS HERE',48C' ')       
/*                                                         


Thanks a lot...
Back to top
View user's profile Send private message
kolusu
Site Admin
Site Admin


Joined: 26 Nov 2002
Posts: 12378
Topics: 75
Location: San Jose

PostPosted: Thu Mar 18, 2004 6:28 am    Post subject: Reply with quote

phantom,

Did you try running the job shown by ravi? It will indeed give you the desired results. All you need to change in that Job is the SYSPRINT DD statement. SORT requires SYSOUT DD SYSOUT=*

Code:

//STEP0100 EXEC PGM=SORT
//SYSOUT   DD  SYSOUT=*
//SORTIN   DD *
HEADER RECORD: FILE1 STARTS HERE
//         DD DSN=TSONJAY.CRD.MERGE.TEST1,DISP=SHR
//         DD *
HEADER RECORD: FILE2 STARTS HERE
//         DD DSN=TSONJAY.CRD.MERGE.TEST2,DISP=SHR
//         DD *
HEADER RECORD: FILE3 STARTS HERE
//         DD DSN=TSONJAY.CRD.MERGE.TEST3,DISP=SHR
//SORTOUT  DD DSN=YOUR OUTPUT DSN,
//            DISP=(NEW,CATLG,DELETE),
//            UNIT=SYSDA,
//            SPACE=(CYL,(1,1),RLSE)
//SYSIN    DD *
  SORT FIELDS=COPY
/*


Hope this helps...

Cheers

Kolusu
_________________
Kolusu
www.linkedin.com/in/kolusu
Back to top
View user's profile Send private message Send e-mail Visit poster's website
Phantom
Data Mgmt Moderator
Data Mgmt Moderator


Joined: 07 Jan 2003
Posts: 1056
Topics: 91
Location: The Blue Planet

PostPosted: Thu Mar 18, 2004 7:15 am    Post subject: Reply with quote

Thanks Kolusu,

The job which Ravi gave me had all the header lines clubbed together. So, they were copied at the end. Your solution will work fine as a Instream JCL. When I invoke the job as a Job-Proc, then I need to have 3 different controlcards / flat files containing the header lines. This is similar to storing the header lines into some flat files and concatenating them while u merge.

I am looking for a solution which requires only one SORT CONTROL CARD and NO FLAT files to hold the header lines. I am sorry for not making this clear in my initial request. Is there any command in SORT or some other utility which can include the header lines in between two different files.

Please let me know whether this could be accomplished.
Back to top
View user's profile Send private message
Phantom
Data Mgmt Moderator
Data Mgmt Moderator


Joined: 07 Jan 2003
Posts: 1056
Topics: 91
Location: The Blue Planet

PostPosted: Thu Mar 18, 2004 7:23 am    Post subject: Reply with quote

Ravi,

I was a little confused... In your solution I thought AAAAA, BBBB....were the header lines that I was talking about. Sorry about that...But still, as I have said, my requirement is to accomplish using only one SORT Control card. Also, I donot want to store the header lines into any flat files. I want to hardcode them somewhere in the sort control card and when the job executes the header lines should be added before each file. Thanks for your help and sorry for confusing.
Back to top
View user's profile Send private message
kolusu
Site Admin
Site Admin


Joined: 26 Nov 2002
Posts: 12378
Topics: 75
Location: San Jose

PostPosted: Thu Mar 18, 2004 8:33 am    Post subject: Reply with quote

Ravi,

Yes you are right about sysout being optional. But I suggest having sysout as it will provide more information regarding your syntax errors as well as sort statistics , like no: of records copied and sortin and sortout dataset properties.


Phantom,

The following DFSORT/ICETOOL JCL will give you the desired results.If you have syncsort at your shop then change the pgm name to synctool.Just make sure that you delete the output file before every run as we are using disp=mod.

Code:

//STEP0100  EXEC  PGM=ICETOOL                                         
//TOOLMSG   DD SYSOUT=*                                               
//DFSMSG    DD SYSOUT=*                                               
//IN1       DD DSN=TSONJAY.CRD.MERGE.TEST1,
//             DISP=SHR
//IN2       DD DSN=TSONJAY.CRD.MERGE.TEST2,
//             DISP=SHR
//IN3       DD DSN=TSONJAY.CRD.MERGE.TEST3,
//             DISP=SHR
//OUT       DD DSN=YOUR OUTPUT DSN,
//             DISP=(MOD,CATLG,DELETE),
//             UNIT=SYSDA,
//             SPACE=(CYL,(1,1),RLSE
//TOOLIN   DD    *                                                     
  COPY FROM(IN1) USING(CTL1)                                           
  COPY FROM(IN2) USING(CTL2)                                           
  COPY FROM(IN3) USING(CTL3)                                           
//CTL1CNTL DD    *                                                     
  OUTFIL FNAMES=OUT,REMOVECC,HEADER1=('HEADER RECORD: FILE1 STARTS')   
//CTL2CNTL DD    *                                                     
  OUTFIL FNAMES=OUT,REMOVECC,HEADER1=('HEADER RECORD: FILE2 STARTS')   
//CTL3CNTL DD    *                                                     
  OUTFIL FNAMES=OUT,REMOVECC,HEADER1=('HEADER RECORD: FILE3 STARTS')
/*


Hope this helps...

Cheers

Kolusu
_________________
Kolusu
www.linkedin.com/in/kolusu
Back to top
View user's profile Send private message Send e-mail Visit poster's website
Phantom
Data Mgmt Moderator
Data Mgmt Moderator


Joined: 07 Jan 2003
Posts: 1056
Topics: 91
Location: The Blue Planet

PostPosted: Thu Mar 18, 2004 9:55 am    Post subject: Reply with quote

Thanks a lot Kolusu, But removing the temporary file (T2) from my code, you have made the code much cleaner...

Sorry to bother you more...Now I am facing another small problem in this...When the above code is put into a proc, I need to create 4 control cards for the SYNCTOOL, one for the TOOLIN and other 3 for the OUTFIL statements. Is there any other way to have the same thing accomplished using only 1 card !!!...

The above requirement can be easily accomplished by having the header records within flat files and concatenate them (as you & Ravi had already given) while sorting. In that case, the programmer needs to create a new header file everytime he adds a new file....If by any chance he forgets to create the file, the job will BLOW. All I wanted is to minimize the programmers work, and inturn minimize the amount of possible errors....

Please let me know if there is any way to accomplish this....

Once again, thanks a lot for your help....
Back to top
View user's profile Send private message
kolusu
Site Admin
Site Admin


Joined: 26 Nov 2002
Posts: 12378
Topics: 75
Location: San Jose

PostPosted: Thu Mar 18, 2004 10:49 am    Post subject: Reply with quote

Phantom,

You can follow ravi's suggested solution if you don't care about the file number on the header. Here is another alternative solution which will create the header with file number and it is done with just one control card and even the no: of passes are also reduced. This also eliminates the programmer having to create the header datasets.

Code:

//STEP0100  EXEC  PGM=SYNCTOOL
//TOOLMSG   DD SYSOUT=*                                   
//DFSMSG    DD SYSOUT=*                                   
//NULL      DD DUMMY,LRECL=80,RECFM=FB,BLKSIZE=0         
//T1        DD DSN=&T1,DISP=(,PASS),SPACE=(TRK,(1,1),RLSE)
//T2        DD DSN=&T2,DISP=(,PASS),SPACE=(TRK,(1,1),RLSE)
//T3        DD DSN=&T3,DISP=(,PASS),SPACE=(TRK,(1,1),RLSE)
//IN        DD DSN=&T1,DISP=OLD,VOL=REF=*.T1   
//          DD DSN=TSONJAY.CRD.MERGE.TEST1,
//             DISP=SHR
//          DD DSN=&T2,DISP=OLD,VOL=REF=*.T2   
//          DD DSN=TSONJAY.CRD.MERGE.TEST2,
//             DISP=SHR
//          DD DSN=&T3,DISP=OLD,VOL=REF=*.T3   
//          DD DSN=TSONJAY.CRD.MERGE.TEST3,
//             DISP=SHR
//OUT       DD DSN=YOUR OUTPUT DSN,
//             DISP=(NEW,CATLG,DELETE),
//             UNIT=SYSDA,
//             SPACE=(CYL,(X,Y),RLSE
//TOOLIN    DD *                                                   
  COPY FROM(NULL) USING(CTL1)                                           
  COPY FROM(IN) TO(OUT)                                                 
//CTL1CNTL DD    *                                                     
  OUTFIL FNAMES=T1,REMOVECC,HEADER1=('HEADER RECORD: FILE1 STARTS',80:X)
  OUTFIL FNAMES=T2,REMOVECC,HEADER1=('HEADER RECORD: FILE2 STARTS',80:X)
  OUTFIL FNAMES=T3,REMOVECC,HEADER1=('HEADER RECORD: FILE3 STARTS',80:X)
/*


Now will your next question be " can it be done without the temp datasets?"

Hope this helps...

Cheers

Kolusu
_________________
Kolusu
www.linkedin.com/in/kolusu
Back to top
View user's profile Send private message Send e-mail Visit poster's website
Phantom
Data Mgmt Moderator
Data Mgmt Moderator


Joined: 07 Jan 2003
Posts: 1056
Topics: 91
Location: The Blue Planet

PostPosted: Fri Mar 19, 2004 3:37 am    Post subject: Reply with quote

Great....

Thanks Kolusu and Ravi....I have learned lots of new things from you both....

Kolusu....I know you will KILL ME If I ask you that question " can it be done without the temp datasets?" Embarassed
Laughing

Thanks to you and to ravi for providing gr8 solutions....
Back to top
View user's profile Send private message
Phantom
Data Mgmt Moderator
Data Mgmt Moderator


Joined: 07 Jan 2003
Posts: 1056
Topics: 91
Location: The Blue Planet

PostPosted: Fri Mar 19, 2004 6:43 am    Post subject: Reply with quote

Kolusu,

when I tried to run the code, the job ended with a RC=16. It doesnot identify REMOVECC parameter in the outfil statement.

Code:

CTL1CNTL :                                                                     
 OUTFIL FNAMES=T1,REMOVECC,HEADER1=('HEADER RECORD: FILE1 STARTS',80:X)       
                  *                                                           
 OUTFIL FNAMES=T2,REMOVECC,HEADER1=('HEADER RECORD: FILE2 STARTS',80:X)       
                  *                                                           
 OUTFIL FNAMES=T3,REMOVECC,HEADER1=('HEADER RECORD: FILE3 STARTS',80:X)       
                  *                                                           
WER038I  WARNING: SYNCSORT MVS IS NOT CERTIFIED TO RUN ON Z/OS. CONTACT SYNCSOR
WER428I  CALLER-PROVIDED IDENTIFIER IS "0001"                                 
WER268A  OUTFIL STATEMENT  : SYNTAX ERROR                                     
WER268A  OUTFIL STATEMENT  : SYNTAX ERROR                                     
WER268A  OUTFIL STATEMENT  : SYNTAX ERROR                                     
WER211B  SYNCSMF  CALLED BY SYNCSORT; RC=0000                                 
WER449I  SYNCSORT GLOBAL DSM SUBSYSTEM ACTIVE                                 


My shop has only SYNCSORT. Do we have this option in SYNCSORT or is there any other command to remove the Carriage control chars in SYNCSORT ?

Please let me know...Thanks
Back to top
View user's profile Send private message
kolusu
Site Admin
Site Admin


Joined: 26 Nov 2002
Posts: 12378
Topics: 75
Location: San Jose

PostPosted: Fri Mar 19, 2004 7:25 am    Post subject: Reply with quote

Phantom,

You need syncsort for z/OS for the parameter removecc to work. You can simply change it without the header parm
Code:

//CTL1CNTL DD    *                                                     
  OUTFIL FNAMES=T1,OUTREC=(C'HEADER RECORD: FILE1 STARTS',80:X)
  OUTFIL FNAMES=T2,OUTREC=(C'HEADER RECORD: FILE2 STARTS',80:X)
  OUTFIL FNAMES=T3,OUTREC=(C'HEADER RECORD: FILE3 STARTS',80:X)
/*


Hope this helps...

Cheers

Kolusu

PS: I cannot think any other way without the creation of temp datasets.
_________________
Kolusu
www.linkedin.com/in/kolusu
Back to top
View user's profile Send private message Send e-mail Visit poster's website
Phantom
Data Mgmt Moderator
Data Mgmt Moderator


Joined: 07 Jan 2003
Posts: 1056
Topics: 91
Location: The Blue Planet

PostPosted: Fri Mar 19, 2004 8:18 am    Post subject: Reply with quote

Thanks Kolusu.... My mind was only thinking of the Header parm. So, I just did not think of this solution.... So simple...buuuut....still... Embarassed

Thanks for pointing it out...
Back to top
View user's profile Send private message
Phantom
Data Mgmt Moderator
Data Mgmt Moderator


Joined: 07 Jan 2003
Posts: 1056
Topics: 91
Location: The Blue Planet

PostPosted: Fri Mar 19, 2004 9:25 am    Post subject: Reply with quote

Kolusu,

I think we are missing something here....The no. of records in the temp. datasets (which hold the Header info.) T1, T2 and T3 will be equal to the no. of records in the DUMMY file (DD Name = NULL). Since DD DUMMY has no records in it. the header info. is not populated in the output. We need to look into this....If you look at the code that I sent first... I had a step which creates a dummy file with only one record in it...

Code:

//R002     EXEC PGM=IEBDG                   
//SYSPRINT DD  SYSOUT=*                     
//OUTDD    DD  DSN=&T4,DISP=(,PASS),         
//            SPACE=(80,(1,1),RLSE),         
//            DCB=(RECFM=FB,LRECL=80)       
//SYSIN    DD  *                             
 DSD     OUTPUT=(OUTDD)                     
 FD      NAME=FILLER1,LENGTH=80,FORMAT=AN   
 CREATE  QUANTITY=1                         
 END                                         
/*                                           
//*                                         


Now, we need to find a way out to get the header records in the file. Your previous solution (below) will give the desired result, but it takes more control cards i.e one control card for every new file that is added.

Code:

//STEP0100  EXEC  PGM=ICETOOL                                         
//TOOLMSG   DD SYSOUT=*                                               
//DFSMSG    DD SYSOUT=*                                               
//IN1       DD DSN=TSONJAY.CRD.MERGE.TEST1,
//             DISP=SHR
//IN2       DD DSN=TSONJAY.CRD.MERGE.TEST2,
//             DISP=SHR
//IN3       DD DSN=TSONJAY.CRD.MERGE.TEST3,
//             DISP=SHR
//OUT       DD DSN=YOUR OUTPUT DSN,
//             DISP=(MOD,CATLG,DELETE),
//             UNIT=SYSDA,
//             SPACE=(CYL,(1,1),RLSE
//TOOLIN   DD    *                                                     
  COPY FROM(IN1) USING(CTL1)                                           
  COPY FROM(IN2) USING(CTL2)                                           
  COPY FROM(IN3) USING(CTL3)                                           
//CTL1CNTL DD    *                                                     
  OUTFIL FNAMES=OUT,REMOVECC,HEADER1=('HEADER RECORD: FILE1 STARTS')   
//CTL2CNTL DD    *                                                     
  OUTFIL FNAMES=OUT,REMOVECC,HEADER1=('HEADER RECORD: FILE2 STARTS')   
//CTL3CNTL DD    *                                                     
  OUTFIL FNAMES=OUT,REMOVECC,HEADER1=('HEADER RECORD: FILE3 STARTS')
/*


please help me out....Thanks a lot
Back to top
View user's profile Send private message
kolusu
Site Admin
Site Admin


Joined: 26 Nov 2002
Posts: 12378
Topics: 75
Location: San Jose

PostPosted: Fri Mar 19, 2004 9:38 am    Post subject: Reply with quote

Phantom,

One thing I forgot to mention is that when you use outrec you cannot have a dummy dsn. You need atleast one record in the null file.

so change your null file allocation to the following.
Code:

//NULL   DD *
DUMMY RECORD


or you can use the 1 line file created using iebdg.

Another alternative solution using header1 parm , but involves an extra parm and an extra pass.

Code:

//STEP0100  EXEC  PGM=SYNCTOOL
//TOOLMSG   DD SYSOUT=*                                           
//DFSMSG    DD SYSOUT=*                                           
//NULL      DD DUMMY,LRECL=80,RECFM=FB                             
//HEAD      DD DSN=&H1,DISP=(,PASS),SPACE=(TRK,(1,1),RLSE)         
//T1        DD DSN=&T1,DISP=(,PASS),SPACE=(TRK,(1,1),RLSE),RECFM=FB
//T2        DD DSN=&T2,DISP=(,PASS),SPACE=(TRK,(1,1),RLSE),RECFM=FB
//T3        DD DSN=&T3,DISP=(,PASS),SPACE=(TRK,(1,1),RLSE),RECFM=FB
//IN        DD DSN=&T1,DISP=OLD,VOL=REF=*.T1                       
//          DD *                                                   
ABC                                                               
//          DD DSN=&T2,DISP=OLD,VOL=REF=*.T2                       
//          DD *                                                   
DEF                                                               
//          DD DSN=&T3,DISP=OLD,VOL=REF=*.T3                       
//          DD *                                                   
DEF                                                               
//OUT       DD SYSOUT=*                                           
//TOOLIN    DD    *                                               
  COPY FROM(NULL) USING(CTL1)                                       
  COPY FROM(HEAD) USING(CTL2)                                       
  COPY FROM(IN) TO(OUT)                                             
//CTL1CNTL DD    *                                                 
  OUTFIL FNAMES=HEAD,                                               
  HEADER1=('HEADER RECORD: FILE1 STARTS',/,                         
           'HEADER RECORD: FILE2 STARTS',/,                         
           'HEADER RECORD: FILE3 STARTS',80:X)                     
//CTL2CNTL DD    *                                                 
  OUTFIL FNAMES=T1,STARTREC=1,ENDREC=1,OUTREC=(2,80)               
  OUTFIL FNAMES=T2,STARTREC=2,ENDREC=2,OUTREC=(2,80)               
  OUTFIL FNAMES=T3,STARTREC=3,ENDREC=3,OUTREC=(2,80)               
/*

Hope this helps...

Cheers

Kolusu
_________________
Kolusu
www.linkedin.com/in/kolusu
Back to top
View user's profile Send private message Send e-mail Visit poster's website
Display posts from previous:   
Post new topic   Reply to topic   printer-friendly view    MVSFORUMS.com Forum Index -> Utilities All times are GMT - 5 Hours
Goto page 1, 2  Next
Page 1 of 2

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum


MVSFORUMS
Powered by phpBB © 2001, 2005 phpBB Group