MVSFORUMS.com Forum Index MVSFORUMS.com
A Community of and for MVS Professionals
 
 FAQFAQ   SearchSearch   Quick Manuals   RegisterRegister 
 ProfileProfile   Log in to check your private messagesLog in to check your private messages   Log inLog in 

Using Sections, Trailer to eliminate duplicate and get count
Goto page 1, 2  Next
 
Post new topic   Reply to topic   printer-friendly view    MVSFORUMS.com Forum Index -> Utilities
View previous topic :: View next topic  
Author Message
naveen_summary
Beginner


Joined: 12 Feb 2007
Posts: 26
Topics: 13
PostPosted: Mon Feb 12, 2007 9:49 am    Post subject: Using Sections, Trailer to eliminate duplicate and get count Reply with quote
Hello gurus

My infile is of length 12735. I need an output file of length 12739 (Infile + Count of duplicates). My job should give me one record for duplicates (SUM FILEDS=NONE) and the count of duplicates. I am using below job but it fails with error:

Quote:


Error Message:

TRAILER3=(1,12735,X,COUNT=(M11,LENGTH=3)))
$
ICE223A F REPORT FIELD ERROR


I then analysed the error and changed the code to
Quote:
TRAILER3=(1,256,X,COUNT=(M11,LENGTH=3)))
.

It ran with maxcc=0 but it gave an outfile of 12735 (with 256 byte data from infile and also without my duplicate count).

I am using below code:


Quote:
//STEP01 EXEC PGM=ICEMAN
//SORTIN DD DSN=FileA,DISP=SHR
//SORTOUT DD DSN=FileB,DISP=(NEW,CATLG,DELETE)
//SYSIN DD *
SORT FIELDS=(34,4,CH,A)
OUTFIL REMOVECC,NODETAIL,
SECTIONS=(34,4,
TRAILER3=(X,COUNT=(M11,LENGTH=3),1,12735))
/*



Why is this happening. Is there another way to acheive my requirement.
Back to top
View user's profile Send private message
kolusu
Site Admin
Site Admin


Joined: 26 Nov 2002
Posts: 12380
Topics: 75
Location: San Jose
PostPosted: Mon Feb 12, 2007 10:37 am    Post subject: Reply with quote
Naveen_summary,

The first error is because you cannot specify more than 256 bytes on the header/trailer parms. ie trailer1=(p,m) P specifies the position and m specifies the length in bytes of the input field. The value for m must be between 1 and 256.

If your intention is just find the count of duplicate records, try these control cards
Code:

 SORT  FIELDS=(34,4,CH,A)   
 OUTFIL REMOVECC,NODETAIL, 
 SECTIONS=(34,4,           
 TRAILER3=(34,4,2X,COUNT=(M11,LENGTH=3),80:X))
/*                         


Hope this helps...

Cheers

Kolusu
_________________
Kolusu
www.linkedin.com/in/kolusu
Back to top
View user's profile Send private message Send e-mail Visit poster's website
naveen_summary
Beginner


Joined: 12 Feb 2007
Posts: 26
Topics: 13
PostPosted: Mon Feb 12, 2007 11:01 am    Post subject: Reply with quote
Kolusu, I need the input record and the count.

Output Record = Last duplicate record + Count of duplicates.

Thanks, Naveen.
Back to top
View user's profile Send private message
kolusu
Site Admin
Site Admin


Joined: 26 Nov 2002
Posts: 12380
Topics: 75
Location: San Jose
PostPosted: Mon Feb 12, 2007 11:42 am    Post subject: Reply with quote
naveen_summary,


Try these control cards

Code:

SORT FIELDS=(34,4,CH,A)
OUTREC FIELDS=(01,12735,3X)
OUTFIL REMOVECC,
SECTIONS=(01,11,           
TRAILER3=(00001,256,       
          00257,256,       
          00513,256,       
          00769,256,       
          01025,256,       
          01281,256,       
          01537,256,       
          01793,256,       
          02049,256,       
          02305,256,       
          02561,256,       
          02817,256,       
          03073,256,       
          03329,256,       
          03585,256,       
          03841,256,       
          04097,256,       
          04353,256,       
          04609,256,       
          04865,256,       
          05121,256,       
          05377,256,       
          05633,256,       
          05889,256,       
          06145,256,       
          06401,256,       
          06657,256,       
          06913,256,       
          07169,256,       
          07425,256,       
          07681,256,       
          07937,256,       
          08193,256,       
          08449,256,       
          08705,256,       
          08961,256,       
          09217,256,       
          09473,256,           
          09729,256,           
          09985,256,           
          10241,256,           
          10497,256,           
          10753,256,           
          11009,256,           
          11265,256,           
          11521,256,           
          11777,256,           
          12033,256,           
          12289,256,           
          12545,191,           
          COUNT=(M11,LENGTH=3)))
/*



Hope this helps...

Cheers

Kolusu
_________________
Kolusu
www.linkedin.com/in/kolusu
Back to top
View user's profile Send private message Send e-mail Visit poster's website
Frank Yaeger
Sort Forum Moderator
Sort Forum Moderator


Joined: 02 Dec 2002
Posts: 1618
Topics: 31
Location: San Jose
PostPosted: Mon Feb 12, 2007 11:49 am    Post subject: Reply with quote
Alernatively, you can use this DFSORT/ICETOOL job ... it takes two passes instead of one, but it doesn't require all of those p,m lines:

Code:

//S1    EXEC  PGM=ICETOOL                                       
//TOOLMSG   DD  SYSOUT=*                                       
//DFSMSG   DD  SYSOUT=*                                         
//IN DD DSN=...  input file (FB/12735)                               
//T1 DD DSN=&&T1,UNIT=SYSDA,SPACE=(CYL,(5,5)),DISP=(,PASS)     
//OUT DD DSN=... output file (FB/12738)                       
//TOOLIN   DD    *                                             
SORT FROM(IN) TO(T1) USING(CTL1)                               
SELECT FROM(T1) TO(OUT) ON(34,4,CH) LAST                       
//CTL1CNTL DD *                                                 
  OPTION EQUALS                                                 
  SORT FIELDS=(34,4,CH,A)                                       
  OUTREC OVERLAY=(12736:SEQNUM,3,ZD,RESTART=(34,4))             
/*

_________________
Frank Yaeger - DFSORT Development Team (IBM)
Specialties: JOINKEYS, FINDREP, WHEN=GROUP, ICETOOL, Symbols, Migration
DFSORT is on the Web at:
www.ibm.com/storage/dfsort
Back to top
View user's profile Send private message Send e-mail Visit poster's website
kolusu
Site Admin
Site Admin


Joined: 26 Nov 2002
Posts: 12380
Topics: 75
Location: San Jose
PostPosted: Mon Feb 12, 2007 12:29 pm    Post subject: Reply with quote
Quote:

it takes two passes instead of one, but it doesn't require all of those p,m lines:


Frank,

I thought SELECT supports INREC processing. In that case wouldn't this 1 pass step get the desired results?

Code:

//S1    EXEC  PGM=ICETOOL                                       
//TOOLMSG   DD  SYSOUT=*                                       
//DFSMSG   DD  SYSOUT=*                                         
//IN DD DSN=...  input file (FB/12735)                               
//OUT DD DSN=... output file (FB/12738)
//TOOLIN    DD *                                       
 SELECT FROM(IN) TO(OUT) ON(01,11,CH) LAST USING(CTL1)
//CTL1CNTL  DD *                                       
 INREC OVERLAY=(12736:SEQNUM,3,ZD,RESTART=(01,11))     
/*


Kolusu
_________________
Kolusu
www.linkedin.com/in/kolusu
Back to top
View user's profile Send private message Send e-mail Visit poster's website
Frank Yaeger
Sort Forum Moderator
Sort Forum Moderator


Joined: 02 Dec 2002
Posts: 1618
Topics: 31
Location: San Jose
PostPosted: Mon Feb 12, 2007 12:59 pm    Post subject: Reply with quote
Kolusu,

Yes, DFSORT's SELECT supports INREC processing. But using INREC that way with SPLICE would only work correctly if the records were already in sorted order, which we don't know to be the case here. Remember that INREC is processed before SELECT. If the records weren't in sorted order, then we wouldn't get the correct count from RESTART since it starts over at 1 each time the key changes. For example, the count might be as follows:

A 1
B 1
B 2
A 1
A 2
B 1
B 2

so the last "count" for A would be 2 instead of 3 and the last count for B would be 2 instead of 4.

That's why the extra SORT operator with OUTREC (processed after SORT) is needed. If we sort the records first, RESTART gives us the count we want:

A 1
A 2
A 3
B 1
B 2
B 3
B 4
_________________
Frank Yaeger - DFSORT Development Team (IBM)
Specialties: JOINKEYS, FINDREP, WHEN=GROUP, ICETOOL, Symbols, Migration
DFSORT is on the Web at:
www.ibm.com/storage/dfsort
Back to top
View user's profile Send private message Send e-mail Visit poster's website
kolusu
Site Admin
Site Admin


Joined: 26 Nov 2002
Posts: 12380
Topics: 75
Location: San Jose
PostPosted: Mon Feb 12, 2007 1:26 pm    Post subject: Reply with quote
Frank,

I completely forgot about the unsorted input :bang_head_icon.Sorry

kolusu
_________________
Kolusu
www.linkedin.com/in/kolusu
Back to top
View user's profile Send private message Send e-mail Visit poster's website
Frank Yaeger
Sort Forum Moderator
Sort Forum Moderator


Joined: 02 Dec 2002
Posts: 1618
Topics: 31
Location: San Jose
PostPosted: Mon Feb 12, 2007 3:23 pm    Post subject: Reply with quote
Nothing to be sorry about. I started out thinking I could do what you said; it took me a while to figure out I couldn't.
_________________
Frank Yaeger - DFSORT Development Team (IBM)
Specialties: JOINKEYS, FINDREP, WHEN=GROUP, ICETOOL, Symbols, Migration
DFSORT is on the Web at:
www.ibm.com/storage/dfsort
Back to top
View user's profile Send private message Send e-mail Visit poster's website
js01
Beginner


Joined: 13 Oct 2005
Posts: 84
Topics: 32
Location: INDIA
PostPosted: Sat Mar 17, 2007 12:25 am    Post subject: Reply with quote
Frank/Kolusu,

when i try above code , getting following error message,

IS my shop doesn't support ICETOOL? can you please help me.

below is the jcl
Code:

//S1    EXEC  PGM=ICETOOL                                     
//TOOLMSG   DD  SYSOUT=*                                     
//DFSMSG   DD  SYSOUT=*                                       
//IN DD *                                                     
  11  AA  BB  00                                             
  22  XX  CC  12                                             
  11  AA  BB  00                                             
  22  XX  CC  12                                             
  33  BB  DD  12                                             
  11  AA      00                                             
/*                                                           
//T1 DD DSN=&&T1,UNIT=SYSDA,SPACE=(CYL,(5,5)),DISP=(,PASS)   
//OUT DD SYSOUT=*                                             
//TOOLIN   DD    *                                           
SORT FROM(IN) TO(T1) USING(CTL1)                             
SELECT FROM(T1) TO(OUT) ON(1,80,CH) LAST                     
//CTL1CNTL DD *                                               
  OPTION EQUALS                                               
  SORT FIELDS=(1,80,CH,A)                                     
  OUTREC OVERLAY=(80:SEQNUM,3,ZD,RESTART=(1,80))             
/*                                                           



below is the job log


Code:

ICE200I 0 IDENTIFIER FROM CALLING PROGRAM IS 0001                               
ICE143I 0 BLOCKSET     SORT  TECHNIQUE SELECTED                                 
ICE000I 0 - CONTROL STATEMENTS FOR 5740-SM1, DFSORT REL 14.0 - 22:20 ON FRI MAR
            OPTION EQUALS                                                       
            SORT FIELDS=(1,80,CH,A)                                             
            OUTREC OVERLAY=(80:SEQNUM,3,ZD,RESTART=(1,80))                     
                   $                                                           
ICE104A 0 INVALID INREC OR OUTREC STATEMENT OPERAND                             
ICE146I 0 END OF STATEMENTS FROM CTL1CNTL - PARAMETER LIST STATEMENTS FOLLOW   
          DEBUG NOABEND,ESTAE                                                   
          OPTION LIST,MSGPRT=ALL,MSGDDN=DFSMSG,RESINV=0,SORTDD=CTL1,SORTIN=IN,SO
                         TOUT=T1,DYNALLOC                                       
ICE012A 3 MISSING FIELDS OPERAND DEFINER                                       
ICE052I 3 END OF DFSORT                                                         
******************************** BOTTOM OF DATA ********************************




********************************* TOP OF DATA **********************************
ICE600I 0 DFSORT ICETOOL UTILITY RUN STARTED                                   
                                                                               
ICE632I 0 SOURCE FOR ICETOOL STATEMENTS:  TOOLIN                               
                                                                               
                                                                               
ICE630I 0 MODE IN EFFECT:  STOP                                                 
                                                                               
          SORT FROM(IN) TO(T1) USING(CTL1)                                     
ICE606I 0 DFSORT CALL 0001 FOR SORT FROM IN       TO T1       USING CTL1CNTL TER
ICE602I 0 OPERATION RETURN CODE:  16                                           
                                                                               
ICE630I 2 MODE IN EFFECT:  SCAN                                                 
                                                                               
          SELECT FROM(T1) TO(OUT) ON(1,80,CH) LAST                             
ICE612I 0 NO ERRORS FOUND IN STATEMENT                                         
                                                                               
                                                                               
ICE601I 0 DFSORT ICETOOL UTILITY RUN ENDED - RETURN CODE:  16
Back to top
View user's profile Send private message
js01
Beginner


Joined: 13 Oct 2005
Posts: 84
Topics: 32
Location: INDIA
PostPosted: Sat Mar 17, 2007 12:49 am    Post subject: Reply with quote
Hello kolusu/frank,

I have used below(link) posted and my issue is resloved, but i would like to know whether my shop is having ICETOLL or not, can you please advise.

http://www.mvsforums.com/helpboards/viewtopic.php?t=6007&highlight=duplicate

thank you
Back to top
View user's profile Send private message
Nic Clouston
Advanced


Joined: 01 Feb 2007
Posts: 1075
Topics: 7
Location: At Home
PostPosted: Sat Mar 17, 2007 9:12 am    Post subject: Reply with quote
If you did not have ICETOOL you would not be getting ICETOOL meassages - your job would abend with load module not found.
_________________
Utility and Program control cards are NOT, repeat NOT, JCL.
Back to top
View user's profile Send private message
Frank Yaeger
Sort Forum Moderator
Sort Forum Moderator


Joined: 02 Dec 2002
Posts: 1618
Topics: 31
Location: San Jose
PostPosted: Sat Mar 17, 2007 10:04 am    Post subject: Reply with quote
js01n

Yes, you have DFSORT's ICETOOL (ifyou have DFSORT, you have DFSORT's ICETOOL). The error indicates you don't have the Dec, 2004 DFSORT PTF which is required for OVERLAY. So your site is quite far behind on DFSORT service. Ask your system programmer to install DFSORT R14 PTF UK90006 (April, 2006). That will get you all of the available DFSORT/ICETOOL function.
_________________
Frank Yaeger - DFSORT Development Team (IBM)
Specialties: JOINKEYS, FINDREP, WHEN=GROUP, ICETOOL, Symbols, Migration
DFSORT is on the Web at:
www.ibm.com/storage/dfsort
Back to top
View user's profile Send private message Send e-mail Visit poster's website
Alain Benveniste
Beginner


Joined: 04 May 2003
Posts: 92
Topics: 4
Location: Paris, France
PostPosted: Sat Mar 17, 2007 8:19 pm    Post subject: Reply with quote
2 hour in the morning...It's not a time to play Smile
Well I think that's what Kolusu & Frank wanted to post. I just concatened their ideas.
Quote:

//STEP0001 EXEC PGM=ICETOOL
//TOOLMSG DD SYSOUT=*
//DFSMSG DD SYSOUT=*
//TOOLIN DD *
SORT FROM(IN) TO(OUTX) USING(ICE0)
/*
//IN DD *
A 1
B 1
B 2
A 1
A 2
B 1
B 2
/*
//OUTX DD SYSOUT=*
//ICE0CNTL DD *
INREC OVERLAY=(10:C'1')
SORT FIELDS=(1,1,CH;A)
OUTFIL FNAMES=OUTX,NODETAIL,
SECTIONS=(1,1,
TRAILER3=(1,3,X,TOTAL=(10,1,ZD,M11,LENGTH=5)))

Frank,
I didn't know that SORT card accepted ; character. I expected a syntax error...

Alain
Back to top
View user's profile Send private message
Alain Benveniste
Beginner


Joined: 04 May 2003
Posts: 92
Topics: 4
Location: Paris, France
PostPosted: Sun Mar 18, 2007 5:04 am    Post subject: Reply with quote
Oups,

I now realize I missed that the 'problem' is the trailer3 restriction.

Alain
Back to top
View user's profile Send private message
Display posts from previous:   
Post new topic   Reply to topic   printer-friendly view    MVSFORUMS.com Forum Index -> Utilities All times are GMT - 5 Hours
Goto page 1, 2  Next
Page 1 of 2

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum


MVSFORUMS
Powered by phpBB © 2001, 2005 phpBB Group