MVSFORUMS.com Forum Index MVSFORUMS.com
A Community of and for MVS Professionals
 
 FAQFAQ   SearchSearch   Quick Manuals   RegisterRegister 
 ProfileProfile   Log in to check your private messagesLog in to check your private messages   Log inLog in 

remove duplicate records

 
Post new topic   Reply to topic   printer-friendly view    MVSFORUMS.com Forum Index -> Utilities
View previous topic :: View next topic  
Author Message
js01
Beginner


Joined: 13 Oct 2005
Posts: 84
Topics: 32
Location: INDIA
PostPosted: Tue Dec 05, 2006 3:04 pm    Post subject: remove duplicate records Reply with quote
Hi all,

I have a problem with the duplicates , i want to remove duplicate record some criteria .

in put file like

Code:

RAJ        AAAA
MAT        XXXX
SAT        BBBB
VAT        CCCC
RAJ        BBBB
VAT        BBBB
XAT        YYYYY


OUT PUR SHOULD BE ( Want to elimenate dupliacte record of partnumber which is not equalt to 'BBBB' , here key is name)

Code:

MAT        XXXX
SAT        BBBB
RAJ        BBBB
VAT        BBBB
XAT        YYYYY


CAN YOU PLEASE HELP

Note : please consider as 80 lrecl
Back to top
View user's profile Send private message
Frank Yaeger
Sort Forum Moderator
Sort Forum Moderator


Joined: 02 Dec 2002
Posts: 1618
Topics: 31
Location: San Jose
PostPosted: Tue Dec 05, 2006 3:39 pm    Post subject: Reply with quote
Here's a DFSORT/ICETOOL job that will do what I think you want based on what you've posted:

Code:

//S1 EXEC PGM=ICETOOL
//TOOLMSG DD SYSOUT=*
//DFSMSG DD SYSOUT=*
//IN DD DSN=...  input file (FB/80)
//OUT DD DSN=...  output file (FB/80)
//TOOLIN DD *
SELECT FROM(IN) TO(OUT) ON(1,3,CH) FIRST USING(CTL1)
/*
//CTL1CNTL DD *
  INREC IFTHEN=(WHEN=INIT,OVERLAY=(81:C'B')),
    IFTHEN=(WHEN=(12,4,CH,EQ,C'BBBB'),OVERLAY=(81:C'A'))
  SORT FIELDS=(1,3,CH,A,81,1,CH,A)
  OUTFIL FNAMES=OUT,BUILD=(1,80)
/*


OUT would have:

Code:

MAT        XXXX   
RAJ        BBBB   
SAT        BBBB   
VAT        BBBB   
XAT        YYYYY 


Note that the records are in sorted order by the name field. If you really want the records in their original order, you could add a sequence number and use a SORT operator to sort on it like this:

Code:

//S2 EXEC PGM=ICETOOL
//TOOLMSG DD SYSOUT=*
//DFSMSG DD SYSOUT=*
//IN DD DSN=...  input file (FB/80)
//T1 DD DSN=&&T1,UNIT=SYSDA,SPACE=(CYL,(5,5)),DISP=(,PASS)
//OUT DD DSN=...  output file (FB/80)
//TOOLIN DD *
SELECT FROM(IN) TO(T1) ON(1,3,CH) FIRST USING(CTL1)
SORT FROM(T1) TO(OUT) USING(CTL2)
/*
//CTL1CNTL DD *
  INREC IFTHEN=(WHEN=INIT,OVERLAY=(81:C'B',82:SEQNUM,8,ZD)),
    IFTHEN=(WHEN=(12,4,CH,EQ,C'BBBB'),OVERLAY=(81:C'A'))
  SORT FIELDS=(1,3,CH,A,81,1,CH,A)
/*
//CTL2CNTL DD *
  SORT FIELDS=(82,8,ZD,A)
  OUTREC BUILD=(1,80)
/*


OUT would have:

Code:

MAT        XXXX   
SAT        BBBB   
RAJ        BBBB   
VAT        BBBB   
XAT        YYYYY 

_________________
Frank Yaeger - DFSORT Development Team (IBM)
Specialties: JOINKEYS, FINDREP, WHEN=GROUP, ICETOOL, Symbols, Migration
DFSORT is on the Web at:
www.ibm.com/storage/dfsort
Back to top
View user's profile Send private message Send e-mail Visit poster's website
js01
Beginner


Joined: 13 Oct 2005
Posts: 84
Topics: 32
Location: INDIA
PostPosted: Tue Dec 05, 2006 5:11 pm    Post subject: Reply with quote
Dear Frank,

Thank you for your quick responce
Can you please give me the link so that i can go thru it and understand.

actually i am getting following error (first time i am using ICETOOL)
Code:

********************************* TOP OF DATA **********************************
ICE200I 0 IDENTIFIER FROM CALLING PROGRAM IS 0001                               
ICE143I 0 BLOCKSET     SORT  TECHNIQUE SELECTED                                 
ICE000I 0 - CONTROL STATEMENTS FOR 5740-SM1, DFSORT REL 14.0 - 14:03 ON TUE DEC
            INREC IFTHEN=(WHEN=INIT,OVERLAY=(81:C'B')),                         
                  $                                                             
ICE104A 0 INVALID INREC OR OUTREC STATEMENT OPERAND                             
            IFTHEN=(WHEN=(12,4,CH,EQ,C'BBBB'),OVERLAY=(81:C'A'))               
            $                                                                   
ICE005A 0 STATEMENT DEFINER ERROR                                               
            SORT FIELDS=(1,3,CH,A,81,1,CH,A)                                   
            OUTFIL FNAMES=OUT,BUILD=(1,80)                                     
                              $                                                 
ICE213A 0 INVALID OUTFIL STATEMENT OPERAND                                     
ICE146I 0 END OF STATEMENTS FROM CTL1CNTL - PARAMETER LIST STATEMENTS FOLLOW   
          DEBUG NOABEND,ESTAE                                                   
          OPTION LIST,MSGPRT=ALL,MSGDDN=DFSMSG,RESINV=0,SORTDD=CTL1,SORTIN=IN,SO
                         TOUT=OUT,DYNALLOC,SZERO,EQUALS,NOVLSHRT,LOCALE=NONE,NOC
                         ECK                                                   
          SORT FIELDS=(1,3,CH,A)                                               

          MODS E35=(ICE35DU,8192)                                               
ICE012A 2 MISSING FIELDS OPERAND DEFINER                                       
ICE052I 3 END OF DFSORT                                                         
******************************** BOTTOM OF DATA ********************************


The JCL is:
Code:

***************************** Top of Data ***********************
//SORTDUP  JOB (VAS1,T),T,MSGCLASS=X,CLASS=Q,COND=(0,NE),       
//          NOTIFY=&SYSUID                                       
//*                                                             
//S1 EXEC PGM=ICETOOL                                           
//TOOLMSG DD SYSOUT=*                                           
//DFSMSG DD SYSOUT=*                                             
//IN DD *                                                       
 RAJ        AAAA                                                 
 MAT        XXXX                                                 
 SAT        BBBB                                                 
 VAT        CCCC                                                 
 RAJ        BBBB                                                 
 VAT        BBBB                                                 
 XAT        YYYYY                                               
/*                                                               
//OUT DD SYSOUT=* OUTPUT FILE (FB/80)                           
//TOOLIN DD *                                                   
SELECT FROM(IN) TO(OUT) ON(1,3,CH) FIRST USING(CTL1)             
/*                                                               
//CTL1CNTL DD *                                         
  INREC IFTHEN=(WHEN=INIT,OVERLAY=(81:C'B')),           
  IFTHEN=(WHEN=(12,4,CH,EQ,C'BBBB'),OVERLAY=(81:C'A')) 
  SORT FIELDS=(1,3,CH,A,81,1,CH,A)                     
  OUTFIL FNAMES=OUT,BUILD=(1,80)                       
/*
Back to top
View user's profile Send private message
Frank Yaeger
Sort Forum Moderator
Sort Forum Moderator


Joined: 02 Dec 2002
Posts: 1618
Topics: 31
Location: San Jose
PostPosted: Tue Dec 05, 2006 5:58 pm    Post subject: Reply with quote
The error message indicates you do not have DFSORT R14 PTF UQ95213 installed. This PTF has been available since Dec, 2004, so your site is quite a bit behind on installing DFSORT service. Ask your System Programmer to install DFSORT R14 PTF UK90006 (April, 2006) - that will get you all of the latest DFSORT functions and then you'll be able to run the job successfully.

For a complete description of all of the Dec, 2004 DFSORT functions, including IFTHEN, see:

www.ibm.com/servers/storage/support/software/sort/mvs/pdug/

For a complete description of all of the April, 2006 DFSORT functions, see:

www.ibm.com/servers/storage/support/software/sort/mvs/peug/

If you're not familiar with DFSORT and DFSORT's ICETOOL, I'd suggest reading through "z/OS DFSORT: Getting Started". It's an excellent tutorial, with lots of examples, that will show you how to use DFSORT, DFSORT's ICETOOL and DFSORT Symbols. You can access it online, along with all of the other DFSORT books, from:

www.ibm.com/servers/storage/support/software/sort/mvs/srtmpub.html
_________________
Frank Yaeger - DFSORT Development Team (IBM)
Specialties: JOINKEYS, FINDREP, WHEN=GROUP, ICETOOL, Symbols, Migration
DFSORT is on the Web at:
www.ibm.com/storage/dfsort
Back to top
View user's profile Send private message Send e-mail Visit poster's website
js01
Beginner


Joined: 13 Oct 2005
Posts: 84
Topics: 32
Location: INDIA
PostPosted: Wed Dec 06, 2006 7:33 pm    Post subject: Reply with quote
Dear Frank,

Thank you very much for provided information.

is there any possiblity to meet my requirement using SORT utility which we have

thank you
Back to top
View user's profile Send private message
Frank Yaeger
Sort Forum Moderator
Sort Forum Moderator


Joined: 02 Dec 2002
Posts: 1618
Topics: 31
Location: San Jose
PostPosted: Wed Dec 06, 2006 7:48 pm    Post subject: Reply with quote
Quote:
is there any possiblity to meet my requirement using SORT utility which we have


The sort utility you have is DFSORT - you just don't have the Dec, 2004 or April, 2006 DFSORT PTFs. You can use this DFSORT/ICETOOL job instead of the first S1 job to do what you want without requiring those PTFs.

Code:

//S1 EXEC PGM=ICETOOL
//TOOLMSG DD SYSOUT=*
//DFSMSG DD SYSOUT=*
//IN DD DSN=...  input file (FB/80)
//T1 DD DSN=&&T1,UNIT=SYSDA,SPACE=(CYL,(5,5)),DISP=(,PASS)
//OUT DD DSN=...  output file (FB/80)
//OUT DD SYSOUT=*
//TOOLIN DD *
COPY FROM(IN) TO(T1) USING(CTL1)
SELECT FROM(T1) TO(OUT) ON(1,3,CH) FIRST USING(CTL2)
/*
//CTL1CNTL DD *
  INREC FIELDS=(1,80,
    81:12,4,CHANGE=(1,C'BBBB',C'A'),NOMATCH=(C'B'))
/*
//CTL2CNTL DD *
  SORT FIELDS=(1,3,CH,A,81,1,CH,A)
  OUTFIL FNAMES=OUT,OUTREC=(1,80)
/*

_________________
Frank Yaeger - DFSORT Development Team (IBM)
Specialties: JOINKEYS, FINDREP, WHEN=GROUP, ICETOOL, Symbols, Migration
DFSORT is on the Web at:
www.ibm.com/storage/dfsort
Back to top
View user's profile Send private message Send e-mail Visit poster's website
Display posts from previous:   
Post new topic   Reply to topic   printer-friendly view    MVSFORUMS.com Forum Index -> Utilities All times are GMT - 5 Hours
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum


MVSFORUMS
Powered by phpBB © 2001, 2005 phpBB Group