MVSFORUMS.com Forum Index MVSFORUMS.com
A Community of and for MVS Professionals
 
 FAQFAQ   SearchSearch   Quick Manuals   RegisterRegister 
 ProfileProfile   Log in to check your private messagesLog in to check your private messages   Log inLog in 

Sort Input file based on certain criteria !!
Goto page Previous  1, 2, 3  Next
 
Post new topic   Reply to topic   printer-friendly view    MVSFORUMS.com Forum Index -> Utilities
View previous topic :: View next topic  
Author Message
kolusu
Site Admin
Site Admin


Joined: 26 Nov 2002
Posts: 12357
Topics: 75
Location: San Jose

PostPosted: Thu Feb 23, 2006 6:22 am    Post subject: Reply with quote

amit,

Try this . I assumed that '*' is pos 63 ( vb file 4 bytes rdw and 1 byte carriage control). However I am not sure if horizontal math functions are supported with DFSORT r14 version.


Code:

//STEP0100 EXEC PGM=SORT                                         
//SYSOUT   DD SYSOUT=*                                           
//SORTIN   DD DSN=YOUR INPUT VBA FILE,
//            DISP=SHR                           
//SORTOUT  DD DSN=&T1,DISP=(,CATLG),SPACE=(CYL,(X,Y),RLSE)
//SYSIN    DD *                                             
  SORT FIELDS=COPY                                           
  INREC FIELDS=(1,5,                                         
                6:63,1,CHANGE=(1,C'*',C'1'),NOMATCH=(C'0'),
                SEQNUM,8,ZD,6)                               
  OUTREC FIELDS=(1,5,                                       
                 6,1,ZD,ADD,7,8,ZD,EDIT=(TTTTTTTT),15)       
/*   
//STEP0200 EXEC PGM=ICETOOL                                 
//TOOLMSG  DD SYSOUT=*                                       
//DFSMSG   DD SYSOUT=*                                       
//IN       DD DSN=&T1,DISP=OLD                               
//OUT      DD SYSOUT=*                                       
//TOOLIN   DD *                                             
 SELECT FROM(IN) TO(OUT) ON(6,8,CH) ALLDUPS USING(CTL1)
//CTL1CNTL DD *                                             
  OUTFIL FNAMES=OUT,
  OUTREC=(1,5,14)
/*


Hope this helps...

Cheers

Kolusu
_________________
Kolusu - DFSORT Development Team (IBM)
DFSORT is on the Web at:
www.ibm.com/storage/dfsort

www.linkedin.com/in/kolusu
Back to top
View user's profile Send private message Send e-mail Visit poster's website
German Castillo
Beginner


Joined: 23 Dec 2005
Posts: 83
Topics: 2
Location: Caracas, Venezuela

PostPosted: Thu Feb 23, 2006 7:40 am    Post subject: Reply with quote

This is a great thread, it has given me some ideas to apply this to a multi-line message, like those in any joblog, like DB2MSTR, in which related massages are something like this

TIMESTAMP DSNXXX MESSAGE_TXT MESSAGE_ID
MESSAGE_ID MORE_MSG_TXT_1
MESSAGE_ID MORE_MSG_TXT_2
....

TIMESTAMP DSNXXX MESSAGE_TXT NEW_MESSAGE_ID
NEW_MESSAGE_ID MORE_MSG_TXT
NEW_MESSAGE_ID MORE_MSG_TXT
....

I already have a Rexx solution to do so, but it would be interesting to do it with ICETOOL, I will give it a try and keep you all posted. Two basic problems I dealt with this information while doing my initial solution:

a) Messages can be "mixed up", that is, before the ending of any message, a new one may be encountered. Simple solution to put them together, is by using the Message ID

b) The Message ID has 3 characters in it, it means that eventually they will be repeated messages, The way I dealt with it, was throuw the Timestamp of the 'Header' message
_________________
Best wishes,

German Castillo
Back to top
View user's profile Send private message
kolusu
Site Admin
Site Admin


Joined: 26 Nov 2002
Posts: 12357
Topics: 75
Location: San Jose

PostPosted: Thu Feb 23, 2006 8:51 am    Post subject: Reply with quote

German Castillo,

Is this how your data looks like?
Code:

08.23.06 xxx13513  DSNT375I  - PLAN=DSNUTIL WITH  356               
   356                     CORRELATION-ID=aaaaaa                   
   356                     CONNECTION-ID=UTILITY                     
   356                     LUW-ID=CORPNET.bbbDB2P.BE635F54E175=2521 
   356                     THREAD-INFO=SYS1GRP:*:*:*                 
   356                     IS DEADLOCKED WITH PLAN=DSNUTIL WITH     
   356                     CORRELATION-ID=cccccc                   
   356                     CONNECTION-ID=UTILITY                     
   356                     LUW-ID=xxxx.bbbDB2P.BE635F54415C=2501 
   356                     THREAD-INFO=SYS1GRP:*:*:*                 
   356                     ON MEMBER DB2P                           
   356                                       


In this 356 is the message id and then you have all the messages underneath it.

Are you looking for specific error messages ? ex: deadlocks (DSNT375I) or Db2 commands(DSNT736I ) or...?

If you are looking for specific error messages then it is easy as the message id is in a fixed position.

In case of DSNT375I the message id is at pos 52 and in case of DSNT736I it is at pos 73.

Kolusu
_________________
Kolusu - DFSORT Development Team (IBM)
DFSORT is on the Web at:
www.ibm.com/storage/dfsort

www.linkedin.com/in/kolusu
Back to top
View user's profile Send private message Send e-mail Visit poster's website
German Castillo
Beginner


Joined: 23 Dec 2005
Posts: 83
Topics: 2
Location: Caracas, Venezuela

PostPosted: Thu Feb 23, 2006 9:04 am    Post subject: Reply with quote

hI kOLOSU

Right, The messages are exactly like those. First of all, a little background, A coupple of months ago I made a generic tool to process the logs, which have those kind of messages, I wanted to make it so general that I made a file of all the messages, no particular distinction so far, since I wanted to apply the solution not only to DB2, but also to other regions like CICS and so on, at this point I just wanted to put all the relevant information in one line, which I succeded by using a rexx macro. Then using ICETOOL, I could use symbolics to process the generated dataset, all I had to do was to set the symbolics to the specific messages and our excellent toy, ICETOOL filtered, reformated, and reported out everything. I am trying to go one step further, now that I have seen your post, by getting rid of the Rexx and let ICETOOL do all the work.
_________________
Best wishes,

German Castillo
Back to top
View user's profile Send private message
German Castillo
Beginner


Joined: 23 Dec 2005
Posts: 83
Topics: 2
Location: Caracas, Venezuela

PostPosted: Thu Feb 23, 2006 9:08 am    Post subject: Reply with quote

Kolosu,

For the sake of the discussion, in the case of DB2 they were indeed deadlook and timeout (375/376) and the corresponding 500/501. But for CICS messages I could look for other messages even the ones in an abend message.
_________________
Best wishes,

German Castillo
Back to top
View user's profile Send private message
German Castillo
Beginner


Joined: 23 Dec 2005
Posts: 83
Topics: 2
Location: Caracas, Venezuela

PostPosted: Thu Feb 23, 2006 9:09 am    Post subject: Reply with quote

And since I could easily parse in the macro I didn't care about the columns position Smile
_________________
Best wishes,

German Castillo
Back to top
View user's profile Send private message
kolusu
Site Admin
Site Admin


Joined: 26 Nov 2002
Posts: 12357
Topics: 75
Location: San Jose

PostPosted: Thu Feb 23, 2006 9:21 am    Post subject: Reply with quote

German Castillo,

I think it is possible , but unfortunately my shop has syncsort which does not support the OVERLAY feature. So my control cards may be off. I assumed that you Log dataset has the following properties.

Basically we are writting the message id at the end of every record and sort on that message id to club all the message group records as one record.

Code:

LRECL=133
RECFM=FB


Code:

//STEP0100 EXEC PGM=SORT                                         
//SYSOUT   DD SYSOUT=*                                           
//SORTIN   DD DSN=YOUR INPUT LOG FILE,
//            DISP=SHR                           
//SORTOUT  DD DSN=&T1,DISP=(,CATLG),SPACE=(CYL,(X,Y),RLSE)
//SYSIN    DD *       
 INREC IFTHEN=(WHEN=(21,8,CH,EQ,C'dsnt375i'),
                OVERLAY=(134:53,3)),
        IFTHEN=(WHEN=(21,8,CH,EQ,C'dsnt376i'),
                OVERLAY=(134:53,3)),
        IFTHEN=(WHEN=NONE,
                OVERLAY=(89:5,3))
 OUTFIL OMIT=(134,3,CH,EQ,C' ')
//*
//STEP0200 EXEC PGM=SORT
//SYSOUT   DD SYSOUT=*
//SORTIN   DD DSN=&T1,
//            DISP=SHR
//SORTOUT  DD SYSOUT=*
//SYSIN    DD *       
  SORT FIELDS=(134,3,CH,A)
  OUTREC FIELDS=(1,133)


Hope this helps...

Cheers

Kolusu
_________________
Kolusu - DFSORT Development Team (IBM)
DFSORT is on the Web at:
www.ibm.com/storage/dfsort

www.linkedin.com/in/kolusu
Back to top
View user's profile Send private message Send e-mail Visit poster's website
German Castillo
Beginner


Joined: 23 Dec 2005
Posts: 83
Topics: 2
Location: Caracas, Venezuela

PostPosted: Thu Feb 23, 2006 9:56 am    Post subject: Reply with quote

I sure helps, It is different but the maintenance is a lot better. With that idea I can use symnames as well. You ar eright My previos overlay went all the way towards the end of the record. Then 'fishing' all the records is easy.

Great idea, thanks!
_________________
Best wishes,

German Castillo
Back to top
View user's profile Send private message
Frank Yaeger
Sort Forum Moderator
Sort Forum Moderator


Joined: 02 Dec 2002
Posts: 1618
Topics: 31
Location: San Jose

PostPosted: Thu Feb 23, 2006 12:32 pm    Post subject: Reply with quote

Quote:
So Frank can you kindly suggest is there any other alternative solution then the one you have send using OVERLAY, as I am not able to use either yours or Kolusu's solution for the first part of my initial query, where I am supposed to find out average time of jobs and list out only distinct jobs.


Amit,

Here's a revised version of my first job that does not use any of the Dec, 2004 DFSORT PTF functions:

Code:

//S1    EXEC  PGM=ICETOOL
//TOOLMSG   DD  SYSOUT=*
//DFSMSG   DD  SYSOUT=*
//IN DD *
JOB1        00:30:00
JOB2        00:34:00
JOB1        00:31:00
JOB3        01:00:00
/*
//T1 DD DSN=&&T1,UNIT=SYSDA,SPACE=(CYL,(5,5)),DISP=(,PASS)
//OUT DD SYSOUT=*
//TOOLIN   DD    *
SORT FROM(IN) USING(CTL1)
COPY FROM(T1) USING(CTL2)
/*
//CTL1CNTL DD *
  SORT FIELDS=(1,4,CH,A)
  OUTREC FIELDS=(1,80,81:(13,2,ZD,MUL,+3600),ADD,
      (16,2,ZD,MUL,+60),ADD,19,2,ZD,TO=ZD,LENGTH=8)
  OUTFIL FNAMES=T1,REMOVECC,NODETAIL,
   SECTIONS=(1,4,
     TRAILER3=(1,80,AVG=(81,8,ZD,M11,LENGTH=8)))
/*
//CTL2CNTL DD *
  INREC FIELDS=(1,88,89:81,8,ZD,DIV,+3600,TO=ZD,LENGTH=2,
    95:81,8,ZD,MOD,+3600,TO=ZD,LENGTH=8)
  OUTREC FIELDS=(1,90,91:95,8,ZD,DIV,+60,TO=ZD,LENGTH=2,
    93:95,8,ZD,MOD,+60,TO=ZD,LENGTH=2)
  OUTFIL FNAMES=OUT,OUTREC=(1,12,
    13:89,6,ZD,EDIT=(TT:TT:TT),
    80:X)
/*

_________________
Frank Yaeger - DFSORT Development Team (IBM)
Specialties: JOINKEYS, FINDREP, WHEN=GROUP, ICETOOL, Symbols, Migration
DFSORT is on the Web at:
www.ibm.com/storage/dfsort
Back to top
View user's profile Send private message Send e-mail Visit poster's website
Frank Yaeger
Sort Forum Moderator
Sort Forum Moderator


Joined: 02 Dec 2002
Posts: 1618
Topics: 31
Location: San Jose

PostPosted: Thu Feb 23, 2006 12:34 pm    Post subject: Reply with quote

Quote:
However I am not sure if horizontal math functions are supported with DFSORT r14 version.


Horizontal math has been supported with DFSORT R14 since Feb, 2003.
_________________
Frank Yaeger - DFSORT Development Team (IBM)
Specialties: JOINKEYS, FINDREP, WHEN=GROUP, ICETOOL, Symbols, Migration
DFSORT is on the Web at:
www.ibm.com/storage/dfsort
Back to top
View user's profile Send private message Send e-mail Visit poster's website
Frank Yaeger
Sort Forum Moderator
Sort Forum Moderator


Joined: 02 Dec 2002
Posts: 1618
Topics: 31
Location: San Jose

PostPosted: Thu Feb 23, 2006 12:46 pm    Post subject: Reply with quote

Code:

 INREC IFTHEN=(WHEN=(1,1,CH,EQ,C'dsnt375i'),
                OVERLAY=(134:53,3)),
        IFTHEN=(WHEN=(1,1,CH,EQ,C'dsnt376i'),
                OVERLAY=(134:53,3)),


Kolusu,

I can't figure out what you're trying to do here since I don't know what the input records look like or what the output records are supposed to look like, but the 1,1,CH, in the tests certainly don't look right to me. This would test for 'd' in position 1. I'm pretty sure you want 8 for the length to correspond to the length of 'dsnt37xi', but I don't know if the starting position of 1 is also wrong or not.

If the idea is to do some kind of group INCLUDE, then the "Include or omit groups of records" Smart DFSORT Trick may be relevant for those who have DFSORT and the Dec, 2004 PTF. See:

http://www.ibm.com/servers/storage/support/software/sort/mvs/tricks/
_________________
Frank Yaeger - DFSORT Development Team (IBM)
Specialties: JOINKEYS, FINDREP, WHEN=GROUP, ICETOOL, Symbols, Migration
DFSORT is on the Web at:
www.ibm.com/storage/dfsort


Last edited by Frank Yaeger on Mon Sep 04, 2006 11:54 am; edited 1 time in total
Back to top
View user's profile Send private message Send e-mail Visit poster's website
kolusu
Site Admin
Site Admin


Joined: 26 Nov 2002
Posts: 12357
Topics: 75
Location: San Jose

PostPosted: Thu Feb 23, 2006 12:56 pm    Post subject: Reply with quote

Quote:

I can't figure out what you're trying to do here since I don't know what the input records look like or what the output records are supposed to look like, but the 1,1,CH, in the tests certainly don't look right to me.



Frank,

I just copied the control cards from one of your example. I changed the string but forgot to change the positions and length. Sad It should be at pos 21 for a length of 8 bytes. I just showed castilo an example control cards and I guess he was able to build on that.

Kolusu
_________________
Kolusu - DFSORT Development Team (IBM)
DFSORT is on the Web at:
www.ibm.com/storage/dfsort

www.linkedin.com/in/kolusu
Back to top
View user's profile Send private message Send e-mail Visit poster's website
Frank Yaeger
Sort Forum Moderator
Sort Forum Moderator


Joined: 02 Dec 2002
Posts: 1618
Topics: 31
Location: San Jose

PostPosted: Thu Feb 23, 2006 2:04 pm    Post subject: Reply with quote

Kolusu,

Ok. I editted your post to have the correct starting position and length.
_________________
Frank Yaeger - DFSORT Development Team (IBM)
Specialties: JOINKEYS, FINDREP, WHEN=GROUP, ICETOOL, Symbols, Migration
DFSORT is on the Web at:
www.ibm.com/storage/dfsort
Back to top
View user's profile Send private message Send e-mail Visit poster's website
amit4u79
Beginner


Joined: 24 Oct 2005
Posts: 109
Topics: 36
Location: India

PostPosted: Fri Feb 24, 2006 2:17 am    Post subject: Reply with quote

Hi Frank/Kolusu, Guys you are just genius.

Frank your solution worked perfectly for me. just wanted to ask a simple query, if suppose there is a record which might have a blank in place of Job name or there might be a record which might not have time under the time taken column, i would like to omit such records from processing, so how can I use OMIT condition to only select records which are 'non-blank' for column 1,4 and 13,8. The idea is to remove unnecessary records, and process only required records.

Kolusu, for your solution of the VBA 137 SYSLOG data set, I am sorting and trying to write it to an output VBA 137 data set, but the SYSLOG has some records which are less than 63 in length and so I get the below error;

ICE218A 3 61 BYTE VARIABLE RECORD IS SHORTER THAN 63 BYTE MINIMUM

I searched SORT Manual and found I can use VLFILL, so I put the below SORT card but even then it gives the same error:

//SYSIN DD *
SORT FIELDS=COPY
INREC FIELDS=(1,5,6:63,1,CHANGE=(1,C'*',C'1'),NOMATCH=(C'0'),
SEQNUM,8,ZD,6)
OUTFIL OUTREC=(1,5,
6,1,ZD,ADD,7,8,ZD,EDIT=(TTTTTTTT),15),VLFILL=C'X'
/*

I know we can use VLFILL only with OUTFIL statement and that is what I did, dont know why still encountering same issue.

Kindly help gentlemen.

Regards,
- Amit.
_________________
I did not fail; I have found 10,000 ways that would not work - Albert Einstein.
Back to top
View user's profile Send private message Send e-mail Yahoo Messenger
kolusu
Site Admin
Site Admin


Joined: 26 Nov 2002
Posts: 12357
Topics: 75
Location: San Jose

PostPosted: Fri Feb 24, 2006 6:17 am    Post subject: Reply with quote

Quote:

so how can I use OMIT condition to only select records which are 'non-blank' for column 1,4 and 13,8. The idea is to remove unnecessary records, and process only required records.


you can add an OMIT cond to CTL1cntl as follows

Code:

//CTL1CNTL DD *
  OMIT COND=(01,04,CH,EQ,C' ',OR,
             13,08,CH,EQ,C' ')
  SORT FIELDS=(1,4,CH,A)
  OUTREC FIELDS=(1,80,81:(13,2,ZD,MUL,+3600),ADD,
      (16,2,ZD,MUL,+60),ADD,19,2,ZD,TO=ZD,LENGTH=8)
  OUTFIL FNAMES=T1,REMOVECC,NODETAIL,
   SECTIONS=(1,4,
     TRAILER3=(1,80,AVG=(81,8,ZD,M11,LENGTH=8)))
/*


Quote:

Kolusu, for your solution of the VBA 137 SYSLOG data set, I am sorting and trying to write it to an output VBA 137 data set, but the SYSLOG has some records which are less than 63 in length and so I get the below error;

ICE218A 3 61 BYTE VARIABLE RECORD IS SHORTER THAN 63 BYTE MINIMUM


Try this .

Code:

//SYSIN    DD *
  OPTION VLSCMP
  SORT FIELDS=COPY                                           
  INREC FIELDS=(1,5,                                         
                6:63,1,CHANGE=(1,C'*',C'1'),NOMATCH=(C'0'),
                SEQNUM,8,ZD,6)                               
  OUTREC FIELDS=(1,5,                                       
                 6,1,ZD,ADD,7,8,ZD,EDIT=(TTTTTTTT),15)       
/*   



Hope this helps...

Cheers

Kolusu
_________________
Kolusu - DFSORT Development Team (IBM)
DFSORT is on the Web at:
www.ibm.com/storage/dfsort

www.linkedin.com/in/kolusu
Back to top
View user's profile Send private message Send e-mail Visit poster's website
Display posts from previous:   
Post new topic   Reply to topic   printer-friendly view    MVSFORUMS.com Forum Index -> Utilities All times are GMT - 5 Hours
Goto page Previous  1, 2, 3  Next
Page 2 of 3

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum


MVSFORUMS
Powered by phpBB © 2001, 2005 phpBB Group