MVSFORUMS.com Forum Index MVSFORUMS.com
A Community of and for MVS Professionals
 
 FAQFAQ   SearchSearch   Quick Manuals   RegisterRegister 
 ProfileProfile   Log in to check your private messagesLog in to check your private messages   Log inLog in 

DFSort performace improvement in Sorting a VSAM file
Goto page 1, 2  Next
 
Post new topic   Reply to topic   printer-friendly view    MVSFORUMS.com Forum Index -> Utilities
View previous topic :: View next topic  
Author Message
Magesh_J
Intermediate


Joined: 21 Jun 2014
Posts: 259
Topics: 54

PostPosted: Mon Jun 23, 2014 1:48 pm    Post subject: DFSort performace improvement in Sorting a VSAM file Reply with quote

Hi,

I have a dfsort step taking 1hr 30 minutes. Need help/advice on optimizing the same.

Code:

//SORTIN DD DSN=VSAMFILE,DISP=SHR,AMP='BUFNI=24,BUFND=08'
//SORTOUT DD DSN=FLATFILE,DISP=SHR
//SYSIN DD *
   SORT FIELDS=(1,26,CH,A)             
   INCLUDE COND=(1,1,CH,EQ,C'R')   
/*
//SORTWK01 DD UNIT=SYSDA,SPACE=(CYL,(100,25))
//SORTWK02 DD UNIT=SYSDA,SPACE=(CYL,(100,25))
//SORTWK03 DD UNIT=SYSDA,SPACE=(CYL,(100,25))
//SORTWK04 DD UNIT=SYSDA,SPACE=(CYL,(100,25))
//SORTWK05 DD UNIT=SYSDA,SPACE=(CYL,(100,25))
//SORTWK06 DD UNIT=SYSDA,SPACE=(CYL,(100,25))


Code:

******************************** Top of Data **************************************** Top of Data *********************
SARPAGE 5              DS=SYSOUT  .STEP01  .PSTEP01 /0000103                                                           
ICE143I 0 BLOCKSET     SORT  TECHNIQUE SELECTED                                                                       
ICE250I 0 VISIT http://www.ibm.com/storage/dfsort FOR DFSORT PAPERS, EXAMPLES AND MORE                                 
ICE000I 1 - CONTROL STATEMENTS FOR XXXX-A01, Z/OS DFSORT V1R10 - 21:00 ON TUE JUN 10, 2014 -                           
           SORT FIELDS=(1,26,CH,A)                                             10000
           INCLUDE COND=(1,1,CH,EQ,C'R')                                       20000
ICE201I H RECORD TYPE IS F - DATA STARTS IN POSITION 1                                                                 
ICE751I 0 C5-K90025 C6-K90025 C7-K54603 C8-K62201 EE-K51707 E4-K51707 C9-BASE   E5-K62201 E7-K62201                   
ICE193I 0 ICEAM1 INVOCATION ENVIRONMENT IN EFFECT - ICEAM1 ENVIRONMENT SELECTED                                       
ICE088I 1 ZZZZZH1 .STEP01  .PSTEP01 , INPUT LRECL = 612, BLKSIZE = 22528, TYPE=   F
ICE093I 0 MAIN STORAGE = (MAX,67108864,67108864)                                                                       
ICE156I 0 MAIN STORAGE ABOVE 16MB = (67051504,67051504)                                                               
ICE127I 0 OPTIONS: OVFLO=RC0 ,PAD=RC0 ,TRUNC=RC0 ,SPANINC=RC16,VLSCMP=N,SZERO=Y,RESET=Y,VSAMEMT=Y,DYNSPC=256           
ICE128I 0 OPTIONS: SIZE=67108864,MAXLIM=2097152,MINLIM=450560,EQUALS=Y,LIST=Y,ERET=RC16 ,MSGDDN=SYSOUT                 
ICE129I 0 OPTIONS: VIO=N,RESDNT=ALL ,SMF=NO   ,WRKSEC=Y,OUTSEC=Y,VERIFY=N,CHALT=N,DYNALOC=N             ,ABCODE=MSG
ICE130I 0 OPTIONS: RESALL=8192,RESINV=0,SVC=109 ,CHECK=Y,WRKREL=Y,OUTREL=Y,CKPT=N,COBEXIT=COB2
ICE131I 0 OPTIONS: TMAXLIM=6291456,ARESALL=0,ARESINV=0,OVERRGN=65536,CINV=Y,CFW Y,DSA=64
ICE132I 0 OPTIONS: VLSHRT=N,ZDPRINT=Y,IEXIT=N,TEXIT=N,LISTX=N,EFS=NONE    ,EXITCK=S,PARMDDN=DFSPARM ,FSZEST=N         
ICE133I 0 OPTIONS: HIPRMAX=OPTIMAL,DSPSIZE=MAX ,ODMAXBF=0,SOLRF=Y,VLLONG=N,VSAMIO=N,MOSIZE=MAX                         
ICE235I 0 OPTIONS: NULLOUT=RC0                                           
ICE084I 0 EXCP ACCESS METHOD USED FOR SORTOUT                           
ICE084I 1 VSAM ACCESS METHOD USED FOR SORTIN                             
ICE750I 0 DC 60857245116 TC 0 CS DSVVV KSZ 30 VSZ 30                     
ICE752I 0 FSZ=99439943 RC  IGN=0 E  AVG=616 0  WSP=79559723 C  DYN=0 0   
ICE751I 1 DE-K61785 D5-K62201 D3-BASE   ED-BASE   E8-K61438             
ICE090I 0 OUTPUT LRECL = 612, BLKSIZE = 27540, TYPE = FB                 
ICE055I 0 INSERT 0, DELETE 97575885                                     
ICE054I 0 RECORDS - IN: 99439943, OUT: 1864058                           
ICE134I 0 NUMBER OF BYTES SORTED: 1140803496                             
ICE253I 0 RECORDS SORTED - PROCESSED: 1864058, EXPECTED: 99439943       
ICE165I 0 TOTAL WORK DATA SET TRACKS ALLOCATED: 9000 , TRACKS USED: 0   
ICE199I 0 MEMORY OBJECT STORAGE USED = 0M BYTES                         
ICE180I 0 HIPERSPACE STORAGE USED = 1132440K BYTES                       
ICE188I 0 DATA SPACE STORAGE USED = 0K BYTES                             
ICE052I 0 END OF DFSORT 


I could see AMP parameter, having more index buffer, may I use
AMP=('ACCBIAS=SO') for DFSORT ?
If I use option DYNALLOC=(,16),FILSZ=E120000000, will there be a improvement

Thanks
Magesh
Back to top
View user's profile Send private message
kolusu
Site Admin
Site Admin


Joined: 26 Nov 2002
Posts: 12378
Topics: 75
Location: San Jose

PostPosted: Mon Jun 23, 2014 4:01 pm    Post subject: Reply with quote

Magesh_J,

Just so you know DFSORT V1R10 is out of service.

1. How did you arrive at the BUFNI and BUFND numbers?
2. Can you run a listcat on the vsam input?

Code:

//STEP0100 EXEC PGM=IDCAMS                 
//SYSPRINT DD SYSOUT=*                     
//SYSIN    DD *                             
  LISTCAT ENT('Your Vsam file') ALL   
//* 


3. Please add the following to your job so I can see the diagnostic mesages:

//SORTDIAG DD DUMMY

Then rerun the job and send the complete JES log including the //SYSOUT messages and the listcat output to dfsort@us.ibm.com


Thanks,
_________________
Kolusu
www.linkedin.com/in/kolusu
Back to top
View user's profile Send private message Send e-mail Visit poster's website
Magesh_J
Intermediate


Joined: 21 Jun 2014
Posts: 259
Topics: 54

PostPosted: Mon Jun 23, 2014 5:28 pm    Post subject: Reply with quote

Thanks Kolusu for the kind response.

I have some questions.

Will DFSORT uses AMP parameter passed externally, or it will take its own Buffers ?

I need to use FILSZ for the number of input records or for the number of records processed ?
in our case processed record is 1864058
and input records are 99439943

I tried with FILSZ=E1864058, which is working fine, Please advice the best practice.

May I mask all the account/application information and send it to dfsort@us.ibm.com or do we need to send exact information ?

Do I need to send it from official email or from personal email ?

Because if I need to send exact info or from official email, I need to get prior approval.

Thanks
Magesh
Back to top
View user's profile Send private message
kolusu
Site Admin
Site Admin


Joined: 26 Nov 2002
Posts: 12378
Topics: 75
Location: San Jose

PostPosted: Mon Jun 23, 2014 5:41 pm    Post subject: Reply with quote

Magesh_J wrote:
Thanks Kolusu for the kind response.

I have some questions.

Will DFSORT uses AMP parameter passed externally, or it will take its own Buffers ?


DFSORT reads the VSAM file sequentially, so BUFNI really does nothing to help in your case.

Magesh_J wrote:

I need to use FILSZ for the number of input records or for the number of records processed ?
in our case processed record is 1864058
and input records are 99439943
I tried with FILSZ=E1864058, which is working fine, Please advice the best practice.



Passing an ESTIMATED file size ONLY helps when DFSORT is unable to determine the number of records (ex: program invoked sorts..) but in this case we are able to determine the file size so we you really don't have to pass the estimated file size.

Magesh_J wrote:

May I mask all the account/application information and send it to dfsort@us.ibm.com or do we need to send exact information ?
Do I need to send it from official email or from personal email ?
Because if I need to send exact info or from official email, I need to get prior approval.



You can mask the account information if you want to. All we need is the DFSORT diagnostic messages. It doesn't matter from which email id you send. As Long as you can receive the information you are good.

Before you send the sysout to DFSORT hotline, how about you run another test with the following. Based on ICE088I message, I am guessing that your CI size is 22528 which means that we can get 33 CIs per CA. So increase the BUFND to 34 which would allow DFSORT to read a full cylinder at a time.

Code:

//SORTIN DD DSN=VSAMFILE,DISP=SHR,AMP='BUFND=34'


send us the sysout from both runs.
_________________
Kolusu
www.linkedin.com/in/kolusu
Back to top
View user's profile Send private message Send e-mail Visit poster's website
kolusu
Site Admin
Site Admin


Joined: 26 Nov 2002
Posts: 12378
Topics: 75
Location: San Jose

PostPosted: Mon Jun 23, 2014 5:48 pm    Post subject: Reply with quote

Also I noticed that your INCLUDE is only selecting records with "R" in position 1. So you can skip sorting that field as it is a constant for all records. It cuts the sorting key by a byte.

Code:

//SYSIN DD *
   INCLUDE COND=(1,1,CH,EQ,C'R')
   SORT FIELDS=(2,25,CH,A)               
//*

_________________
Kolusu
www.linkedin.com/in/kolusu
Back to top
View user's profile Send private message Send e-mail Visit poster's website
Magesh_J
Intermediate


Joined: 21 Jun 2014
Posts: 259
Topics: 54

PostPosted: Mon Jun 23, 2014 6:35 pm    Post subject: Reply with quote

Changed code as per your advice.
Code:

//RSLTSORT EXEC PGM=SORT                                     
//SORTIN   DD  DSN=VSAMFILE,DISP=SHR,     
//         AMP='BUFND=34'                                   
//SORTOUT  DD  DSN=FLATFILE,           
//             DISP=(,CATLG,DELETE),                         
//             UNIT=SYSDA,                                   
//             SPACE=(CYL,(350,70),RLSE),                   
//             DCB=(DSORG=PS,RECFM=FB,LRECL=612,BLKSIZE=0)   
//SYSOUT DD SYSOUT=*                                         
//SORTDIAG DD DUMMY                                         
//SYSIN    DD  *                                             
 INCLUDE COND=(1,1,CH,EQ,C'R')                               
 SORT FIELDS=(1,26,CH,A)                                     
 OPTION DYNALLOC=(,16)                                       
/*                                                           
//STEP0100 EXEC PGM=IDCAMS                                   
//SYSPRINT DD SYSOUT=*                                       
//SYSIN    DD *                                             
   LISTCAT ENT('VSAMFILE') ALL   
/*                                 


Code:

1ICE805I 1 JOBNAME: YYYYYYTP , STEPNAME: RSLTSORT
 ICE802I 0 BLOCKSET     TECHNIQUE IN CONTROL
 ICE201I H RECORD TYPE IS F - DATA STARTS IN POSITION 1
 ICE992I 0 RA 0 WR 0 TR 1
 ICE751I 0 C5-K90025 C6-K90025 C7-K54603 C8-K62201 EE-K51707 E4-K51707 C9-BASE   E5-K62201 E6-K51707 C4-K62201 E7-K62201
 ICE143I 0 BLOCKSET     SORT  TECHNIQUE SELECTED
 ICE250I 0 VISIT http://www.ibm.com/storage/dfsort FOR DFSORT PAPERS, EXAMPLES AND MORE
 ICE000I 1 - CONTROL STATEMENTS FOR XXXX-A01, Z/OS DFSORT V1R10 - 15:50 ON MON JUN 23, 2014 -
0           INCLUDE COND=(1,1,CH,EQ,C'R')
            SORT FIELDS=(1,26,CH,A)
            OPTION DYNALLOC=(,16)
 ICE201I H RECORD TYPE IS F - DATA STARTS IN POSITION 1
 ICE992I 0 RA 0 WR 0 TR 1
 ICE751I 0 C5-K90025 C6-K90025 C7-K54603 C8-K62201 EE-K51707 E4-K51707 C9-BASE   E5-K62201 E6-K51707 C4-K62201 E7-K62201
 ICE193I 0 ICEAM1 INVOCATION ENVIRONMENT IN EFFECT - ICEAM1 ENVIRONMENT SELECTED
 ICE088I 1 YYYYYYTP.RSLTSORT.        , INPUT LRECL = 612, BLKSIZE = 22528, TYPE = F
 ICE093I 0 MAIN STORAGE = (MAX,67108864,67108864)
 ICE156I 0 MAIN STORAGE ABOVE 16MB = (67051504,67051504)
 ICE127I 0 OPTIONS: OVFLO=RC0 ,PAD=RC0 ,TRUNC=RC0 ,SPANINC=RC16,VLSCMP=N,SZERO=Y,RESET=Y,VSAMEMT=Y,DYNSPC=256
 ICE128I 0 OPTIONS: SIZE=67108864,MAXLIM=2097152,MINLIM=450560,EQUALS=Y,LIST=Y,ERET=RC16 ,MSGDDN=SYSOUT
 ICE129I 0 OPTIONS: VIO=N,RESDNT=ALL ,SMF=NO   ,WRKSEC=Y,OUTSEC=Y,VERIFY=N,CHALT=N,DYNALOC=(SYSDA   ,016),ABCODE=MSG
 ICE130I 0 OPTIONS: RESALL=8192,RESINV=0,SVC=109 ,CHECK=Y,WRKREL=Y,OUTREL=Y,CKPT=N,COBEXIT=COB2
 ICE131I 0 OPTIONS: TMAXLIM=6291456,ARESALL=0,ARESINV=0,OVERRGN=65536,CINV=Y,CFW=Y,DSA=64
 ICE132I 0 OPTIONS: VLSHRT=N,ZDPRINT=Y,IEXIT=N,TEXIT=N,LISTX=N,EFS=NONE    ,EXITCK=S,PARMDDN=DFSPARM ,FSZEST=N
 ICE133I 0 OPTIONS: HIPRMAX=OPTIMAL,DSPSIZE=MAX ,ODMAXBF=0,SOLRF=Y,VLLONG=N,VSAMIO=N,MOSIZE=MAX
 ICE235I 0 OPTIONS: NULLOUT=RC0
 ICE084I 0 EXCP ACCESS METHOD USED FOR SORTOUT
 ICE084I 1 VSAM ACCESS METHOD USED FOR SORTIN
 ICE750I 0 DC 60999634260 TC 0 CS DSVVV KSZ 30 VSZ 30
 ICE887I 0 CSES 0,0,0 ES 0,0,0
 ICE886I 0 SYS 0 TSTG 0 FS 0 INIT 0 MAX 0 LEN 0
 ICE752I 0 FSZ=99672605 RC  IGN=0 E  AVG=616 0  WSP=79745871 C  DYN=1403055 53216
 ICE915I 0 MOFSZ=60075,MOSZ=0,MOSYS=200(2),MOSTG=6738,MEML=200(1)
 ICE916I 0 MOFR=0104,MOVR=VV
 ICE996I 0 ESM=2568704,ESO=2568704,ESR=0,ESP=4096,ESS=16384,CES=2568704,HSZ=524288
 ICE997I 0 HWSP=37474563,HMAX=1725069,HES=1725069,ASV=1725069,EQ=I2,HN=1
 ICE898I 0 OMAX=553150,NMAX=1725069,ENQT=1725069,CMAX=524288,HU=99,BUN=53216,MD=M3,DU=90,DR=0,HN=1
 ICE880I 0 QP=129 QA=129 HI=422 LI=294 MI=422 TZ=524941 N1=524288 N2=524288 SZ=129 HN=1
 ICE889I 0 CT=MAX     , SB=8, L=0, D=0000, CCW=1MAM
 ICE901I 0 W 04NP17 0FNP17 0DNP17 05NP17 0CNP17 10NP17 0ANP17 03NP17
 ICE901I 0 W 02NP17 07NP17 0BNP17 08NP17 09NP17 06NP17 01PP17 0ENP13
 ICE902I 0 O NP10  I PP10
 ICE751I 1 DE-K61785 D5-K62201 D3-BASE   ED-BASE   E8-K61438
 ICE900I 0 CON=1,MUV=0,VOL=1,ENU=0,SBK=0,SRC=0,VEM=0
 ICE999I 0 PWK=15 PSP=1403055 SWK=0 SSP=0 TWK=0 TSP=0 RWK=0 RSP=0 AWK=15 AWP=1403055
 ICE090I 0 OUTPUT LRECL = 612, BLKSIZE = 27540, TYPE = FB
 ICE055I 0 INSERT 0, DELETE 97833227
 ICE054I 0 RECORDS - IN: 99701740, OUT: 1868513
 ICE134I 0 NUMBER OF BYTES SORTED: 1143529956
 ICE253I 0 RECORDS SORTED - PROCESSED: 1868513, EXPECTED: 99672605
 ICE165I 0 TOTAL WORK DATA SET TRACKS ALLOCATED: 450555 , TRACKS USED: 0
 ICE199I 0 MEMORY OBJECT STORAGE USED = 0M BYTES
 ICE180I 0 HIPERSPACE STORAGE USED = 1135092K BYTES
 ICE188I 0 DATA SPACE STORAGE USED = 0K BYTES
 ICE891I 1 67099336 WMAIN, 9528 CMAIN, MAX CALLOC, N SCN, B BA, 0 AZ, 0 BZ, NN QC, 0 CZ, 0 DZ, 1 PLE
 ICE892I 1 612 RIN  612 BLI  27540 BLO  616 RUN  53216 BUN  2818 CPU  00 CVC
 ICE893I 1 200 XIN 14 WIN 104721 GIN NDEY PFP80 B00 CMV00 CIC UPTH LMI VSI  RUX
 ICE894I 0 1 STR  1 MOR  867 IPB  747 OPB  0 CYL  O   MN
1ICE881I 0 EQ=I2 DX=0 D2=58 D3=69 D4=2 AS=0 SA=0 SB=0 SC=0 HN=1
 ICE885I 0 DAT 00 DSR 010B BINS 0 BSZ 0 RCP 0 AJC 0 RLC 0 DUNIT 0
 ICE895I 0 202 MUNIT  42 SUNIT  31 OUNIT
 ICE896I 0 101 SET  100 DEXTOT  0 BLK   99672605 CSZ  31 WE
 ICE804I 1 SORTWK10 EXCP COUNT: 2
 ICE804I 1 SORTWK0F EXCP COUNT: 2
 ICE804I 1 SORTWK0E EXCP COUNT: 1
 ICE804I 1 SORTWK0D EXCP COUNT: 2
 ICE804I 1 SORTWK0C EXCP COUNT: 2
 ICE804I 1 SORTWK0B EXCP COUNT: 2
 ICE804I 1 SORTWK0A EXCP COUNT: 2
 ICE804I 1 SORTWK09 EXCP COUNT: 2
 ICE804I 1 SORTWK08 EXCP COUNT: 2
 ICE804I 1 SORTWK07 EXCP COUNT: 2
 ICE804I 1 SORTWK06 EXCP COUNT: 2
 ICE804I 1 SORTWK05 EXCP COUNT: 2
 ICE804I 1 SORTWK04 EXCP COUNT: 2
 ICE804I 1 SORTWK03 EXCP COUNT: 2
 ICE804I 1 SORTWK02 EXCP COUNT: 2
 ICE804I 1 SORTWK01 EXCP COUNT: 2
 ICE804I 1 SORTOUT  EXCP COUNT: 66
 ICE899I 0 HSR=551,HSW=1554,HRE=21751,HWE=1576,HRP=282763,HWP=282763,HWM=283773,HNM=1
 ICE052I 0 END OF DFSORT


Code:

1IDCAMS  SYSTEM SERVICES                                           TIME: 15:56:46        06/23/14     PAGE      1
0
    LISTCAT ENT('VSAMFILE') ALL
0CLUSTER ------- VSAMFILE
      IN-CAT --- CATALOG.VOLUXXX
      HISTORY
        DATASET-OWNER-----(NULL)     CREATION--------2014.172
        RELEASE----------------2     EXPIRATION------0000.000
      SMSDATA
        STORAGECLASS ---STANDARD     MANAGEMENTCLASS-PRODVSAM
        DATACLASS ------XTNDADDR     LBACKUP ---0000.000.0000
        EATTR-------------(NULL)
        BWO STATUS------00000000     BWO TIMESTAMP---00000 00:00:00.0
        BWO---------------(NULL)
      RLSDATA
        LOG ----------------(NULL)   RECOVERY REQUIRED --(NO)     FRLOG ------------(NULL)
        VSAM QUIESCED -------(NO)    RLS IN USE ---------(NO)
0       LOGSTREAMID-----------------------------(NULL)
        RECOVERY TIMESTAMP LOCAL-----X'0000000000000000'
        RECOVERY TIMESTAMP GMT-------X'0000000000000000'
      PROTECTION-PSWD-----(NULL)     RACF----------------(NO)
      ASSOCIATIONS
        DATA-----VSAMFILE.ZZZZZZZZ.DATA
        INDEX----VSAMFILE.ZZZZZZZZ.INDX
0   DATA ------- VSAMFILE.ZZZZZZZZ.DATA
      IN-CAT --- CATALOG.VOLUXXX
      HISTORY
        DATASET-OWNER-----(NULL)     CREATION--------2014.172
        RELEASE----------------2     EXPIRATION------0000.000
        ACCOUNT-INFO-----------------------------------(NULL)
      PROTECTION-PSWD-----(NULL)     RACF----------------(NO)
      ASSOCIATIONS
        CLUSTER--VSAMFILE
      ATTRIBUTES
        KEYLEN----------------27     AVGLRECL--------------80     BUFSPACE-----------49152     CISIZE-------------22528
        RKP--------------------0     MAXLRECL-------------612     EXCPEXIT----------(NULL)     CI/CA-----------------33
        STRIPE-COUNT-----------1
        ACT-DIC-TOKEN----X'4000000A01000540070208FE05FE0DFE0EFE0AFE00000000000000000000000000000000'
        SHROPTNS(2,3)      SPEED     UNIQUE           NOERASE     INDEXED       NOWRITECHK     NOIMBED       NOREPLICAT
        UNORDERED        NOREUSE     NONSPANNED    COMP-FORMT     EXTENDED        EXT-ADDR
      STATISTICS
        REC-TOTAL-------99672605     SPLITS-CI--------------0     EXCPS-------------238518
        REC-DELETED------------0     SPLITS-CA--------------0     EXTENTS----------------3
        REC-INSERTED--------5854     FREESPACE-%CI---------15     SYSTEM-TIMESTAMP:
        REC-UPDATED------------0     FREESPACE-%CA---------10          X'CD586AE157189D08'
        REC-RETRIEVED--299145797     FREESPC-------1523185664
        USER-DATA-SIZE----------------------------24244909162     COMP-USER-DATA-SIZE------------------------9794693864
      ALLOCATION
        SPACE-TYPE------CYLINDER     HI-A-RBA-----13381632000
1IDCAMS  SYSTEM SERVICES                                           TIME: 15:56:46        06/23/14     PAGE      2
0       SPACE-PRI-----------1000     HI-U-RBA-----13044860928
        SPACE-SEC-----------1000
      VOLUME
        VOLSER------------VVSM08     PHYREC-SIZE---------5632     HI-A-RBA-----13381632000     EXTENT-NUMBER----------3
        DEVTYPE------X'3010200F'     PHYRECS/TRK------------9     HI-U-RBA-----13044860928     EXTENT-TYPE--------X'40'
        VOLFLAG------------PRIME     TRACKS/CA-------------15
        EXTENTS:
        LOW-CCHH-----X'00BE0000'     LOW-RBA----------------0     TRACKS-------------15000
        HIGH-CCHH----X'04A5000E'     HIGH-RBA-------743423999
        LOW-CCHH-----X'04E20000'     LOW-RBA--------743424000     TRACKS-------------45000
        HIGH-CCHH----X'1099000E'     HIGH-RBA------2973695999
        LOW-CCHH-----X'38E10000'     LOW-RBA-------2973696000     TRACKS------------210000
        HIGH-CCHH----X'6F90000E'     HIGH-RBA-----13381631999
      VOLUME
        VOLSER-----------------*     PHYREC-SIZE------------0     HI-A-RBA---------------0     EXTENT-NUMBER----------0
        DEVTYPE------X'3010200F'     PHYRECS/TRK------------0     HI-U-RBA---------------0     EXTENT-TYPE--------X'FF'
        VOLFLAG--------CANDIDATE     TRACKS/CA--------------0
      VOLUME
        VOLSER-----------------*     PHYREC-SIZE------------0     HI-A-RBA---------------0     EXTENT-NUMBER----------0
        DEVTYPE------X'3010200F'     PHYRECS/TRK------------0     HI-U-RBA---------------0     EXTENT-TYPE--------X'FF'
        VOLFLAG--------CANDIDATE     TRACKS/CA--------------0
      VOLUME
        VOLSER-----------------*     PHYREC-SIZE------------0     HI-A-RBA---------------0     EXTENT-NUMBER----------0
        DEVTYPE------X'3010200F'     PHYRECS/TRK------------0     HI-U-RBA---------------0     EXTENT-TYPE--------X'FF'
        VOLFLAG--------CANDIDATE     TRACKS/CA--------------0
0   INDEX ------ VSAMFILE.ZZZZZZZZ.INDX
      IN-CAT --- CATALOG.VOLUXXX
      HISTORY
        DATASET-OWNER-----(NULL)     CREATION--------2014.172
        RELEASE----------------2     EXPIRATION------0000.000
      PROTECTION-PSWD-----(NULL)     RACF----------------(NO)
      ASSOCIATIONS
        CLUSTER--VSAMFILE
      ATTRIBUTES
        KEYLEN----------------27     AVGLRECL---------------0     BUFSPACE---------------0     CISIZE--------------4096
        RKP--------------------0     MAXLRECL------------4089     EXCPEXIT----------(NULL)     CI/CA-----------------12
        SHROPTNS(2,3)   RECOVERY     UNIQUE           NOERASE     NOWRITECHK       NOIMBED     NOREPLICAT     UNORDERED
        NOREUSE         EXTENDED     EXT-ADDR
      STATISTICS
        REC-TOTAL----------17611     SPLITS-CI--------------0     EXCPS-------------145969     INDEX:
        REC-DELETED------------0     SPLITS-CA--------------0     EXTENTS----------------2     LEVELS-----------------3
        REC-INSERTED-----------0     FREESPACE-%CI----------0     SYSTEM-TIMESTAMP:            ENTRIES/SECT-----------5
        REC-UPDATED------------0     FREESPACE-%CA----------0          X'CD586AE157189D08' SEQ-SET-RBA----------------0
        REC-RETRIEVED----------0     FREESPC----------1593344                              HI-LEVEL-RBA---------1245184
      ALLOCATION
        SPACE-TYPE------CYLINDER     HI-A-RBA--------73728000
        SPACE-PRI-------------60     HI-U-RBA--------72134656
        SPACE-SEC-------------20
      VOLUME
        VOLSER------------VVSM08     PHYREC-SIZE---------4096     HI-A-RBA--------73728000     EXTENT-NUMBER----------2
1IDCAMS  SYSTEM SERVICES                                           TIME: 15:56:46        06/23/14     PAGE      3
0       DEVTYPE------X'3010200F'     PHYRECS/TRK-----------12     HI-U-RBA--------72134656     EXTENT-TYPE--------X'40'
        VOLFLAG------------PRIME     TRACKS/CA--------------1
        EXTENTS:
        LOW-CCHH-----X'04A60000'     LOW-RBA----------------0     TRACKS---------------900
        HIGH-CCHH----X'04E1000E'     HIGH-RBA--------44236799
        LOW-CCHH-----X'109A0000'     LOW-RBA---------44236800     TRACKS---------------600
        HIGH-CCHH----X'10C1000E'     HIGH-RBA--------73727999
      VOLUME
        VOLSER-----------------*     PHYREC-SIZE------------0     HI-A-RBA---------------0     EXTENT-NUMBER----------0
        DEVTYPE------X'3010200F'     PHYRECS/TRK------------0     HI-U-RBA---------------0     EXTENT-TYPE--------X'FF'
        VOLFLAG--------CANDIDATE     TRACKS/CA--------------0
      VOLUME
        VOLSER-----------------*     PHYREC-SIZE------------0     HI-A-RBA---------------0     EXTENT-NUMBER----------0
        DEVTYPE------X'3010200F'     PHYRECS/TRK------------0     HI-U-RBA---------------0     EXTENT-TYPE--------X'FF'
        VOLFLAG--------CANDIDATE     TRACKS/CA--------------0
      VOLUME
        VOLSER-----------------*     PHYREC-SIZE------------0     HI-A-RBA---------------0     EXTENT-NUMBER----------0
        DEVTYPE------X'3010200F'     PHYRECS/TRK------------0     HI-U-RBA---------------0     EXTENT-TYPE--------X'FF'
        VOLFLAG--------CANDIDATE     TRACKS/CA--------------0
1IDCAMS  SYSTEM SERVICES                                           TIME: 15:56:46        06/23/14     PAGE      4
0         THE NUMBER OF ENTRIES PROCESSED WAS:
                    AIX -------------------0
                    ALIAS -----------------0
                    CLUSTER ---------------1
                    DATA ------------------1
                    GDG -------------------0
                    INDEX -----------------1
                    NONVSAM ---------------0
                    PAGESPACE -------------0
                    PATH ------------------0
                    SPACE -----------------0
                    USERCATALOG -----------0
                    TAPELIBRARY -----------0
                    TAPEVOLUME ------------0
                    TOTAL -----------------3
0         THE NUMBER OF PROTECTED ENTRIES SUPPRESSED WAS 0
0IDC0001I FUNCTION COMPLETED, HIGHEST CONDITION CODE WAS 0
0
0IDC0002I IDCAMS PROCESSING COMPLETE. MAXIMUM CONDITION CODE WAS 0


Code:

************************************************************************************************************************************
 *                                                                                                                                  *
 *                                                 JOB STEP TERMINATION REPORTING                                                   *
 *                                                                                                         HH.MM.SS.TH     MMM.THT  *
 *    STEPNAME              PRIV AREA SIZE       6144K       PAGES SWAP IN        0      STEP VECTOR TIME  00.00.00.00              *
 *    PROCSTEP  RSLTSORT    PRIV USED TOP         340K       PAGES SWAP OUT       0      INIT VECTOR TIME  00.00.00.00              *
 *    PGM NAME  SORT        PRIV USED BOT        2128K       REREADS              0      CPU TCB TIME      00.05.33.24       5.554  *
 *    COMP CODE     0000    STEP SERVICE    6,854,148        TAPE MOUNTS          0      CPU SRB TIME      00.00.00.79       0.013  *
 *                          MEMLIMIT SIZE             200MB                              CPU TIME                          307.163  *
 *                                                                                                                                  *
 *  UNIT-T-DDNAME------EXCP CNT---MB TRANSFER  UNIT-T-DDNAME------EXCP CNT---MB TRANSFER  UNIT-T-DDNAME------EXCP CNT---MB TRANSFER *
 *  730E   SORTIN        87,737     1,357.817  3949   SORTOUT           66         1.817  3A38   SORTWK04           2         0.030 *
 *  3A48   SORTWK0F           2         0.030  3A82   SORTWK0D           2         0.030  3E87   SORTWK05           2         0.030 *
 *  381C   SORTWK0C           2         0.030  383A   SORTWK10           2         0.030  383B   SORTWK0A           2         0.030 *
 *  3852   SORTWK03           2         0.030  3874   SORTWK02           2         0.030  4D56   SORTWK07           2         0.030 *
 *  4E2D   SORTWK0B           2         0.030  4E2E   SORTWK08           2         0.030  4E3A   SORTWK09           2         0.030 *
 *  4E38   SORTWK06           2         0.030  6510   SORTWK01           2         0.030  395A   SORTWK0E           1         0.015 *
 *                                                                                                                                  *
 *  EXCP TOTAL           87,834                TOTAL DASD MB TRANSFERS         1,360.099  TOTAL TAPE MB TRANSFERS             0.000 *
 *                                                                                                                                  *
 ************************************************************************************************************************************
 IEF373I STEP/RSLTSORT/START 2014174.1550
 IEF374I STEP/RSLTSORT/STOP  2014174.1556 CPU    5MIN 33.24SEC SRB    0MIN 00.79SEC VIRT  2128K SYS   340K EXT   66004K SYS   12644K


Existing job
Code:

//ZZZZZZTP JOB (COPY,T),T,MSGCLASS=X,CLASS=S,REGION=6144K,   
//             NOTIFY=&SYSUID                               
//RSLTSORT EXEC PGM=SORT                                       
//SORTIN DD DSN=VSAMFILE,DISP=SHR,           
//       AMP='BUFNI=24,BUFND=08'                               
//SORTOUT  DD  DSN=FLATFILE,             
//             DISP=(,CATLG,DELETE),                           
//             UNIT=SYSDA,                                     
//             SPACE=(CYL,(350,70),RLSE),                     
//             DCB=(DSORG=PS,RECFM=FB,LRECL=612,BLKSIZE=0)     
//SYSOUT DD SYSOUT=*                                           
//SORTDIAG DD DUMMY                                           
//SYSIN DD *                                                   
   SORT FIELDS=(1,26,CH,A)                                     
   INCLUDE COND=(1,1,CH,EQ,C'R')                               
/*                                                             
//SORTWK01 DD UNIT=SYSDA,SPACE=(CYL,(100,25))                 
//SORTWK02 DD UNIT=SYSDA,SPACE=(CYL,(100,25))                 
//SORTWK03 DD UNIT=SYSDA,SPACE=(CYL,(100,25))                 
//SORTWK04 DD UNIT=SYSDA,SPACE=(CYL,(100,25))                 
//SORTWK05 DD UNIT=SYSDA,SPACE=(CYL,(100,25))                 
//SORTWK06 DD UNIT=SYSDA,SPACE=(CYL,(100,25))     

Code:

1ICE805I 1 JOBNAME: ZZZZZZTP , STEPNAME: RSLTSORT
 ICE802I 0 BLOCKSET     TECHNIQUE IN CONTROL
 ICE201I H RECORD TYPE IS F - DATA STARTS IN POSITION 1
 ICE992I 0 RA 0 WR 0 TR 0
 ICE751I 0 C5-K90025 C6-K90025 C7-K54603 C8-K62201 EE-K51707 E4-K51707 C9-BASE   E5-K62201 E7-K62201
 ICE143I 0 BLOCKSET     SORT  TECHNIQUE SELECTED
 ICE250I 0 VISIT http://www.ibm.com/storage/dfsort FOR DFSORT PAPERS, EXAMPLES AND MORE
 ICE000I 1 - CONTROL STATEMENTS FOR XXXX-A01, Z/OS DFSORT V1R10 - 16:15 ON MON JUN 23, 2014 -
0             SORT FIELDS=(1,26,CH,A)
              INCLUDE COND=(1,1,CH,EQ,C'R')
 ICE201I H RECORD TYPE IS F - DATA STARTS IN POSITION 1
 ICE992I 0 RA 0 WR 0 TR 0
 ICE751I 0 C5-K90025 C6-K90025 C7-K54603 C8-K62201 EE-K51707 E4-K51707 C9-BASE   E5-K62201 E7-K62201
 ICE193I 0 ICEAM1 INVOCATION ENVIRONMENT IN EFFECT - ICEAM1 ENVIRONMENT SELECTED
 ICE088I 1 ZZZZZZTP.RSLTSORT.        , INPUT LRECL = 612, BLKSIZE = 22528, TYPE = F
 ICE093I 0 MAIN STORAGE = (MAX,67108864,67108864)
 ICE156I 0 MAIN STORAGE ABOVE 16MB = (67051504,67051504)
 ICE127I 0 OPTIONS: OVFLO=RC0 ,PAD=RC0 ,TRUNC=RC0 ,SPANINC=RC16,VLSCMP=N,SZERO=Y,RESET=Y,VSAMEMT=Y,DYNSPC=256
 ICE128I 0 OPTIONS: SIZE=67108864,MAXLIM=2097152,MINLIM=450560,EQUALS=Y,LIST=Y,ERET=RC16 ,MSGDDN=SYSOUT
 ICE129I 0 OPTIONS: VIO=N,RESDNT=ALL ,SMF=NO   ,WRKSEC=Y,OUTSEC=Y,VERIFY=N,CHALT=N,DYNALOC=N             ,ABCODE=MSG
 ICE130I 0 OPTIONS: RESALL=8192,RESINV=0,SVC=109 ,CHECK=Y,WRKREL=Y,OUTREL=Y,CKPT=N,COBEXIT=COB2
 ICE131I 0 OPTIONS: TMAXLIM=6291456,ARESALL=0,ARESINV=0,OVERRGN=65536,CINV=Y,CFW=Y,DSA=64
 ICE132I 0 OPTIONS: VLSHRT=N,ZDPRINT=Y,IEXIT=N,TEXIT=N,LISTX=N,EFS=NONE    ,EXITCK=S,PARMDDN=DFSPARM ,FSZEST=N
 ICE133I 0 OPTIONS: HIPRMAX=OPTIMAL,DSPSIZE=MAX ,ODMAXBF=0,SOLRF=Y,VLLONG=N,VSAMIO=N,MOSIZE=MAX
 ICE235I 0 OPTIONS: NULLOUT=RC0
 ICE084I 0 EXCP ACCESS METHOD USED FOR SORTOUT
 ICE084I 1 VSAM ACCESS METHOD USED FOR SORTIN
 ICE750I 0 DC 60999634260 TC 0 CS DSVVV KSZ 30 VSZ 30
 ICE887I 0 CSES 0,0,0 ES 0,0,0
 ICE886I 0 SYS 0 TSTG 0 FS 0 INIT 0 MAX 0 LEN 0
 ICE752I 0 FSZ=99672605 RC  IGN=0 E  AVG=616 0  WSP=79745871 C  DYN=0 0
 ICE915I 0 MOFSZ=60075,MOSZ=0,MOSYS=200(2),MOSTG=6725,MEML=200(1)
 ICE916I 0 MOFR=0104,MOVR=VV
 ICE996I 0 ESM=2568704,ESO=2568704,ESR=0,ESP=4096,ESS=16384,CES=2568704,HSZ=524288
 ICE997I 0 HWSP=37474563,HMAX=1721787,HES=1721787,ASV=1721787,EQ=I2,HN=1
 ICE898I 0 OMAX=552964,NMAX=1721787,ENQT=1721787,CMAX=524288,HU=99,BUN=53216,MD=M3,DU=90,DR=0,HN=1
 ICE880I 0 QP=129 QA=129 HI=421 LI=293 MI=421 TZ=525755 N1=524288 N2=524288 SZ=129 HN=1
 ICE889I 0 CT=MAX     , SB=8, L=0, D=0000, CCW=1MAM
 ICE901I 0 W 01NP17 02NP17 03NP17 04NP17 05NP17 06NP17
 ICE902I 0 O NP10  I PP10
 ICE751I 1 DE-K61785 D5-K62201 D3-BASE   ED-BASE   E8-K61438
 ICE900I 0 CON=1,MUV=0,VOL=1,ENU=0,SBK=0,SRC=0,VEM=0
 ICE090I 0 OUTPUT LRECL = 612, BLKSIZE = 27540, TYPE = FB
 ICE055I 0 INSERT 0, DELETE 97833663
 ICE054I 0 RECORDS - IN: 99702176, OUT: 1868513
 ICE134I 0 NUMBER OF BYTES SORTED: 1143529956
 ICE253I 0 RECORDS SORTED - PROCESSED: 1868513, EXPECTED: 99672605
 ICE165I 0 TOTAL WORK DATA SET TRACKS ALLOCATED: 9000 , TRACKS USED: 0
 ICE199I 0 MEMORY OBJECT STORAGE USED = 0M BYTES
 ICE180I 0 HIPERSPACE STORAGE USED = 1135092K BYTES
 ICE188I 0 DATA SPACE STORAGE USED = 0K BYTES
 ICE891I 1 67099384 WMAIN, 9480 CMAIN, MAX CALLOC, N SCN, B BA, 0 AZ, 0 BZ, NN QC, 0 CZ, 0 DZ, 1 PLE
 ICE892I 1 612 RIN  612 BLI  27540 BLO  616 RUN  53216 BUN  2818 CPU  00 CVC
 ICE893I 1 200 XIN 14 WIN 104748 GIN NDEY PFP80 B00 CMV00 CIC UPTH LMK VSI  RUX
 ICE894I 0 1 STR  1 MOR  867 IPB  747 OPB  0 CYL  O   MN
 ICE881I 0 EQ=I2 DX=0 D2=59 D3=69 D4=1 AS=0 SA=0 SB=0 SC=0 HN=1
 ICE885I 0 DAT 00 DSR 010B BINS 0 BSZ 0 RCP 0 AJC 0 RLC 0 DUNIT 0
 ICE895I 0 202 MUNIT  42 SUNIT  31 OUNIT
1ICE896I 0 101 SET  100 DEXTOT  0 BLK   99672605 CSZ  12 WE
 ICE804I 1 SORTWK06 EXCP COUNT: 2
 ICE804I 1 SORTWK05 EXCP COUNT: 2
 ICE804I 1 SORTWK04 EXCP COUNT: 2
 ICE804I 1 SORTWK03 EXCP COUNT: 2
 ICE804I 1 SORTWK02 EXCP COUNT: 2
 ICE804I 1 SORTWK01 EXCP COUNT: 2
 ICE804I 1 SORTOUT  EXCP COUNT: 66
 ICE899I 0 HSR=551,HSW=1554,HRE=21751,HWE=1576,HRP=282763,HWP=282763,HWM=283773,HNM=1
 ICE052I 0 END OF DFSORT


Code:

************************************************************************************************************************************
 *                                                                                                                                  *
 *                                                 JOB STEP TERMINATION REPORTING                                                   *
 *                                                                                                         HH.MM.SS.TH     MMM.THT  *
 *    STEPNAME              PRIV AREA SIZE       6144K       PAGES SWAP IN        0      STEP VECTOR TIME  00.00.00.00              *
 *    PROCSTEP  RSLTSORT    PRIV USED TOP         304K       PAGES SWAP OUT       0      INIT VECTOR TIME  00.00.00.00              *
 *    PGM NAME  SORT        PRIV USED BOT        2132K       REREADS              0      CPU TCB TIME      00.05.38.22       5.637  *
 *    COMP CODE     0000    STEP SERVICE    6,967,756        TAPE MOUNTS          0      CPU SRB TIME      00.00.00.96       0.016  *
 *                          MEMLIMIT SIZE             200MB                              CPU TIME                          311.908  *
 *                                                                                                                                  *
 *  UNIT-T-DDNAME------EXCP CNT---MB TRANSFER  UNIT-T-DDNAME------EXCP CNT---MB TRANSFER  UNIT-T-DDNAME------EXCP CNT---MB TRANSFER *
 *  730E   SORTIN       105,292     1,629.498  480B   SORTOUT           66         1.817  4848   SORTWK01           2         0.030 *
 *  4816   SORTWK02           2         0.030  3A32   SORTWK03           2         0.030  4E33   SORTWK04           2         0.030 *
 *  384B   SORTWK05           2         0.030  4E30   SORTWK06           2         0.030                                            *
 *                                                                                                                                  *
 *  EXCP TOTAL          105,370                TOTAL DASD MB TRANSFERS         1,631.495  TOTAL TAPE MB TRANSFERS             0.000 *
 *                                                                                                                                  *
 ************************************************************************************************************************************
 IEF373I STEP/RSLTSORT/START 2014174.1615
 IEF374I STEP/RSLTSORT/STOP  2014174.1622 CPU    5MIN 38.22SEC SRB    0MIN 00.96SEC VIRT  2132K SYS   304K EXT   66020K SYS   12052K
 ************************************************************************************************************************************


Thanks
Magesh
Back to top
View user's profile Send private message
William Collins
Supermod


Joined: 03 Jun 2012
Posts: 437
Topics: 0

PostPosted: Tue Jun 24, 2014 2:44 am    Post subject: Reply with quote

This was interesting from your first post:

Code:
ICE165I 0 TOTAL WORK DATA SET TRACKS ALLOCATED: 9000 , TRACKS USED: 0


Now looking at the LISTCAT, you are "sorting" on the VSAM key. This is unnecessary, as a VSAM KSDS is always in key order when read sequentially.

Make it a COPY instead of an actual SORT.

You have specified an average record-length of 80, and a maximum of 612 and are treating the records as F. That implies a lot of padding of the short records (if the average is correct).

You have Shareoptions 2,3 and the number of records has changed between the two runs you have shown. Is the file open at the time the SORT step is run?
Back to top
View user's profile Send private message
Magesh_J
Intermediate


Joined: 21 Jun 2014
Posts: 259
Topics: 54

PostPosted: Tue Jun 24, 2014 3:32 am    Post subject: Reply with quote

Hi William
Its a online file.. yes the file was opened.
Key length 1,27 but sort required for 1,26 or as per kolusu advice 2,25
I tried with copy, output record sequence mismatched.

I will run with a closed file and let u know the results.


Thanks
Magesh
Back to top
View user's profile Send private message
William Collins
Supermod


Joined: 03 Jun 2012
Posts: 437
Topics: 0

PostPosted: Tue Jun 24, 2014 4:19 am    Post subject: Reply with quote

You've not answered about the record-lengths.

You key is indeed 27 bytes in length, which means that it is "sorted" already on any length of key starting from position one (although duplicates are certain/potential depending on the length from position one).

You have a flat file, where can it contain about sequence mismatched? Is it going into another KSDS later? Are you sure you don't have duplicate keys when considering only the first 26 of they key?
Back to top
View user's profile Send private message
kolusu
Site Admin
Site Admin


Joined: 26 Nov 2002
Posts: 12378
Topics: 75
Location: San Jose

PostPosted: Tue Jun 24, 2014 10:32 am    Post subject: Reply with quote

Magesh_J,

I do see an improvement with BUFND=34 in both CPU time and the number of EXCP's

Code:

--------------------------------------------------|
              | BUFND=34      | BUFND=8           |
--------------------------------------------------|
CPU TIME      | 00.05.33.24   | 00.05.38.22       |
EXCP'S SORTIN |      87,737   |     105,292       |


I wonder why you think "I have a dfsort step taking 1hr 30 minutes" as the CPU time is less than 6 mins. Is it the elapsed time that you are talking about?

Magesh_J wrote:

Hi William
Its a online file.. yes the file was opened.
Key length 1,27 but sort required for 1,26 or as per kolusu advice 2,25


If your first byte is "R" it doesn't need to be sorted. And you did NOT show us the results from the run with SORT FIELDS=(2,25,CH,A) . But if your key is 27 bytes why are you only sorting on the first 26 bytes?

William Collins wrote:
This was interesting from your first post:

Code:
ICE165I 0 TOTAL WORK DATA SET TRACKS ALLOCATED: 9000 , TRACKS USED: 0



Check the message
Code:

ICE180I 0 HIPERSPACE STORAGE USED = 1135092K BYTES


Well DFSORT was able to get enough HIPERSPACE so it never used the sortwk datasets
_________________
Kolusu
www.linkedin.com/in/kolusu
Back to top
View user's profile Send private message Send e-mail Visit poster's website
Magesh_J
Intermediate


Joined: 21 Jun 2014
Posts: 259
Topics: 54

PostPosted: Tue Jun 24, 2014 12:16 pm    Post subject: Reply with quote

Hi Kolusu,

Yes there is a good improvement in compared with the existing code, thank you very much for the kind advice.

Quote:

I wonder why you think "I have a dfsort step taking 1hr 30 minutes" as the CPU time is less than 6 mins. Is it the elapsed time that you are talking about?


Yes elapsed time, I think space manager is restricting the allocation of work dataset and CPU availability.

Quote:

If your first byte is "R" it doesn't need to be sorted. And you did NOT show us the results from the run with SORT FIELDS=(2,25,CH,A) . But if your key is 27 bytes why are you only sorting on the first 26 bytes?


To be honest I don't know why developer have this sort criteria, I couldn't see any impact technically down the line.

But when I change the code to
option copy or sort field = copy, and compared against sorted output
the record mismatch in 386th position.

When I have option copy or sort fields=copy the output flat which looks incorrect
Code:

 BROWSE    zzzzzzzzzzzzzzzzzzzzzzzzzzzzzz             Line 00006990 Col 321 400
 Command ===>                                                  Scroll ===> CSR 
...................00000  0000000000000000000000000000...........YY.............
...................00000  0000000000000000000000000000...........YY.............
...................00000  0000000000000000000000000000...........YY.............
...................00000  0000000000000000000000000000...........YY.............
............&......00000  0000000000000000000000000000...........YY.............
...................00000  0000000000000000000000000000...........YY.............
............&......00000  0000000000000000000000000000...........YY.............


when I have sort fields=(2,25,CH,A) which looks correct as input file
Code:


 BROWSE    zzzzzzzzzzzzzzzzzzzzzzzzzzzzz              Line 00006990 Col 321 400
 Command ===>                                                  Scroll ===> CSR 
...................00000  0000000000000000000000000000...........YY.............
...................00000  0000000000000000000000000000                         
...................00000  0000000000000000000000000000                         
...................00000  0000000000000000000000000000                         
............&......00000  0000000000000000000000000000                         
...................00000  0000000000000000000000000000                         
............&......00000  0000000000000000000000000000                         


All the records matches, i.e all the other columns matches except the 386th position.
Looks really strange, don't know why this '.....YY....' is being copied on other records.


Input VSAM file
Code:

 File-AID - Browse - VSAMFILE                   ------------------- COL 320  399
 COMMAND ===>                                                  SCROLL ===> CSR 
2----+----3----+----4----+----5----+----6----+----7----+----8----+----9----+----
                    00000  0000000000000000000000000000           YY           
                    00000  0000000000000000000000000000                         
                    00000  0000000000000000000000000000                         
                    00000  0000000000000000000000000000                         
             &      00000  0000000000000000000000000000                         
                    00000  0000000000000000000000000000                         
             &      00000  0000000000000000000000000000                         
Back to top
View user's profile Send private message
kolusu
Site Admin
Site Admin


Joined: 26 Nov 2002
Posts: 12378
Topics: 75
Location: San Jose

PostPosted: Tue Jun 24, 2014 12:22 pm    Post subject: Reply with quote

Magesh_J wrote:

But when I change the code to
option copy or sort field = copy, and compared against sorted output
the record mismatch in 386th position.

When I have option copy or sort fields=copy the output flat which looks incorrect


The residual data is because of short records. When your VSAM file is read as
DFSORT Application Programming Guide wrote:

Fixed-length (F). The records are processed without an RDW, so the data starts in position 1. Control statement positions should be specified accordingly.

An RRDS can always be processed as fixed-length. A KSDS, ESDS or VRRDS used for input should only be processed as fixed-length if all of its records have a length equal to the maximum record size defined for the cluster. Otherwise, input records which are shorter than the maximum record size are padded with bytes that may or may not be zeros (that is, "garbage" bytes).

_________________
Kolusu
www.linkedin.com/in/kolusu
Back to top
View user's profile Send private message Send e-mail Visit poster's website
kolusu
Site Admin
Site Admin


Joined: 26 Nov 2002
Posts: 12378
Topics: 75
Location: San Jose

PostPosted: Tue Jun 24, 2014 12:34 pm    Post subject: Reply with quote

Magesh_J,

Try reading your VSAM input file as RECORD TYPE=V and use the following control cards to COPY the file to a sequential file.

Code:

//SYSIN    DD *
  OPTION COPY
  INCLUDE COND=(5,1,CH,EQ,C'R')
  RECORD TYPE=V
  OUTFIL VTOF,BUILD=(5,612)
//*

_________________
Kolusu
www.linkedin.com/in/kolusu
Back to top
View user's profile Send private message Send e-mail Visit poster's website
Magesh_J
Intermediate


Joined: 21 Jun 2014
Posts: 259
Topics: 54

PostPosted: Tue Jun 24, 2014 4:04 pm    Post subject: Reply with quote

Kolusu,

Changed the code as per your advice, It works like a charm, it consume little more CPU because of this conversion, but never mind, because existing code has a bug and our code doesn't use workspace so it will not wait for workspace allocation, hence elapsed time will get reduced during the batch hours.

Thank you very much for all the advice.

Code:

************************************************************************************************************************************
 *                                                                                                                                  *
 *                                                 JOB STEP TERMINATION REPORTING                                                   *
 *                                                                                                         HH.MM.SS.TH     MMM.THT  *
 *    STEPNAME              PRIV AREA SIZE       6144K       PAGES SWAP IN        0      STEP VECTOR TIME  00.00.00.00              *
 *    PROCSTEP  RSLTSORT    PRIV USED TOP         300K       PAGES SWAP OUT       0      INIT VECTOR TIME  00.00.00.00              *
 *    PGM NAME  SORT        PRIV USED BOT        2132K       REREADS              0      CPU TCB TIME      00.06.18.27       6.304  *
 *    COMP CODE     0000    STEP SERVICE    7,780,571        TAPE MOUNTS          0      CPU SRB TIME      00.00.00.96       0.016  *
 *                          MEMLIMIT SIZE             200MB                              CPU TIME                          348.710  *
 *                                                                                                                                  *
 *  UNIT-T-DDNAME------EXCP CNT---MB TRANSFER  UNIT-T-DDNAME------EXCP CNT---MB TRANSFER  UNIT-T-DDNAME------EXCP CNT---MB TRANSFER *
 *  4844   SORTIN        18,955       293.347  4E2B   SORTIN        11,148       172.526  3870   SORTIN         6,795       105.159 *
 *  387E   SORTIN         5,324        82.394  3E87   SORTIN        12,128       187.692  3A86   SORTIN         7,768       120.217 *
 *  4847   SORTIN        11,116       172.031  3841   SORTIN        10,612       164.231  3877   SORTIN         3,923        60.712 *
 *  4848   SORTOUT       10,381       285.892                                                                                       *
 *                                                                                                                                  *
 *  EXCP TOTAL           98,150                TOTAL DASD MB TRANSFERS         1,644.201  TOTAL TAPE MB TRANSFERS             0.000 *
 *                                                                                                                                  *
 ************************************************************************************************************************************
 IEF373I STEP/RSLTSORT/START 2014175.1048
 IEF374I STEP/RSLTSORT/STOP  2014175.1057 CPU    6MIN 18.27SEC SRB    0MIN 00.96SEC VIRT  2132K SYS   300K EXT    6880K SYS   12052K
 ************************************************************************************************************************************
 *                                                                                                                                  *
 *                                                 JOB END TERMINATION REPORTING                                                    *
 *                                                                                                      HH.MM.SS.TH     MMM.THT     *
 *    JOBNAME ZZZZZZZZ    DASD MB XFERS     1,644.201    CPU SERVICE    7,711,860     JOB VECTOR TIME   00.00.00.00                 *
 *    SYSID       XXX    TAPE MB XFERS         0.000    SRB SERVICE       19,568     INIT VECTOR TIME  00.00.00.00                 *
 *    SRVCLAS TSTBATMD    TAPE MOUNTS               0    MSO SERVICE            0     CPU TCB TIME      00.06.18.27       6.304     *
 *    CLASS          S                                   I/O SERVICE       49,143     CPU SRB TIME      00.00.00.96       0.016     *
 *                                                       TOT SERVICE    7,780,571     CPU TIME                          348.710     *
 *                                                                                                                                  *
 ************************************************************************************************************************************
 IEF375I  JOB/ZZZZZZZZ/START 2014175.1048
 IEF376I  JOB/ZZZZZZZZ/STOP  2014175.1057 CPU    6MIN 18.27SEC SRB    0MIN 00.96SEC


Code:

1ICE805I 1 JOBNAME: ZZZZZTP , STEPNAME: RSLTSORT
 ICE802I 0 BLOCKSET     TECHNIQUE IN CONTROL
 ICE201I H RECORD TYPE IS V - DATA STARTS IN POSITION 5
 ICE858I 0 LA=409060, DA=6144, AA=402916, BA=216, CP=1, TA=265
 ICE859I 0 LB=6052, DB=2064, AB=3988, BB=0, CP=0, RS=0, TB=20
 ICE860I 0 F=YN, P=2, M=N, B=256
 ICE751I 0 C5-K90025 C6-K90025 C7-K54603 C8-K62201 EE-K51707 E9-K60823 C9-BASE   E5-K62201 E7-K62201
 ICE143I 0 BLOCKSET     COPY  TECHNIQUE SELECTED
 ICE250I 0 VISIT http://www.ibm.com/storage/dfsort FOR DFSORT PAPERS, EXAMPLES AND MORE
 ICE000I 1 - CONTROL STATEMENTS FOR XXXX-A01, Z/OS DFSORT V1R10 - 10:48 ON TUE JUN 24, 2014 -
0             OPTION COPY
              INCLUDE COND=(5,1,CH,EQ,C'R')
              RECORD TYPE=V
              OUTFIL VTOF,BUILD=(5,612)
 ICE201I H RECORD TYPE IS V - DATA STARTS IN POSITION 5
 ICE858I 0 LA=409060, DA=6144, AA=402916, BA=216, CP=1, TA=265
 ICE859I 0 LB=6052, DB=2064, AB=3988, BB=0, CP=0, RS=0, TB=20
 ICE860I 0 F=YN, P=2, M=N, B=256
 ICE751I 0 C5-K90025 C6-K90025 C7-K54603 C8-K62201 EE-K51707 E9-K60823 C9-BASE   E5-K62201 E7-K62201
 ICE193I 0 ICEAM1 INVOCATION ENVIRONMENT IN EFFECT - ICEAM1 ENVIRONMENT SELECTED
 ICE088I 0 ZZZZZTP.RSLTSORT.        , INPUT LRECL = 616, BLKSIZE = 22528, TYPE = V
 ICE093I 0 MAIN STORAGE = (MAX,6291456,6291456)
 ICE156I 0 MAIN STORAGE ABOVE 16MB = (6234096,6234096)
 ICE127I 0 OPTIONS: OVFLO=RC0 ,PAD=RC0 ,TRUNC=RC0 ,SPANINC=RC16,VLSCMP=N,SZERO=Y,RESET=Y,VSAMEMT=Y,DYNSPC=256
 ICE128I 0 OPTIONS: SIZE=6291456,MAXLIM=2097152,MINLIM=450560,EQUALS=N,LIST=Y,ERET=RC16 ,MSGDDN=SYSOUT
 ICE129I 0 OPTIONS: VIO=N,RESDNT=ALL ,SMF=NO   ,WRKSEC=Y,OUTSEC=Y,VERIFY=N,CHALT=N,DYNALOC=N             ,ABCODE=MSG
 ICE130I 0 OPTIONS: RESALL=8192,RESINV=0,SVC=109 ,CHECK=Y,WRKREL=Y,OUTREL=Y,CKPT=N,COBEXIT=COB2
 ICE131I 0 OPTIONS: TMAXLIM=6291456,ARESALL=0,ARESINV=0,OVERRGN=65536,CINV=Y,CFW=Y,DSA=0
 ICE132I 0 OPTIONS: VLSHRT=N,ZDPRINT=Y,IEXIT=N,TEXIT=N,LISTX=N,EFS=NONE    ,EXITCK=S,PARMDDN=DFSPARM ,FSZEST=N
 ICE133I 0 OPTIONS: HIPRMAX=OPTIMAL,DSPSIZE=MAX ,ODMAXBF=262144,SOLRF=Y,VLLONG=N,VSAMIO=N,MOSIZE=MAX
 ICE235I 0 OPTIONS: NULLOUT=RC0
 ICE084I 1 VSAM ACCESS METHOD USED FOR SORTIN
 ICE889I 0 CT=MAX     , SB=3, L=0, D=0000, CCW=1MAM
 ICE902I 0 O       I NP10
 ICE231I 0 STORAGE USED FOR OUTFIL : BELOW 16M = 20480, ABOVE 16M = 271360
 ICE855I 0 SORTOUT  : TX=N, R=J, L=J, B=D, BL=0, BR=0, DCT=4, VS=N, RU=X, SB=8
 ICE210I 0 SORTOUT  : EXCP USED, LRECL = 612, BLKSIZE = 27540, TYPE = FB
 ICE751I 1 EF-K49535 CB-K64631 F0-K66716 E8-K61438
 ICE900I 0 CON=1,MUV=1,VOL=1,ENU=0,SBK=0,SRC=0,VEM=0
 ICE055I 0 INSERT 0, DELETE 97839942
 ICE054I 0 RECORDS - IN: 99708456, OUT: 1868514
 ICE227I 0 SORTOUT  : DELETED = 0, REPORT = 0, DATA = 1868514
 ICE228I 0 SORTOUT  : TOTAL IN = 1868514, TOTAL OUT = 1868514
 ICE804I 5 OUTFIL   EXCP COUNT: 10381
 ICE891I 1 6283792 WMAIN, 7664 CMAIN, MAX CALLOC, N SCN, B BA, 0 AZ, 0 BZ, NY QC, 0 CZ, 0 DZ, 1 PLE
 ICE892I 1 616 RIN  620 BLI  620 BLO  613 RUN  0 BUN  2818 CPU  00 CVC
 ICE052I 0 END OF DFSORT




Thanks
Magesh
Back to top
View user's profile Send private message
kolusu
Site Admin
Site Admin


Joined: 26 Nov 2002
Posts: 12378
Topics: 75
Location: San Jose

PostPosted: Tue Jun 24, 2014 5:08 pm    Post subject: Reply with quote

Magesh_J,

Did you have BUFND=34 on the COPY step?

Can you run another test with the following control cards

Code:

//SYSIN    DD *                                                 
  OPTION COPY                                                   
  INCLUDE COND=(1,1,CH,EQ,C'R')                                 
  INREC OVERLAY=(612:612,1,CHANGE=(1,C' ',C' '),NOMATCH=(612,1))
//*

_________________
Kolusu
www.linkedin.com/in/kolusu
Back to top
View user's profile Send private message Send e-mail Visit poster's website
Display posts from previous:   
Post new topic   Reply to topic   printer-friendly view    MVSFORUMS.com Forum Index -> Utilities All times are GMT - 5 Hours
Goto page 1, 2  Next
Page 1 of 2

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum


MVSFORUMS
Powered by phpBB © 2001, 2005 phpBB Group