Posted: Tue Sep 26, 2017 6:12 pm Post subject: SORT EXCP on QSAM/BSAM
Sort EXCP is always less, when i am reading a Physical sequential file compared to Cobol.
How does sort allocates Buffer space and buffer number for the Physical sequential dataset ?
And what else parameters are considered for reading and writing a dataset.
The reason why i am asking is, Cobol is taking/consuming lot of excp, when i provide BUFNO=30, it reduces and when I increase the BUFNO to 50 EXCP increases little.
There should be some rule to identify the optimized bufno.
Also Please advise if we can have any other parameters for reducing the EXCP for a physical sequential file.
Joined: 26 Nov 2002 Posts: 12357 Topics: 75 Location: San Jose
Posted: Tue Oct 03, 2017 10:36 am Post subject:
Magesh_J,
DFSORT writes its own channel program for EXCP access method. It will look up the available storage and allocate the buffers dynamically. If your input dataset has properties (PDSE, EXTENDED/COMPRESSED datasets...) that require BSAM access method, then DFSORT calls BSAM under the covers.
If your COBOL program is doing a lot of I/O then check the file declaration and make sure you have "Block contains 0 records" _________________ Kolusu - DFSORT Development Team (IBM)
DFSORT is on the Web at:
www.ibm.com/storage/dfsort
Joined: 26 Nov 2002 Posts: 12357 Topics: 75 Location: San Jose
Posted: Tue Oct 03, 2017 7:45 pm Post subject:
Magesh_J wrote:
I also noticed, DSNTYPE=LARGE specified for the data sets having more EXCP.
do we need to change/handle anything in COBOL for accessing DSNTYPE=LARGE data sets ?
Regards,
Magesh
Magesh_J,
If you have DSNTYPE=LARGE, then you need BSAM access method. _________________ Kolusu - DFSORT Development Team (IBM)
DFSORT is on the Web at:
www.ibm.com/storage/dfsort
Joined: 26 Nov 2002 Posts: 12357 Topics: 75 Location: San Jose
Posted: Sun Oct 08, 2017 5:10 am Post subject:
Magesh_J wrote:
Hi Kolusu,
Thanks for the kind info.
Please advise how can i make the program to use BSAM access method.
Do we have any parameters in JCL or Cobol to use BSAM access method ?
Or when ever I use the dataset, i should specify DSNTYPE=LARGE, so that Cobol will use BSAM access method ?
Regards,
Magesh
If you have DSNTYPE=LARGE , you are getting BSAM access method. I am not sure as to what you will achieve by switching to BSAM. Unless you have zHPF or high performing channel program, I don't see the advantage of switching to BSAM. With BSAM you can actually override the number of buffers allocated.
Try specifying NCP value of 128 in JCL for the DDname in question. Read upon NCP parm in JCL reference manual
You are looking at c++ manual. You need to look up the jcl manual. The limit is 255
Magesh_j wrote:
Also, it appears we can specify BUFNO or NCP, the system will decide to use NCP or BUFNO based on the method of access.
I tried with NCP=99, Job ran in 23 minutes with EXCP TOTAL 3,081,485
I tried with NCP=40, Job ran in 16 minutes with EXCP TOTAL 2,803,225
I tried with BUFNO=50, Job ran in 12 minutes with EXCP TOTAL 2,901,623
I understand elapsed time always varies depends on the CPU load, but the % of cpu utilized when i specify BUFNO=50 is greater than NCP=40
sdsf;da CPU% have 40 to 50 when BUFNO=50 and 25 to 30 when NCP=40 and 18 to 25 when NCP=99.
I ran twice each job to confirm if it is because of CPU load, since its a weekend CPU is available in plenty.
It always follow the same pattern.
I believe there should be some calculation to find the optimized buffer NCP/BUFNO for a dataset based on LRECL and blocksize.
Please advise
Thanks
Magesh
You need to ask the right question as to how to find the optimal bufno or ncp values but you started off with the access method question. So are you concerned abt the input file excps or the output files? Also what is the lrecl and recfm of the files involved? _________________ Kolusu - DFSORT Development Team (IBM)
DFSORT is on the Web at:
www.ibm.com/storage/dfsort
Joined: 26 Nov 2002 Posts: 12357 Topics: 75 Location: San Jose
Posted: Tue Oct 10, 2017 1:24 am Post subject:
Magesh_J wrote:
DCB=(DSORG=PS,RECFM=FB,LRECL=5448,BLKSIZE=0,BUFNO=08)
//*
//*All files are opened in Output mode.
Also same files are used as input in many jobs, so we also need to handle it using input mode, but that is a different job.
Thanks
Magesh
Your output file LRECL is 5448 and hence the ideal BLKSIZE would be 27240 means you can write 5 records per block. so were the datasets allocated with that BLKSIZE or a different block size?
So specify a BUFNO/NCP which is a multiple of 5 for better results. Just specifying a higher BUFNO will not help as , you also need to provide the below the line storage. Code a REGION=0M parm on the jobcard or step(Hope your shop does NOT have an IEFUSI exit to limit the storage). Also remember that your number of excps are dependent on the number of records written.
Specifying a BUFNO on IEFBR14 step will do NOTHING. The Bufno value comes into picture only when you are writing or reading. _________________ Kolusu - DFSORT Development Team (IBM)
DFSORT is on the Web at:
www.ibm.com/storage/dfsort
Your output file LRECL is 5448 and hence the ideal BLKSIZE would be 27240 means you can write 5 records per block. so were the datasets allocated with that BLKSIZE or a different block size?
Code:
Device type . . . . : 3390
Data class . . . . . : DCDFAULT
Organization . . . : PS
Record format . . . : FB
Record length . . . : 5448
Block size . . . . : 27240
1st extent cylinders: 600
Secondary cylinders : 400
Data set name type : LARGE
SMS Compressible. . : NO
kolusu wrote:
Just specifying a higher BUFNO will not help as , you also need to provide the below the line storage.
Code a REGION=0M parm on the jobcard or step
Region=0m is already in place for the step.
Kolusu wrote:
so specify a BUFNO/NCP which is a multiple of 5 for better results.
So the formula is BLKSIZE/LRECL ?
For example if i have a data set of LRECL=120, and the blocksize would be 27960
Joined: 26 Nov 2002 Posts: 12357 Topics: 75 Location: San Jose
Posted: Wed Oct 11, 2017 9:40 am Post subject:
Magesh_J wrote:
So the formula is BLKSIZE/LRECL ?
For example if i have a data set of LRECL=120, and the blocksize would be 27960
27960/120 = 233.
Should i specify BUFNO=233
And for LRECL=80
27920/80 = 349
I think, i misunderstood, please advise.
Thanks
Magesh
Not really. For an LRECL of 120 , the ideal blksize is 27960 which means you can read/write 27960/120 = 233 records at a time. So each block can contain 233 records. Your BUFNO is the number of such blocks. If you specify BUFNO=5 you will have 5 blocks each that can contain 233 records.
Also don't specify a high number of buffers which will take up your below the line storage. _________________ Kolusu - DFSORT Development Team (IBM)
DFSORT is on the Web at:
www.ibm.com/storage/dfsort
Joined: 26 Nov 2002 Posts: 12357 Topics: 75 Location: San Jose
Posted: Thu Oct 12, 2017 7:51 am Post subject:
Magesh_J wrote:
Thanks Kolusu,
Kolusu wrote:
Also don't specify a high number of buffers which will take up your below the line storage
below - the - line - storage
Is it 16 MB ?
16x1024x1024 = 16777216 Bytes.
Which mean, My entire buffer should not cross 16 MB ?
And is it applicable for one file or for all files ?
In the example, i have 6 Large files, I have to make sure my buffer size should not cross 16777216 bytes for each file or for all files ?
What will happen if it crosses 16 MB ?
Thanks
Magesh
Magesh,
Even though it is 16 mb storage most shop restrict that to 10 mb. And that values is applicable for all the ddnames. Check the IEF032I or IEF033I which will display the storage.
If you exceed that limit with high bufno value you would end up with s878 abends _________________ Kolusu - DFSORT Development Team (IBM)
DFSORT is on the Web at:
www.ibm.com/storage/dfsort
You cannot post new topics in this forum You cannot reply to topics in this forum You cannot edit your posts in this forum You cannot delete your posts in this forum You cannot vote in polls in this forum