MVSFORUMS.com Forum Index MVSFORUMS.com
A Community of and for MVS Professionals
 
 FAQFAQ   SearchSearch   Quick Manuals   RegisterRegister 
 ProfileProfile   Log in to check your private messagesLog in to check your private messages   Log inLog in 

VSAM Bufnd, Bufni and CI/CA

 
Post new topic   Reply to topic   printer-friendly view    MVSFORUMS.com Forum Index -> Data Management
View previous topic :: View next topic  
Author Message
Magesh_J
Intermediate


Joined: 21 Jun 2014
Posts: 259
Topics: 54

PostPosted: Tue Jul 22, 2014 4:41 am    Post subject: VSAM Bufnd, Bufni and CI/CA Reply with quote

Hi,

Please help me in identifying the BUFNI and BUFND for the below vsam file.

When I access this file sequentially, what should be my BUFND, BUFNI
When I access this file Randomly, what should be my BUFND, BUFNI
When I access dynamically :
1. More sequential and Less random, what should be my BUFND, BUFNI
2. Less sequential and More random, What should be my BUFND, BUFNI.

Also please advice how should I allocate CISZ for index and data for the below vsam file.
1. If I have more random access
2. If I have more sequential access.

My understanding :

For sequential access

BUFND will be CI/CA of data which is 33 adding 1 sequence set it is 34. BUFND=34 and No BUFNI

For Random access

Code:

CISZ   CI/CA   HI U RBA        TI      CASZ        BUFNI
22528   33   13044860928   17611   743424   65


BUFNI=65,BUFND=34

Please advice one dynamic allocation i.e more sequential and less sequential.



Code:

    LISTCAT ENT('VSAMFILE') ALL
 0CLUSTER ------- VSAMFILE
       IN-CAT --- CATALOG.VOLUXXX
       HISTORY
         DATASET-OWNER-----(NULL)     CREATION--------2014.172
         RELEASE----------------2     EXPIRATION------0000.000
       SMSDATA
         STORAGECLASS ---STANDARD     MANAGEMENTCLASS-PRODVSAM
         DATACLASS ------XTNDADDR     LBACKUP ---0000.000.0000
         EATTR-------------(NULL)
         BWO STATUS------00000000     BWO TIMESTAMP---00000 00:00:00.0
         BWO---------------(NULL)
       RLSDATA
         LOG ----------------(NULL)   RECOVERY REQUIRED --(NO)     FRLOG ------------(NULL)
         VSAM QUIESCED -------(NO)    RLS IN USE ---------(NO)
 0       LOGSTREAMID-----------------------------(NULL)
         RECOVERY TIMESTAMP LOCAL-----X'0000000000000000'
         RECOVERY TIMESTAMP GMT-------X'0000000000000000'
       PROTECTION-PSWD-----(NULL)     RACF----------------(NO)
       ASSOCIATIONS
         DATA-----VSAMFILE.ZZZZZZZZ.DATA
         INDEX----VSAMFILE.ZZZZZZZZ.INDX
 0   DATA ------- VSAMFILE.ZZZZZZZZ.DATA
       IN-CAT --- CATALOG.VOLUXXX
       HISTORY
         DATASET-OWNER-----(NULL)     CREATION--------2014.172
         RELEASE----------------2     EXPIRATION------0000.000
         ACCOUNT-INFO-----------------------------------(NULL)
       PROTECTION-PSWD-----(NULL)     RACF----------------(NO)
       ASSOCIATIONS
         CLUSTER--VSAMFILE
       ATTRIBUTES
         KEYLEN----------------27     AVGLRECL--------------80     BUFSPACE-----------49152     CISIZE-------------22528
         RKP--------------------0     MAXLRECL-------------612     EXCPEXIT----------(NULL)     CI/CA-----------------33
         STRIPE-COUNT-----------1
         ACT-DIC-TOKEN----X'4000000A01000540070208FE05FE0DFE0EFE0AFE00000000000000000000000000000000'
         SHROPTNS(2,3)      SPEED     UNIQUE           NOERASE     INDEXED       NOWRITECHK     NOIMBED       NOREPLICAT
         UNORDERED        NOREUSE     NONSPANNED    COMP-FORMT     EXTENDED        EXT-ADDR
       STATISTICS
         REC-TOTAL-------99672605     SPLITS-CI--------------0     EXCPS-------------238518
         REC-DELETED------------0     SPLITS-CA--------------0     EXTENTS----------------3
         REC-INSERTED--------5854     FREESPACE-%CI---------15     SYSTEM-TIMESTAMP:
         REC-UPDATED------------0     FREESPACE-%CA---------10          X'CD586AE157189D08'
         REC-RETRIEVED--299145797     FREESPC-------1523185664
         USER-DATA-SIZE----------------------------24244909162     COMP-USER-DATA-SIZE------------------------9794693864
       ALLOCATION
         SPACE-TYPE------CYLINDER     HI-A-RBA-----13381632000
 1IDCAMS  SYSTEM SERVICES                                           TIME: 15:56:46        06/23/14     PAGE      2
 0       SPACE-PRI-----------1000     HI-U-RBA-----13044860928
         SPACE-SEC-----------1000
       VOLUME
         VOLSER------------VVSM08     PHYREC-SIZE---------5632     HI-A-RBA-----13381632000     EXTENT-NUMBER----------3
         DEVTYPE------X'3010200F'     PHYRECS/TRK------------9     HI-U-RBA-----13044860928     EXTENT-TYPE--------X'40'
         VOLFLAG------------PRIME     TRACKS/CA-------------15
         EXTENTS:
         LOW-CCHH-----X'00BE0000'     LOW-RBA----------------0     TRACKS-------------15000
         HIGH-CCHH----X'04A5000E'     HIGH-RBA-------743423999
         LOW-CCHH-----X'04E20000'     LOW-RBA--------743424000     TRACKS-------------45000
         HIGH-CCHH----X'1099000E'     HIGH-RBA------2973695999
         LOW-CCHH-----X'38E10000'     LOW-RBA-------2973696000     TRACKS------------210000
         HIGH-CCHH----X'6F90000E'     HIGH-RBA-----13381631999
       VOLUME
         VOLSER-----------------*     PHYREC-SIZE------------0     HI-A-RBA---------------0     EXTENT-NUMBER----------0
         DEVTYPE------X'3010200F'     PHYRECS/TRK------------0     HI-U-RBA---------------0     EXTENT-TYPE--------X'FF'
         VOLFLAG--------CANDIDATE     TRACKS/CA--------------0
       VOLUME
         VOLSER-----------------*     PHYREC-SIZE------------0     HI-A-RBA---------------0     EXTENT-NUMBER----------0
         DEVTYPE------X'3010200F'     PHYRECS/TRK------------0     HI-U-RBA---------------0     EXTENT-TYPE--------X'FF'
         VOLFLAG--------CANDIDATE     TRACKS/CA--------------0
       VOLUME
         VOLSER-----------------*     PHYREC-SIZE------------0     HI-A-RBA---------------0     EXTENT-NUMBER----------0
         DEVTYPE------X'3010200F'     PHYRECS/TRK------------0     HI-U-RBA---------------0     EXTENT-TYPE--------X'FF'
         VOLFLAG--------CANDIDATE     TRACKS/CA--------------0
 0   INDEX ------ VSAMFILE.ZZZZZZZZ.INDX
       IN-CAT --- CATALOG.VOLUXXX
       HISTORY
         DATASET-OWNER-----(NULL)     CREATION--------2014.172
         RELEASE----------------2     EXPIRATION------0000.000
       PROTECTION-PSWD-----(NULL)     RACF----------------(NO)
       ASSOCIATIONS
         CLUSTER--VSAMFILE
       ATTRIBUTES
         KEYLEN----------------27     AVGLRECL---------------0     BUFSPACE---------------0     CISIZE--------------4096
         RKP--------------------0     MAXLRECL------------4089     EXCPEXIT----------(NULL)     CI/CA-----------------12
         SHROPTNS(2,3)   RECOVERY     UNIQUE           NOERASE     NOWRITECHK       NOIMBED     NOREPLICAT     UNORDERED
         NOREUSE         EXTENDED     EXT-ADDR
       STATISTICS
         REC-TOTAL----------17611     SPLITS-CI--------------0     EXCPS-------------145969     INDEX:
         REC-DELETED------------0     SPLITS-CA--------------0     EXTENTS----------------2     LEVELS-----------------3
         REC-INSERTED-----------0     FREESPACE-%CI----------0     SYSTEM-TIMESTAMP:            ENTRIES/SECT-----------5
         REC-UPDATED------------0     FREESPACE-%CA----------0          X'CD586AE157189D08' SEQ-SET-RBA----------------0
         REC-RETRIEVED----------0     FREESPC----------1593344                              HI-LEVEL-RBA---------1245184
       ALLOCATION
         SPACE-TYPE------CYLINDER     HI-A-RBA--------73728000
         SPACE-PRI-------------60     HI-U-RBA--------72134656
         SPACE-SEC-------------20
       VOLUME
         VOLSER------------VVSM08     PHYREC-SIZE---------4096     HI-A-RBA--------73728000     EXTENT-NUMBER----------2
 1IDCAMS  SYSTEM SERVICES                                           TIME: 15:56:46        06/23/14     PAGE      3
 0       DEVTYPE------X'3010200F'     PHYRECS/TRK-----------12     HI-U-RBA--------72134656     EXTENT-TYPE--------X'40'
         VOLFLAG------------PRIME     TRACKS/CA--------------1
         EXTENTS:
         LOW-CCHH-----X'04A60000'     LOW-RBA----------------0     TRACKS---------------900
         HIGH-CCHH----X'04E1000E'     HIGH-RBA--------44236799
         LOW-CCHH-----X'109A0000'     LOW-RBA---------44236800     TRACKS---------------600
         HIGH-CCHH----X'10C1000E'     HIGH-RBA--------73727999
       VOLUME
         VOLSER-----------------*     PHYREC-SIZE------------0     HI-A-RBA---------------0     EXTENT-NUMBER----------0
         DEVTYPE------X'3010200F'     PHYRECS/TRK------------0     HI-U-RBA---------------0     EXTENT-TYPE--------X'FF'
         VOLFLAG--------CANDIDATE     TRACKS/CA--------------0
       VOLUME
         VOLSER-----------------*     PHYREC-SIZE------------0     HI-A-RBA---------------0     EXTENT-NUMBER----------0
         DEVTYPE------X'3010200F'     PHYRECS/TRK------------0     HI-U-RBA---------------0     EXTENT-TYPE--------X'FF'
         VOLFLAG--------CANDIDATE     TRACKS/CA--------------0
       VOLUME
         VOLSER-----------------*     PHYREC-SIZE------------0     HI-A-RBA---------------0     EXTENT-NUMBER----------0
         DEVTYPE------X'3010200F'     PHYRECS/TRK------------0     HI-U-RBA---------------0     EXTENT-TYPE--------X'FF'
         VOLFLAG--------CANDIDATE     TRACKS/CA--------------0
 1IDCAMS  SYSTEM SERVICES                                           TIME: 15:56:46        06/23/14     PAGE      4
 0         THE NUMBER OF ENTRIES PROCESSED WAS:
                     AIX -------------------0
                     ALIAS -----------------0
                     CLUSTER ---------------1
                     DATA ------------------1
                     GDG -------------------0
                     INDEX -----------------1
                     NONVSAM ---------------0
                     PAGESPACE -------------0
                     PATH ------------------0
                     SPACE -----------------0
                     USERCATALOG -----------0
                     TAPELIBRARY -----------0
                     TAPEVOLUME ------------0
                     TOTAL -----------------3
 0         THE NUMBER OF PROTECTED ENTRIES SUPPRESSED WAS 0
 0IDC0001I FUNCTION COMPLETED, HIGHEST CONDITION CODE WAS 0



Thanks
Magesh
Back to top
View user's profile Send private message
William Collins
Supermod


Joined: 03 Jun 2012
Posts: 437
Topics: 0

PostPosted: Tue Jul 22, 2014 5:18 am    Post subject: Reply with quote

Do you need those SHAREOPTIONS (2,3)? I'm pretty sure they're going to mess you up. Because another task can have the file open for update, a sequential read is not going to use the sequence set.

Do a before LISTCAT, REPRO to a PS, then an after LISTCAT.

Then use ALTER to change it to shareoptions 1,3. Run the before/REPRO/after again. (if you've not read the data in the meantime, you can skip the before LISTCAT on this one).

Use BUFNI and BUFND of 50. This is not to see how wonderful BUFNI of 50 is (it probably won't be) but to see them under equal conditions. What we'd be looking for is a big change in the number of I/Os as the second REPRO will use the sequence set, and the first one not.
Back to top
View user's profile Send private message
expat
Intermediate


Joined: 01 Mar 2007
Posts: 475
Topics: 9
Location: Welsh Wales

PostPosted: Tue Jul 22, 2014 5:24 am    Post subject: Reply with quote

There is no 'rule of thumb for the calculation of VSAM buffers for a file.

For sequential I usually use between 5 and 10 for BUFND and 3 for BUFNI.

For RANDOM access one of the best tips is to sort the reference file into the order of the keys of the VSAM file. This means that you will only ever read the index and data portions only once to process the reference file. If it isn't sorted you could be loading the same index and data CI multiple times throughout the job.

If you are using SKIP SEQUENTIAL access remember that every time that you change access from RANDOM / KEYED to SEQUENTIAL you will flush all of the buffers, so probably best to keep to the system defaults.

Also consider that the more buffers you allocate the more storage you will consume, be it physical or virtual.
_________________
If it's true that we are here to help others,
then what exactly are the others here for ?
Back to top
View user's profile Send private message
Magesh_J
Intermediate


Joined: 21 Jun 2014
Posts: 259
Topics: 54

PostPosted: Tue Jul 22, 2014 7:53 am    Post subject: Reply with quote

Thanks Expat and William for the kind answers.

expat wrote:
There is no 'rule of thumb for the calculation of VSAM buffers for a file.


My understanding if we have CI/CA + 1 will help in reading full cylinder at a time. Please advice

expat wrote:
For sequential I usually use between 5 and 10 for BUFND and 3 for BUFNI.


For sequential read, please advice why we need index buffer.

William Collins wrote:

Do you need those SHAREOPTIONS (2,3)? I'm pretty sure they're going to mess you up. Because another task can have the file open for update, a sequential read is not going to use the sequence set.


This is a file used in Online, so I think they have given 2,3 option, while batch is running, online will be down, Only one job update it at a time and rest of the jobs wait for the update to complete.

William Collins wrote:

Use BUFNI and BUFND of 50. This is not to see how wonderful BUFNI of 50 is (it probably won't be) but to see them under equal conditions. What we'd be looking for is a big change in the number of I/Os as the second REPRO will use the sequence set, and the first one not.


My understanding was having some random buffer number is not advisable.
But not sure..

Thanks
Magesh
Back to top
View user's profile Send private message
William Collins
Supermod


Joined: 03 Jun 2012
Posts: 437
Topics: 0

PostPosted: Tue Jul 22, 2014 8:11 am    Post subject: Reply with quote

If only one task can update at a time, and you don't need to read it whilst it is open for update, you don't need 2,3.

Data records on a KSDS can be splattered all over the place. The index tells AMS where the next record is. For sequential processing, only the lowest-level index is needed, the so-called isequence-set. The sequence-set contains horizontal links to the next index in the sequence-set, so only the lowest-level index is needed for sequential processing. BFUNI=1 is sufficient, greater than one may save a few IOs, but not substantially.

With 2,3, I think the entire index is used for sequential processing, so each index record may be read multiple times. Try it out as I suggested and let us know.
Back to top
View user's profile Send private message
expat
Intermediate


Joined: 01 Mar 2007
Posts: 475
Topics: 9
Location: Welsh Wales

PostPosted: Tue Jul 22, 2014 8:22 am    Post subject: Reply with quote

Magesh_J wrote:
My understanding if we have CI/CA + 1 will help in reading full cylinder at a time. Please advice

Possibly in the first instance, but as each buffer is exhausted it will be refreshed by the next CI to be read in. The buffers are allocated enough memory for one CI, not one CA.

Magesh_J wrote:
For sequential read, please advice why we need index buffer.

If you only use a single index buffer you have to wait for that buffer to be finished with before it can be refreshed. With multiple buffers they will be refreshed as and when they are exhausted.

It may also be worth a chat with your storage management people to see if they are using SMB - Sytem Managed Buffering which does everything for you Shocked
_________________
If it's true that we are here to help others,
then what exactly are the others here for ?
Back to top
View user's profile Send private message
kolusu
Site Admin
Site Admin


Joined: 26 Nov 2002
Posts: 12354
Topics: 75
Location: San Jose

PostPosted: Tue Jul 22, 2014 10:49 am    Post subject: Reply with quote

Magesh_J,

Please search before posting. Check this link

http://www.mvsforums.com/helpboards/viewtopic.php?p=43364

I suggest reading chapter 4 in this "VSAM Demystified" redbook.

http://www.redbooks.ibm.com/abstracts/sg246105.html?Open

and this presentation about VSAM Exploitation by Stev Glodowski

ftp://public.dhe.ibm.com/eserver/zseries/zos/vse/pdf3/wavv08/VSAM_Exploitation.pdf
_________________
Kolusu - DFSORT Development Team (IBM)
DFSORT is on the Web at:
www.ibm.com/storage/dfsort

www.linkedin.com/in/kolusu
Back to top
View user's profile Send private message Send e-mail Visit poster's website
Magesh_J
Intermediate


Joined: 21 Jun 2014
Posts: 259
Topics: 54

PostPosted: Wed Jul 23, 2014 7:36 am    Post subject: Reply with quote

Sorry here after will search entire website, I searched in Data Management, didn't search in application programing.

As per your advice gone thru the PDF and come to know SMF is doing this job for us. Same was mentioned by expat

My understanding are follows.
Small CI data allocation for Random access
Large CI data allocation for sequential access.

1. More sequential and Less random, what should be my BUFND, BUFNI
AMP=('ACCBIAS=SW')
2. Less sequential and More random, What should be my BUFND, BUFNI
AMP=('ACCBIAS=DW')
3. only Random => AMP=('ACCBIAS=DO')
4. only Sequential ==> AMP=('ACCBIAS=SO')

PPT is in little high level for me to understand, but some of things I could understand.

But this PDF and expat advice is useful.

Thanks
Magesh
Back to top
View user's profile Send private message
Display posts from previous:   
Post new topic   Reply to topic   printer-friendly view    MVSFORUMS.com Forum Index -> Data Management All times are GMT - 5 Hours
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum


MVSFORUMS
Powered by phpBB © 2001, 2005 phpBB Group