MVSFORUMS.com Forum Index MVSFORUMS.com
A Community of and for MVS Professionals
 
 FAQFAQ   SearchSearch   Quick Manuals   RegisterRegister 
 ProfileProfile   Log in to check your private messagesLog in to check your private messages   Log inLog in 

DFSORT FILSZ Ennn Required Accuracy

 
Post new topic   Reply to topic   printer-friendly view    MVSFORUMS.com Forum Index -> Utilities
View previous topic :: View next topic  
Author Message
twb
Beginner


Joined: 15 Jun 2006
Posts: 5
Topics: 2
Location: San Francisco

PostPosted: Mon Aug 28, 2017 2:56 pm    Post subject: DFSORT FILSZ Ennn Required Accuracy Reply with quote

Does anyone know how wrong a DFSORT FILSZ=Ennn value can be before the sort abends? Is it a number, a percentage, something else? Are the upper and lower limits the same?

The manuals recommend a "reasonably accurate" estimate. IBM and I do not always agree on what is reasonable. I see that I can code a Unnn value which is only used for sizing, not for record counts. I may go that way.

For context, I am asking SAS to pass the results of a view to DFSORT, so the size of the input is not available to DFSORT.
Back to top
View user's profile Send private message
kolusu
Site Admin
Site Admin


Joined: 26 Nov 2002
Posts: 12358
Topics: 75
Location: San Jose

PostPosted: Mon Aug 28, 2017 4:31 pm    Post subject: Re: DFSORT FILSZ Ennn Required Accuracy Reply with quote

twb wrote:
Does anyone know how wrong a DFSORT FILSZ=Ennn value can be before the sort abends? Is it a number, a percentage, something else? Are the upper and lower limits the same?


The question is why do you need to specify the FILSZ? DFSORT automatically calculates the file size and determines the right amount of storage required for the job complete. However when customers are inserting records via an E15 exit or Invoking DFSORT via a program, DFSORT will have no way of knowing an accurate record count for the file size without it being specified. Is that your case? The FILSZ parameter specifies either the exact number of records to be sorted or merged, or an estimate of the number of records to be sorted.

The abend depends on the available resources(Memory objects/hiperspaces/Sortwork disk space..) on your system when the job is running. There is no upper or lower limit

twb wrote:

The manuals recommend a "reasonably accurate" estimate. IBM and I do not always agree on what is reasonable.


Usually DFSORT requires resources about 1.1 - 1.2x of the filesize to be sorted. As for reasonable accuracy, we recommend that you give 20% for future growth of data. For if you are currently sorting 10 million records we recommend coding an estimated Filesz of 12 million for future growth.

twb wrote:

I see that I can code a Unnn value which is only used for sizing, not for record counts. I may go that way.
For context, I am asking SAS to pass the results of a view to DFSORT, so the size of the input is not available to DFSORT.


As I mentioned earlier if SAS is invoking DFSORT then it is better that it passes the FILSZ so that DFSORT can determine the right amount of storage required for the job complete.
_________________
Kolusu - DFSORT Development Team (IBM)
DFSORT is on the Web at:
www.ibm.com/storage/dfsort

www.linkedin.com/in/kolusu
Back to top
View user's profile Send private message Send e-mail Visit poster's website
twb
Beginner


Joined: 15 Jun 2006
Posts: 5
Topics: 2
Location: San Francisco

PostPosted: Mon Aug 28, 2017 4:56 pm    Post subject: Reply with quote

If I coded FILSZ=Unnn, then I would expect an abend if available storage was insufficient. If I code FILSZ=Ennn, I still expect to fail if I exceed available resources, but I also expect to fail simply if my estimate is bad, for some definition of bad. For example, if I code FILSZ=E123456789 and the actual input only has 17 records, I expect the job to abend for a bad estimate (an excessive estimate in this example).

To clarify the context, a SAS view allows me to start reading a file, decide whether or not to keep a particular record, and pass any records I keep to DFSORT. I expect there are a couple of buffers involved, but SAS does not read any significant number of records before it starts passing data to DFSORT. If I were willing to pay the I/O to write all selected records to a SAS library before calling DFSORT, SAS would pass the exact record count to DFSORT.
Back to top
View user's profile Send private message
kolusu
Site Admin
Site Admin


Joined: 26 Nov 2002
Posts: 12358
Topics: 75
Location: San Jose

PostPosted: Mon Aug 28, 2017 5:04 pm    Post subject: Reply with quote

twb wrote:
If I coded FILSZ=Unnn, then I would expect an abend if available storage was insufficient. If I code FILSZ=Ennn, I still expect to fail if I exceed available resources, but I also expect to fail simply if my estimate is bad, for some definition of bad. For example, if I code FILSZ=E123456789 and the actual input only has 17 records, I expect the job to abend for a bad estimate (an excessive estimate in this example).


twb,

DFSORT ignores the bad filesize estimate when it can determine the size of the file to be sorted. Here is an example of it
Code:

//STEP0100 EXEC PGM=SORT   
//SYSOUT   DD SYSOUT=*     
//SORTDIAG DD DUMMY       
//SORTIN   DD *           
AAA                       
BBB                       
CCC                       
DDD                       
CCC                       
ZZZ                       
//SORTOUT  DD SYSOUT=*     
//SYSIN    DD *           
  OPTION FILSZ=E123456789 
  SORT FIELDS=(1,4,CH,D)   
//*


Just because you provided an inaccurate value for an optional parm, the job does not need to abend. DFSORT tries it use it ONLY when it canNOT determine the file size.
twb wrote:

To clarify the context, a SAS view allows me to start reading a file, decide whether or not to keep a particular record, and pass any records I keep to DFSORT. I expect there are a couple of buffers involved, but SAS does not read any significant number of records before it starts passing data to DFSORT. If I were willing to pay the I/O to write all selected records to a SAS library before calling DFSORT, SAS would pass the exact record count to DFSORT.


In case you are not sure about the number of records to be passed then make sure you have enough sort disk space. Increase the dynamic allocation from the default of 4 datasets to a higher number(may be 32)
_________________
Kolusu - DFSORT Development Team (IBM)
DFSORT is on the Web at:
www.ibm.com/storage/dfsort

www.linkedin.com/in/kolusu
Back to top
View user's profile Send private message Send e-mail Visit poster's website
Display posts from previous:   
Post new topic   Reply to topic   printer-friendly view    MVSFORUMS.com Forum Index -> Utilities All times are GMT - 5 Hours
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum


MVSFORUMS
Powered by phpBB © 2001, 2005 phpBB Group