MVSFORUMS.com Forum Index MVSFORUMS.com
A Community of and for MVS Professionals
 
 FAQFAQ   SearchSearch   Quick Manuals   RegisterRegister 
 ProfileProfile   Log in to check your private messagesLog in to check your private messages   Log inLog in 

Selecting distinct value from Large VSAM File

 
Post new topic   Reply to topic   printer-friendly view    MVSFORUMS.com Forum Index -> Data Management
View previous topic :: View next topic  
Author Message
mls
Beginner


Joined: 10 Nov 2005
Posts: 9
Topics: 5

PostPosted: Mon Dec 24, 2007 5:37 am    Post subject: Selecting distinct value from Large VSAM File Reply with quote

Hi,

I have a big VSAM file with millions of records in the file. It has a specific 2 digit code at column 10. I need to extract all the unique codes from this file. Since the file is very large, I am getting memory problem when I am using SORT with SUM FIELDS=NONE.

Please let me know if there is any other way to extract unique codes from the file from a very large VSAM file.

Thanks in Advance,
MLS
Back to top
View user's profile Send private message
dbzTHEdinosauer
Supermod


Joined: 20 Oct 2006
Posts: 1411
Topics: 26
Location: germany

PostPosted: Mon Dec 24, 2007 5:42 am    Post subject: Reply with quote

you are only concerned with 2 characters starting at column 10. I imagine that you are sorting the complete record. With inrec, pass only 4 or 5 char (2 char) to the sort. That would probably reduce you memory requirements.
_________________
Dick Brenholtz
American living in Varel, Germany
Back to top
View user's profile Send private message
mls
Beginner


Joined: 10 Nov 2005
Posts: 9
Topics: 5

PostPosted: Mon Dec 24, 2007 7:57 am    Post subject: Reply with quote

I am sorting on the 2 characters in column 10 only. I am not sorting on the entire record. I am still gettin virtual memory problem since there are millions of records in the file.

Pls suggest any other way.

Thanks,
MLS
Back to top
View user's profile Send private message
Terry_Heinze
Supermod


Joined: 31 May 2004
Posts: 391
Topics: 4
Location: Richfield, MN, USA

PostPosted: Mon Dec 24, 2007 8:44 am    Post subject: Reply with quote

I think you might have misinterpreted Dick's suggestion. Are you using the INREC parameter as he suggested?
_________________
....Terry
Back to top
View user's profile Send private message Send e-mail
Frank Yaeger
Sort Forum Moderator
Sort Forum Moderator


Joined: 02 Dec 2002
Posts: 1618
Topics: 31
Location: San Jose

PostPosted: Mon Dec 24, 2007 11:57 am    Post subject: Reply with quote

If you just want to extract the unique ids, you can use this DFSORT/ICETOOL job:

Code:

//S1    EXEC  PGM=ICETOOL
//TOOLMSG DD SYSOUT=*
//DFSMSG  DD SYSOUT=*
//IN DD DSN=...  input file
//OUT DD DSN=...  output file
//TOOLIN DD *
OCCUR FROM(IN) LIST(OUT) NOHEADER ON(10,2,CH) VSAMTYPE(F)
/*


It uses INREC automatically.

Alternatively, you could use INREC with SUM FIELDS=NONE yourself as others suggested, like this:

Code:

    RECORD TYPE=F
    INREC BUILD=(10,2)
    SORT FIELDS=(1,2,CH,A)
    SUM FIELDS=NONE


Quote:
I am getting memory problem


It would have helped to be more specific about the "problem" you're having, e.g. show the error messages.
_________________
Frank Yaeger - DFSORT Development Team (IBM)
Specialties: JOINKEYS, FINDREP, WHEN=GROUP, ICETOOL, Symbols, Migration
DFSORT is on the Web at:
www.ibm.com/storage/dfsort
Back to top
View user's profile Send private message Send e-mail Visit poster's website
mls
Beginner


Joined: 10 Nov 2005
Posts: 9
Topics: 5

PostPosted: Wed Dec 26, 2007 6:03 am    Post subject: Reply with quote

Thanks a lot for your responses!

I tried following Syntax using ICETOOL and it worked!
OCCUR FROM(IN) LIST(OUT) NOHEADER ON(10,2,CH) VSAMTYPE(F)

Regards,
Mona
Back to top
View user's profile Send private message
Display posts from previous:   
Post new topic   Reply to topic   printer-friendly view    MVSFORUMS.com Forum Index -> Data Management All times are GMT - 5 Hours
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum


MVSFORUMS
Powered by phpBB © 2001, 2005 phpBB Group