MVSFORUMS.com Forum Index MVSFORUMS.com
A Community of and for MVS Professionals
 
 FAQFAQ   SearchSearch   Quick Manuals   RegisterRegister 
 ProfileProfile   Log in to check your private messagesLog in to check your private messages   Log inLog in 

Match merge with a 40 Million records File

 
Post new topic   Reply to topic   printer-friendly view    MVSFORUMS.com Forum Index -> Utilities
View previous topic :: View next topic  
Author Message
kris_madras
Beginner


Joined: 07 Apr 2004
Posts: 41
Topics: 30

PostPosted: Wed Nov 24, 2004 6:29 am    Post subject: Match merge with a 40 Million records File Reply with quote

Hi,
I need your help. The requirement is as follows :

File 1 : Store-Master
CMF-No : 1-8(starting position is 1 and length is 2)
Store-Code : 44-2

File 2 : History file

CMF-No : 178-8


Read the History file CMF no and match the CMF-No from Store-Master if found move the Store-Code from Store-Master
to output History by combining Store-Master and CMF-No at position

176-10(New cmf starts at position 176 of length 10

Ex :

Store-Master History Output History
1-----8 44 178------ 176-10
12345678 AR 12345678 AR12345678


What I did is converted the Store-Master file into VSAM and performed the match merge with Input History file

The real problem encountered is the Input History file is now 40 Million records and Store-Master file is 30,609 records.

The Job is in execution but I am not sure that this is a good way interms of performance consideration(with production ID it is running past 4 hours!!!)

I request you to kindly suggest the alternative ways(in SORT....)...?


Thanks,
Kris
Back to top
View user's profile Send private message
Phantom
Data Mgmt Moderator
Data Mgmt Moderator


Joined: 07 Jan 2003
Posts: 1056
Topics: 91
Location: The Blue Planet

PostPosted: Wed Nov 24, 2004 7:32 am    Post subject: Reply with quote

kris_madras,

Will your history file contain any duplicates on the key field (in ur case the CMF-No).

If not its, very easy in sort.

Thanks,
Phantom
Back to top
View user's profile Send private message
kris_madras
Beginner


Joined: 07 Apr 2004
Posts: 41
Topics: 30

PostPosted: Wed Nov 24, 2004 7:40 am    Post subject: Reply with quote

Hi,
The STORE-MASTER file contains no Duplicates but the History file contains Duplicates.

I am curious to know how can u code a logic if there are no duplicates in Input History file(40 mil.records)


Thanks,
Kris
Back to top
View user's profile Send private message
Phantom
Data Mgmt Moderator
Data Mgmt Moderator


Joined: 07 Jan 2003
Posts: 1056
Topics: 91
Location: The Blue Planet

PostPosted: Wed Nov 24, 2004 7:48 am    Post subject: Reply with quote

kris_madras,

The following post should answer all ur questions. Try this.
http://www.mvsforums.com/helpboards/viewtopic.php?t=3121

Let us know if you still find some difficulties in matching records. The simplest logic is to concatenate the two files (make sure the matching key is in same place in both the files - may need some reformatting of input files if necessary), Insert a constant character say '1' at the end of layout in both files, then sort on the key field and sum on the constant char.

If the sum exceeds 1 then the records matched, if not the records remain separated.

Hope this helps,

Thanks,
Phantom
Back to top
View user's profile Send private message
kolusu
Site Admin
Site Admin


Joined: 26 Nov 2002
Posts: 12378
Topics: 75
Location: San Jose

PostPosted: Wed Nov 24, 2004 8:56 am    Post subject: Reply with quote

Kris

Quote:

What I did is converted the Store-Master file into VSAM and performed the match merge with Input History file


Which language did you use to perform the match ? Cobol? For 31,000 records , your job should run in less than an hour.

Do you have easytrieve in your shop?

Kolusu
_________________
Kolusu
www.linkedin.com/in/kolusu
Back to top
View user's profile Send private message Send e-mail Visit poster's website
Phantom
Data Mgmt Moderator
Data Mgmt Moderator


Joined: 07 Jan 2003
Posts: 1056
Topics: 91
Location: The Blue Planet

PostPosted: Wed Nov 24, 2004 9:50 am    Post subject: Reply with quote

Kolusu,

I think the 40 million records file is stored in a tape. Probably that could have caused the delay.

Only kris can answer this .

Thanks,
Phantom
Back to top
View user's profile Send private message
Display posts from previous:   
Post new topic   Reply to topic   printer-friendly view    MVSFORUMS.com Forum Index -> Utilities All times are GMT - 5 Hours
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum


MVSFORUMS
Powered by phpBB © 2001, 2005 phpBB Group