View previous topic :: View next topic |
Author |
Message |
kris_madras Beginner
Joined: 07 Apr 2004 Posts: 41 Topics: 30
|
Posted: Wed Nov 24, 2004 6:29 am Post subject: Match merge with a 40 Million records File |
|
|
Hi,
I need your help. The requirement is as follows :
File 1 : Store-Master
CMF-No : 1-8(starting position is 1 and length is 2)
Store-Code : 44-2
File 2 : History file
CMF-No : 178-8
Read the History file CMF no and match the CMF-No from Store-Master if found move the Store-Code from Store-Master
to output History by combining Store-Master and CMF-No at position
176-10(New cmf starts at position 176 of length 10
Ex :
Store-Master History Output History
1-----8 44 178------ 176-10
12345678 AR 12345678 AR12345678
What I did is converted the Store-Master file into VSAM and performed the match merge with Input History file
The real problem encountered is the Input History file is now 40 Million records and Store-Master file is 30,609 records.
The Job is in execution but I am not sure that this is a good way interms of performance consideration(with production ID it is running past 4 hours!!!)
I request you to kindly suggest the alternative ways(in SORT....)...?
Thanks,
Kris |
|
Back to top |
|
 |
Phantom Data Mgmt Moderator

Joined: 07 Jan 2003 Posts: 1056 Topics: 91 Location: The Blue Planet
|
Posted: Wed Nov 24, 2004 7:32 am Post subject: |
|
|
kris_madras,
Will your history file contain any duplicates on the key field (in ur case the CMF-No).
If not its, very easy in sort.
Thanks,
Phantom |
|
Back to top |
|
 |
kris_madras Beginner
Joined: 07 Apr 2004 Posts: 41 Topics: 30
|
Posted: Wed Nov 24, 2004 7:40 am Post subject: |
|
|
Hi,
The STORE-MASTER file contains no Duplicates but the History file contains Duplicates.
I am curious to know how can u code a logic if there are no duplicates in Input History file(40 mil.records)
Thanks,
Kris |
|
Back to top |
|
 |
Phantom Data Mgmt Moderator

Joined: 07 Jan 2003 Posts: 1056 Topics: 91 Location: The Blue Planet
|
Posted: Wed Nov 24, 2004 7:48 am Post subject: |
|
|
kris_madras,
The following post should answer all ur questions. Try this.
http://www.mvsforums.com/helpboards/viewtopic.php?t=3121
Let us know if you still find some difficulties in matching records. The simplest logic is to concatenate the two files (make sure the matching key is in same place in both the files - may need some reformatting of input files if necessary), Insert a constant character say '1' at the end of layout in both files, then sort on the key field and sum on the constant char.
If the sum exceeds 1 then the records matched, if not the records remain separated.
Hope this helps,
Thanks,
Phantom |
|
Back to top |
|
 |
kolusu Site Admin

Joined: 26 Nov 2002 Posts: 12378 Topics: 75 Location: San Jose
|
Posted: Wed Nov 24, 2004 8:56 am Post subject: |
|
|
Kris
Quote: |
What I did is converted the Store-Master file into VSAM and performed the match merge with Input History file
|
Which language did you use to perform the match ? Cobol? For 31,000 records , your job should run in less than an hour.
Do you have easytrieve in your shop?
Kolusu _________________ Kolusu
www.linkedin.com/in/kolusu |
|
Back to top |
|
 |
Phantom Data Mgmt Moderator

Joined: 07 Jan 2003 Posts: 1056 Topics: 91 Location: The Blue Planet
|
Posted: Wed Nov 24, 2004 9:50 am Post subject: |
|
|
Kolusu,
I think the 40 million records file is stored in a tape. Probably that could have caused the delay.
Only kris can answer this .
Thanks,
Phantom |
|
Back to top |
|
 |
|
|