Nah dont work like that.
wish it did. here's pseudocode of what needs to be done.
1. Read in and store reference set one ( 500,000 lines);
2. Read in and store reference set two ( 400,000) lines);
3. Read in and store Reference set three ( 200,000) lines;
4. Process results file
readln from results file
Parse and extract values.
do lookups and extract additional values from 3 lookups
assemble result string;
write out.
you could say read it all in to a database and do it there but the analysis files (22 of them) are all in excess of 2,000,000 lines and it's quicker to read parse lookup and write. believe me I've tried everything.
I got this process down to 3msec and it worked fine using sql it took 40msec. OK if you are doing one or 2 but Gigs of transactions!
one text file I had to process was 900GB....
what I need is very quick in memory lookups. nothing more that that. every millisec added adds hours or days to the processing.