Just advise

[JLWest]

I'm reading 8,000,000+ records out of a text file into a listbox. But I only need to examine 78,000 plus or minus. The rest I just throw on the floor.
 Of the 78,000 record I need 3 pieces of data.

1 ICAO Airport  Example: KLAX, EGGE KJFK
2 Latitude of the airport.
3 Longitude of the airport.

Several gotcha along the way.
The Lat and Lon may be in a "1302 data_lat" or a 1302 data_lon.
They would look like this in the file.
1302 data_lat 32.15477889
1302 data_lon -14.25897456
Problem is there are version of of this file so the 1302 data_lat and 1302 dat_lon  might be there but empty.

So you try and get the location of the Tower. It may have a Lat/Lon and may not.

Finally you go for the center line of one of the runways. That is required. Can be a couple miles off in really big airports. But if you land the plane 2 miles east of Gatwick on a city street who's gonna complain.

Now here is what i'm doing:

Step 1: (Done)
I read the 8 million saving off in listboxs1
RecType   1    Airport  ICAO,
              14   Tower
              16   Seaplane Base
              17   Helicapoter pad
             100  Runway(s)

The last record in the file is 99.

So starting at the last record I read backwards looking for a Record type 1, 16 or 17 loading them into listbox3 as i go. Once loaded I can parse out my data based on the record type format a record to go into Listbox2 that looks like:

KLAX 45.254879 -124.24587

Should be about 35,000 of those but with them I can calculate anywhere to anywhere.

Step 2: (Trouble)

After Processing a screen of data from listbox3 to listbox2 I would like to delete them from listbox1.
I either get out of bounds error or it wont delete them.

Step 3: (Done Untested)

Write Listbox2 to a file.
   




 

[Trenatos]

Code: Pascal  [Select][+][-]

1. ListBox.Clear;

Do you really need to display most of this on-screen though?

[JLWest]

No I don't need to display it on screen. but, I can see what going on a little easier.

As far as listbox.clear. Which one. I have three. Listbox3 I clear before every load. Process the load into Listbox2 and then clear listbox3.

Listbox2 is where I store my results and listbox1 has 79,000 I have to examine.

[dbannon]

Not sure what your question is, is it about getting an error when clearing ListBox1 ?

I guess its possible you have exceeded the capacity of the ListBox in some way ?  I doubt the designers expected anyone to load that many lines into it and perhaps they neglected to prevent that from happening ? 78K records, 100 bytes each record, 8Meg ?  maybe .....

It should work OK and if that really is the problem, I'd see it as a reportable bug but its still a lot of data in a visual component....
 
I think loading into a TStringList would be a heap faster and we know for sure it can handle that sort of data.  Might be a good place to start.

Davo

EDIT - just re-read the question, you initially put all 8Million records into the ListBox ? Oh. wow ? These LCL components are pretty good arn't they ?

[lucamar]

Step 2: (Trouble)

After Processing a screen of data from listbox3 to listbox2 I would like to delete them from listbox1.
I either get out of bounds error or it wont delete them.

If I were you I would delete the item from ListBox1 as soon as you have read it. Something like this (seudo-)code:

Code: Pascal  [Select][+][-]

1.   repeat
2.     AStr := Listbox1.Items[Listbox1.Items.Count-1];
3.     ListBox.Items.Delete(Listbox1.Items.Count-1)
4.     if GoesToList3(AStr) then
5.       AddToList3(AStr);
6.   until ListBox.Items.Count = 0;
7.  

You get the idea, I'm sure. It has the advantage that you'll not end up consuming all the memory

ETA: dbannon's idea of using a TStringList is good too, and it may speed up things by avoiding the visual updates of the list box.

[Trenatos]

Yeah, maybe I misunderstood what you're asking..

Do you want to delete a single item from the listbox? Or the entire thing?

What's the end goal of your processing?

[JLWest]

The goal is to process all the record and write out a file with three fields:

KLAX 44.125891 -12.45122

I just got this working I think.

Deleting records in Listboxz1 as there are stored in Listbox3 would have been the way to go.
Easier to write.

[jamie]

list boxes and the like are limited of what They can hold...

You are better off filtering the unwanted as you load them.

Plus they slow down a lot when handling large data.

[JLWest]

I do filter from a file of 8 million down to 104,000 on the initial on listbox1.

I think I could load into a list and gain speed. But I like the visual defect, something happening on the screen that tells me and the user that it's not in a locked loop.

[lucamar]

I think I could load into a list and gain speed. But I like the visual defect, something happening on the screen that tells me and the user that it's not in a locked loop.

If that's all, use a TProgressBar!