Recent

Author Topic: DataProblems Maybe  (Read 4743 times)

JLWest

  • Hero Member
  • *****
  • Posts: 545
Re: DataProblems Maybe
« Reply #60 on: April 28, 2019, 10:19:56 pm »
I have made a change to the program: I added  'FileLocSW  : Boolean = TRUE;' as a Global Var;

When set to true the program and data files can be dumped into a temp directory and executed from one directory.

When Your done delete the directory.

No path problems or changes needed.

Program Attached



   
JLWEST
Lazuras ver 2.0.2 
 FPC 3.0.4, Lazarus IDE v1.8.2 Windows 10 Pro
 Intel i7 770K CPU 4.2GHz 32702MB Ram
GeForce GTX 1080 Graphics - 8 Gig
3952 GB (1.5 SSD)

lucamar

  • Hero Member
  • *****
  • Posts: 1810
Re: DataProblems Maybe
« Reply #61 on: April 28, 2019, 11:19:12 pm »
[...] wanted to include some documentation on your copyright [...]

There is no copyright, don't worry about it. Almost all of the code (my own, of course) I post in the forum is dedicated to the public domain: I'd find ridiculous for it to be otherwise; I mean, full-fledged copyright for less than 20 common LOC??? If anything, a bare citation down in the credits suffices: "lucamar helped me!"  :P


About the CC0 in full program...

The full programs in attachments now have a CC0 dedication. That's is mainly because our lawyer recommended it.

He said that having a full-deed, legal dedication might strengthen it in countries (like Spain and most of continental Europe) where some author rights are non-renounceable (v.g. moral rights). In those cases the legal deed leaves clear that you renounce all other "rights" and that you "promise" to not enforce those you can't renounce.

As that is (almost) exactly what I want, I followed his advice. :)


ETA: By the way, I checked the data files and all of them are pure test except MstRegions.txt, which has a UTF8 BOM, but that one can easily be converted with cleanbom.

But I won't be able to check on the program until late tomorrow, sorry. :(

I've given it a quick visual and you still use too much the old style of file management. When will you learn to use streams or the numerous "LoadFrom/SaveToFile" which most classes dealing with strings have? ;D
« Last Edit: April 28, 2019, 11:28:08 pm by lucamar »
Turbo Pascal 3 CP/M - Amstrad PCW 8256 (512 KB !!!) :P
Lazarus 1.8.4 & 2.0.2 w/FPC 3.0.4 on:
(K|L)Ubuntu 12..16, Windows XP SP3, various DOSes.

JLWest

  • Hero Member
  • *****
  • Posts: 545
Re: DataProblems Maybe
« Reply #62 on: April 29, 2019, 12:11:47 am »
"ETA: By the way, I checked the data files and all of them are pure test except MstRegions.txt, which has a UTF8 BOM, but that one can easily be converted with cleanbom."

I can't get Cleanbom to work. I Have never run the DOS box programs. (Years a go)



I think I get in a dos box cd to where the Cleanbom resides and do the following

Cleanbom E:\files\Regions\MstRegions.txt  E:\files\Regions\MstRegions.bak


But I won't be able to check on the program until late tomorrow, sorry. :

"When will you learn to use streams or the numerous "LoadFrom/SaveToFile" which most classes dealing with strings"

I'll work on trying to use the "LoadFrom/SaveToFile" and if I can figure it out I will make the change.

Take a look at my next post it may help some.



JLWEST
Lazuras ver 2.0.2 
 FPC 3.0.4, Lazarus IDE v1.8.2 Windows 10 Pro
 Intel i7 770K CPU 4.2GHz 32702MB Ram
GeForce GTX 1080 Graphics - 8 Gig
3952 GB (1.5 SSD)

JLWest

  • Hero Member
  • *****
  • Posts: 545
Re: DataProblems Maybe
« Reply #63 on: April 29, 2019, 12:26:29 am »
Some Things I have tried.

Remember Your:  Test/demo ExtractWord
                ref: DataProblems Maybe
                     https://forum.lazarus.freepascal.org/index.php/topic,45150.0.html 

I Started it and pointed it a the MstRegions file.

On the first record it Decomposed 11 fields. (That''''s really a nice little utility program).
But all the rest of the records were good.

Just a thought: You also wrote a procedure to get rid of the UTF8 BOM file and save it to a ASCII file.

So I'm thinking about taking the ExtractWord Demo and add the procedure to it.

That way I could bring up a file, see if it is extracting ok, if not then save it as an ASCII.

All of this data comes from one source = Apt.Dat, 7.0 Million lines.

 A program reads that and produces the ByAirports.txt.


          Apt.dat  7.9 Million 

     ByAirports.txt          36,000 +- Extrated from Apt.dat file
        MstRegions.txt       Just an extract of ByAirports.txt records without and 'Nil'
        CountryRegions.txt   Extracted from ByAirports.txt

In the program MstRegions I load in the left hand listbox a list of all the airport ICAO's  (KPHX, KLAS ect.) extracted from MstRegions.txt  when I load MstRegions.txt.

In the middle of the screen, Top istbox I load  ByAirports.txt 36,000 +- record. And in the lower middle box I load MstRegions.txt 9,600+-.
In the far right Lstbox I load CountryRegions.txt which was extracted from MstRegions. this was done in a seperat program.

In order to do what I really want to do I need a clean file with no 'Nil''s in any record and all records accounted for.

So I took the ByAirports.txt file and extracted record with no 'Nil's and made MstRegions.txt.(seperate program).

Now I can determine the Country field of each record that is 'Nil' in two way's.

First Way:
 
Region field isn't Nil but has a valid code determined from the right listbox then I can plug in the country. This doesn't always work because some countries (i.e US, China, Russia all have multi Regional ). But countries have 1 region.

Second Method:

Use the Haverstine formula. Take the record with the Nil country Listbox2 Click on it and it becomes the target record (Displayed in the TEdit contro at the top of the screen).

Click on a MstRegions record (Lower Middle Listbox). It's record will be displayed a TEdit located in the middle of the screen.

Now click on the Calc Dist button.

It will give you the distance between the two airports. (Worked properly I think).

I checked against Online site.

Record: [00E][6781690][Nil][Nil][Nil][33.49421222][-111.64124806] which
is a few records down in the top listbox is good one to try.

Click on it.

In the bottom Listbox MstRegions try and find the record:

[AZ66][7175661][Mesa][United States][K2][33.340547222][-109.494466667] and click on it.

It's going to tell you that you are 124 mi  from Mesa Arizona.

The web says: 4227 North 88th Street, Mesa, Arizona 85207, United States.
So I may have a bug in my haverstine routing.

Under the middle listboxs there is a TEdit labeled Country with a small button next to it. Press the small button and it changes the target record Country to United States.

With the listboxes and data I can write code to fill in a lot of the 'Nil'. 

After running the program I can save a new MstRegions file.

It has a few bugs I need to fix and resolve the data issue.

NOTE:----------------------------------------------------
So after writing this up I got to thinking.

All data comes from Apt.Dat. I generate a 36,000 ByAirports.txt  file in a seperate program.

   I change the MstRegions program to load the ByAirports.txt and then populate  the Listbox1, Listbox3, and Listbox5 from Listbox2.

So far I no longer get the error on the first record in listbox2.

 

 
 




                               

« Last Edit: April 29, 2019, 06:51:37 am by JLWest »
JLWEST
Lazuras ver 2.0.2 
 FPC 3.0.4, Lazarus IDE v1.8.2 Windows 10 Pro
 Intel i7 770K CPU 4.2GHz 32702MB Ram
GeForce GTX 1080 Graphics - 8 Gig
3952 GB (1.5 SSD)

Thausand

  • Full Member
  • ***
  • Posts: 227
Re: DataProblems Maybe
« Reply #64 on: May 02, 2019, 05:03:52 am »
....
It has a few bugs I need to fix and resolve the data issue.

NOTE:----------------------------------------------------
So after writing this up I got to thinking.

All data comes from Apt.Dat. I generate a 36,000 ByAirports.txt  file in a seperate program.

   I change the MstRegions program to load the ByAirports.txt and then populate  the Listbox1, Listbox3, and Listbox5 from Listbox2.

So far I no longer get the error on the first record in listbox2.
Is good no have error any more  :)

I forget i make small program make data verificate and show. This is tell that data have problem in other way (not only you error).

Code: Text  [Select]
  1. Processing file "ByAirport.txt"
  2.  
  3. ... show unique count
  4.     Line   Error    ICAO    hash    City Country  Region     Lat     lon    Time
  5.    34680       0   34680   34680   14138     259     248   34457   34537   35 sec (968.17 lines/second))
  6.  
  7. Statistics dtICAO :
  8.      Shortest = "00C"  (3 characters)
  9.      Longest  = "XCYCDL8"  (7 characters)
  10.  Lowest count = "00C"  (1 times)
  11. Highest count = "00CA"  (1 times)
  12.  
  13. Statistics dtHash :
  14.      Shortest = "4"  (1 characters)
  15.      Longest  = "1022949"  (7 characters)
  16.  Lowest count = "1021615"  (1 times)
  17. Highest count = "1022949"  (1 times)
  18.  
  19. Statistics dtCity :
  20.      Shortest = "Ie"  (2 characters)
  21.      Longest  = "Aberdeen Proving Grounds(Abe"  (28 characters)
  22.  Lowest count = "(Cagayan De Oro)"  (1 times)
  23. Highest count = "Nil"  (9335 times)
  24.  
  25. Statistics dtCountry :
  26.      Shortest = "CA"  (2 characters)
  27.      Longest  = "British Indian Ocean Territo"  (28 characters)
  28.  Lowest count = "Anguilla"  (1 times)
  29. Highest count = "United States"  (14645 times)
  30.  
  31. Statistics dtRegion :
  32.      Shortest = "AG"  (2 characters)
  33.      Longest  = "Nil"  (3 characters)
  34.  Lowest count = "AN"  (1 times)
  35. Highest count = "Nil"  (22463 times)
  36.  
  37. Statistics dtLat :
  38.      Shortest = "9"  (1 characters)
  39.      Longest  = "33.1497222222222"  (16 characters)
  40.  Lowest count = "-0.019722222"  (1 times)
  41. Highest count = "-23.59055556"  (6 times)
  42.  
  43. Statistics dtLon :
  44.      Shortest = "23"  (2 characters)
  45.      Longest  = "-106.869166666667"  (17 characters)
  46.  Lowest count = "-0.000944"  (1 times)
  47. Highest count = "-46.65805556"  (4 times)
  48.  
  49. ---------- List dtCountry ----------
  50.    1 : (10) "Afghanistan"
  51.    2 : (5) "Albania"
  52.    3 : (47) "Algeria"
  53.    4 : (3) "American Samoa"
  54.    5 : (27) "Angola"
  55.    6 : (1) "Anguilla"
  56.    7 : (9) "Antarctica"
  57.    8 : (3) "Antigua and Barbuda"
  58.    9 : (164) "Argentina"
  59.   10 : (1) "Arizona"
  60.   11 : (5) "Armenia"
  61.   12 : (1) "Aruba"
  62.   13 : (1486) "Australia"
  63.   14 : (54) "Austria"
  64.   15 : (9) "Azerbaijan"
  65.   16 : (38) "Bahamas"
  66.   17 : (2) "Bahrain"
  67.   18 : (13) "Bangladesh"
  68.   19 : (1) "Barbados"
  69.   20 : (9) "Belarus"
  70.   21 : (43) "Belgium"
  71.   22 : (1) "Belize"
  72.   23 : (2) "Benin"
  73.   24 : (1) "Bermuda"
  74.   25 : (3) "Bhutan"
  75.   26 : (38) "Bolivia"
  76.   27 : (4) "Bosnia and Herzegovina"
  77.   28 : (28) "Botswana"
  78.   29 : (3758) "Brazil"
  79.   30 : (1) "British Indian Ocean Territo"
  80.   31 : (2) "British Virgin Islands"
  81.   32 : (2) "Brunei"
  82.   33 : (22) "Bulgaria"
  83.   34 : (2) "Burkina Faso"
  84.   35 : (1) "Burkina Faso7"
  85.   36 : (36) "Burma"
  86.   37 : (1) "Burundi"
  87.   38 : (14) "CA"
  88.   39 : (6) "Cambodia"
  89.   40 : (10) "Cameroon"
  90.   41 : (675) "Canada"
  91.   42 : (9) "Cape Verde"
  92.   43 : (3) "Caribbean Netherlands"
  93.   44 : (1) "Carroll"
  94.   45 : (3) "Cayman Islands"
  95.   46 : (2) "Central African Republic"
  96.   47 : (5) "Chad"
  97.   48 : (176) "Chile"
  98.   49 : (102) "China"
  99.   50 : (1) "Christmas Island"
  100.   51 : (1) "Cocos (Keeling) Islands"
  101.   52 : (92) "Colombia"
  102.   53 : (3) "Comoros"
  103.   54 : (4) "Congo (Brazzaville)"
  104.   55 : (27) "Congo (Kinshasa)"
  105.   56 : (10) "Cook Islands"
  106.   57 : (26) "Costa Rica"
  107.   58 : (18) "Croatia"
  108.   59 : (31) "Cuba"
  109.   60 : (1) "Curaçao"
  110.   61 : (3) "Cyprus"
  111.   62 : (99) "Czechia"
  112.   63 : (7) "Côte d'Ivoire"
  113.   64 : (87) "Denmark"
  114.   65 : (1) "Djibouti"
  115.   66 : (2) "Dominica"
  116.   67 : (15) "Dominican Republic"
  117.   68 : (50) "Ecuador"
  118.   69 : (56) "Egypt"
  119.   70 : (4) "El Salvador"
  120.   71 : (2) "Equatorial Guinea"
  121.   72 : (3) "Eritrea"
  122.   73 : (13) "Estonia"
  123.   74 : (22) "Ethiopia"
  124.   75 : (2) "Falkland Islands"
  125.   76 : (8) "Faroe Islands"
  126.   77 : (21) "Fiji"
  127.   78 : (92) "Finland"
  128.   79 : (3) "FR"
  129.   80 : (441) "France"
  130.   81 : (5) "French Guiana"
  131.   82 : (48) "French Polynesia"
  132.   83 : (11) "Gabon"
  133.   84 : (1) "Gambia"
  134.   85 : (9) "Georgia"
  135.   86 : (504) "Germany"
  136.   87 : (8) "Ghana"
  137.   88 : (1) "Gibraltar"
  138.   89 : (56) "Greece"
  139.   90 : (17) "Greenland"
  140.   91 : (1) "Greenville"
  141.   92 : (2) "Grenada"
  142.   93 : (5) "Guadeloupe"
  143.   94 : (2) "Guam"
  144.   95 : (10) "Guatemala"
  145.   96 : (2) "Guernsey"
  146.   97 : (4) "Guinea"
  147.   98 : (3) "Guinea-Bissau"
  148.   99 : (16) "Guyana"
  149.  100 : (6) "Haiti"
  150.  101 : (10) "Honduras"
  151.  102 : (1) "Hong Kong"
  152.  103 : (76) "Hungary"
  153.  104 : (81) "Iceland"
  154.  105 : (195) "India"
  155.  106 : (181) "Indonesia"
  156.  107 : (88) "Iran"
  157.  108 : (16) "Iraq"
  158.  109 : (36) "Ireland"
  159.  110 : (1) "Isle of Man"
  160.  111 : (21) "Israel"
  161.  112 : (120) "Italy"
  162.  113 : (6) "Jamaica"
  163.  114 : (130) "Japan"
  164.  115 : (1) "Jersey"
  165.  116 : (8) "Jordan"
  166.  117 : (1) "K1"
  167.  118 : (21) "Kazakhstan"
  168.  119 : (14) "Kenya"
  169.  120 : (20) "Kiribati"
  170.  121 : (1) "Kosovo"
  171.  122 : (3) "Kuwait"
  172.  123 : (4) "Kyrgyzstan"
  173.  124 : (8) "Laos"
  174.  125 : (7) "Latvia"
  175.  126 : (3) "Lebanon"
  176.  127 : (5) "Lesotho"
  177.  128 : (2) "Liberia"
  178.  129 : (41) "Libya"
  179.  130 : (1) "Liechtenstein"
  180.  131 : (7) "Lithuania"
  181.  132 : (3) "Luxembourg"
  182.  133 : (1) "Macau"
  183.  134 : (12) "Macedonia"
  184.  135 : (28) "Madagascar"
  185.  136 : (7) "Malawi"
  186.  137 : (43) "Malaysia"
  187.  138 : (6) "Maldives"
  188.  139 : (8) "Mali"
  189.  140 : (2) "Malta"
  190.  141 : (7) "Marshall Islands"
  191.  142 : (1) "Martinique"
  192.  143 : (10) "Mauritania"
  193.  144 : (2) "Mauritius"
  194.  145 : (1) "Mayotte"
  195.  146 : (132) "Mexico"
  196.  147 : (4) "Micronesia"
  197.  148 : (3) "Moldova"
  198.  149 : (1) "Monaco"
  199.  150 : (22) "Mongolia"
  200.  151 : (3) "Montenegro"
  201.  152 : (1) "Montserrat"
  202.  153 : (26) "Morocco"
  203.  154 : (18) "Mozambique"
  204.  155 : (1) "Myanmar"
  205.  156 : (55) "Namibia"
  206.  157 : (1) "Namibia."
  207.  158 : (1) "Nauru"
  208.  159 : (41) "Nepal"
  209.  160 : (34) "Netherlands"
  210.  161 : (17) "New Caledonia"
  211.  162 : (109) "New Zealand"
  212.  163 : (10) "Nicaragua"
  213.  164 : (7) "Niger"
  214.  165 : (27) "Nigeria"
  215.  166 : (7099) "Nil"
  216.  167 : (1) "Niue"
  217.  168 : (1) "Norfolk Island"
  218.  169 : (1) "North Korea"
  219.  170 : (9) "Northern Mariana Islands"
  220.  171 : (142) "Norway"
  221.  172 : (6) "Oman"
  222.  173 : (68) "Pakistan"
  223.  174 : (3) "Palau"
  224.  175 : (2) "Palestinian Territory"
  225.  176 : (22) "Panama"
  226.  177 : (75) "Papua New Guinea"
  227.  178 : (11) "Paraguay"
  228.  179 : (1) "Peoples Republic of China"
  229.  180 : (1) "Peru"
  230.  181 : (79) "Perú"
  231.  182 : (73) "Philippines"
  232.  183 : (92) "Poland"
  233.  184 : (52) "Portugal"
  234.  185 : (31) "Puerto Rico"
  235.  186 : (3) "Qatar"
  236.  187 : (1) "Republic of Ireland (Eire)"
  237.  188 : (39) "Romania"
  238.  189 : (167) "Russia"
  239.  190 : (7) "Russian Federation"
  240.  191 : (1) "Russin Federation"
  241.  192 : (5) "Rwanda"
  242.  193 : (2) "Réunion"
  243.  194 : (1) "Saint Barthélemy"
  244.  195 : (2) "Saint Helena"
  245.  196 : (2) "Saint Kitts and Nevis"
  246.  197 : (2) "Saint Lucia"
  247.  198 : (1) "Saint Martin"
  248.  199 : (2) "Saint Pierre and Miquelon"
  249.  200 : (5) "Saint Vincent and the Grenad"
  250.  201 : (4) "Samoa"
  251.  202 : (2) "Sao Tome and Principe"
  252.  203 : (62) "Saudi Arabia"
  253.  204 : (10) "Senegal"
  254.  205 : (7) "Serbia"
  255.  206 : (8) "Seychelles"
  256.  207 : (1) "Sierra Leone"
  257.  208 : (6) "Singapore"
  258.  209 : (30) "Slovakia"
  259.  210 : (16) "Slovenia"
  260.  211 : (23) "Solomon Islands"
  261.  212 : (6) "Somalia"
  262.  213 : (109) "South Africa"
  263.  214 : (77) "South Korea"
  264.  215 : (4) "South Sudan"
  265.  216 : (107) "Spain"
  266.  217 : (10) "Sri Lanka"
  267.  218 : (11) "Sudan"
  268.  219 : (8) "Suriname"
  269.  220 : (1) "Swaziland"
  270.  221 : (164) "Sweden"
  271.  222 : (1) "Swiss"
  272.  223 : (70) "Switzerland"
  273.  224 : (23) "Syria"
  274.  225 : (21) "Taiwan"
  275.  226 : (6) "Tajikistan"
  276.  227 : (14) "Tanzania"
  277.  228 : (61) "Thailand"
  278.  229 : (6) "Timor-Leste"
  279.  230 : (2) "Togo"
  280.  231 : (6) "Tonga"
  281.  232 : (2) "Trinidad and Tobago"
  282.  233 : (13) "Tunisia"
  283.  234 : (74) "Turkey"
  284.  235 : (10) "Turkmenistan"
  285.  236 : (7) "Turks and Caicos Islands"
  286.  237 : (1) "Tuvalu"
  287.  238 : (6) "U.S. Virgin Islands"
  288.  239 : (1) "U.S.A."
  289.  240 : (18) "Uganda"
  290.  241 : (26) "Ukraine"
  291.  242 : (19) "United Arab Emirates"
  292.  243 : (251) "United Kingdom"
  293.  244 : (1) "United Sates"
  294.  245 : (14645) "United States"
  295.  246 : (4) "United States Minor Outlying"
  296.  247 : (2) "United States of America"
  297.  248 : (11) "Uruguay"
  298.  249 : (25) "US"
  299.  250 : (26) "USA"
  300.  251 : (15) "Uzbekistan"
  301.  252 : (23) "Vanuatu"
  302.  253 : (100) "Venezuela"
  303.  254 : (24) "Vietnam"
  304.  255 : (2) "Wallis and Futuna"
  305.  256 : (3) "Western Sahara"
  306.  257 : (13) "Yemen"
  307.  258 : (8) "Zambia"
  308.  259 : (17) "Zimbabwe"
  309.  

  • City name cut
  • Country name cut
  • Other error country name

I not have print list city and is many big and make read.

The country name error is not very good  :(

Have example how many time want write United States ?

I have make highlight lines that is no good and i not even good know all country english write ...

Is city and country cut origin apt.dat ? How big apt.dat unpack ? And can download apt.dat ?

JLWest

  • Hero Member
  • *****
  • Posts: 545
Re: DataProblems Maybe
« Reply #65 on: May 02, 2019, 06:38:35 am »
@Thausand

I'll download your demo procedure and run it up.

The apt.dat file is 7.9 million text record.  Can't upload but, I can post it on my Gdrive and post the link here.

It will take me bit.



JLWEST
Lazuras ver 2.0.2 
 FPC 3.0.4, Lazarus IDE v1.8.2 Windows 10 Pro
 Intel i7 770K CPU 4.2GHz 32702MB Ram
GeForce GTX 1080 Graphics - 8 Gig
3952 GB (1.5 SSD)

lucamar

  • Hero Member
  • *****
  • Posts: 1810
Re: DataProblems Maybe
« Reply #66 on: May 02, 2019, 07:45:18 am »
I can't get Cleanbom to work. I Have never run the DOS box programs. (Years a go)

I think I get in a dos box cd to where the Cleanbom resides and do the following

Cleanbom E:\files\Regions\MstRegions.txt  E:\files\Regions\MstRegions.bak

Basically, yes that's the procedure: put the cleanbom.exe in the same directory where the data files are, open a console window (or DOS prompt, or however it's called in your system), CD to that directory and type the command, Basically:
Code: [Select]
E:
cd E:\files\Regions
cleanbom MstRegions.txt
exit

It it cleaned anything, the clean file will be MstRegions.txt and the original will have been renamed to MstRegions.bak.

I haven't had time to give your program more than a quick look, sorry, but I think today or tomorrow I'll have some free hours to dedicate it. Cross your fingers!

Quote
So I'm thinking about taking the ExtractWord Demo and add the procedure to it.
That way I could bring up a file, see if it is extracting ok, if not then save it as an ASCII.

Yes, you could do that too. Or take all the ideas together and make a "validation" program like Thausand's. It shouldn't be too long. And you get to play with streams ;)

Quote
   I change the MstRegions program to load the ByAirports.txt and then populate  the Listbox1, Listbox3, and Listbox5 from Listbox2.

So far I no longer get the error on the first record in listbox2.

That's because ByAirports.txt is clean already, at least the one I downloaded here. I suspect that at some point or other you created or edited some of those files with an editor that wrote them back as UTF8 with BOM, and there started your problems.

Another good, solid solution may be to create a TFileStream descendant which ignored that BOM and returned only the normal data: you could then use all those nice LoadFromStream methods :) . It shouldn't be too difficult...
« Last Edit: May 02, 2019, 07:52:07 am by lucamar »
Turbo Pascal 3 CP/M - Amstrad PCW 8256 (512 KB !!!) :P
Lazarus 1.8.4 & 2.0.2 w/FPC 3.0.4 on:
(K|L)Ubuntu 12..16, Windows XP SP3, various DOSes.

JLWest

  • Hero Member
  • *****
  • Posts: 545
Re: DataProblems Maybe
« Reply #67 on: May 02, 2019, 08:17:09 am »
@lucumar

I uploaded everything to my  posted links. Program, data files.

Everything can be dumped into a single folder and run from there. Evan a short Readme.


"Yes, you could do that too. Or take all the ideas together and make a "validation" program like Thausand's. It shouldn't be too long. And you get to play with streams"

I have, loading one of the listboxes with streams. CountryRegions.txt. Works like a dream. Can't believe you didn't tell me about this long ago. Lol.

I have to take a look at  @Thausand latest.

I may be offline for a day. Doctors, Hospital. The last time I went in for this I was there for a week.

At the time my kid got some program that would allow me to operate my computer from his laptop using the hospital's network log in as a guest. Pretty neat.

 and 
JLWEST
Lazuras ver 2.0.2 
 FPC 3.0.4, Lazarus IDE v1.8.2 Windows 10 Pro
 Intel i7 770K CPU 4.2GHz 32702MB Ram
GeForce GTX 1080 Graphics - 8 Gig
3952 GB (1.5 SSD)

lucamar

  • Hero Member
  • *****
  • Posts: 1810
Re: DataProblems Maybe
« Reply #68 on: May 02, 2019, 11:58:55 am »
I may be offline for a day. Doctors, Hospital. The last time I went in for this I was there for a week.

At the time my kid got some program that would allow me to operate my computer from his laptop using the hospital's network log in as a guest. Pretty neat.

Yeah, I know how you feel. I passed a week in the hospital after one the surgeries, got back home ... and had to go back next day with a meningitis. One month in the hospital, all told!

That program ... a (more or less) standard remote desktop, isn't it? I tried some but the hospital's outbound net had so many redirections and indirections that I ended up just reading novels in the tablet :D
« Last Edit: May 02, 2019, 12:01:24 pm by lucamar »
Turbo Pascal 3 CP/M - Amstrad PCW 8256 (512 KB !!!) :P
Lazarus 1.8.4 & 2.0.2 w/FPC 3.0.4 on:
(K|L)Ubuntu 12..16, Windows XP SP3, various DOSes.

Thausand

  • Full Member
  • ***
  • Posts: 227
Re: DataProblems Maybe
« Reply #69 on: May 02, 2019, 06:37:37 pm »
Hi hello JLWest,

I have to take a look at  @Thausand latest.
Oh, i sorry. I forget attach (i make now).

I have make new program (i learn use other class) and is make different idea is verificate. Other program is idea process. Because new program i not have finish (and you was working so i not continue work program). I now have make comment and have refactor. Is no finish but is now better read (i hope)  :)

If want more option then can add you or write what want and maybe i can add. Better is you make add  :P

Please no upload me apt.dat if zip more big 1GB. Is text so make > 10-20GB unpack and no have space for test now.

Quote
I may be offline for a day. Doctors, Hospital. The last time I went in for this I was there for a week.
I make wish well for you.

lucamar

  • Hero Member
  • *****
  • Posts: 1810
Re: DataProblems Maybe
« Reply #70 on: May 02, 2019, 08:14:06 pm »
Please no upload me apt.dat if zip more big 1GB. Is text so make > 10-20GB unpack and no have space for test now.

It's not so big, just about 300 MiB and as it's text it compresses well: down to around 100 MiB.
Turbo Pascal 3 CP/M - Amstrad PCW 8256 (512 KB !!!) :P
Lazarus 1.8.4 & 2.0.2 w/FPC 3.0.4 on:
(K|L)Ubuntu 12..16, Windows XP SP3, various DOSes.

JLWest

  • Hero Member
  • *****
  • Posts: 545
Re: DataProblems Maybe
« Reply #71 on: May 02, 2019, 10:45:40 pm »
@Tausand

If it's to big then open with Notepad and cut out a hundred records of so.
You can do that with all the text file. 
JLWEST
Lazuras ver 2.0.2 
 FPC 3.0.4, Lazarus IDE v1.8.2 Windows 10 Pro
 Intel i7 770K CPU 4.2GHz 32702MB Ram
GeForce GTX 1080 Graphics - 8 Gig
3952 GB (1.5 SSD)

Thausand

  • Full Member
  • ***
  • Posts: 227
Re: DataProblems Maybe
« Reply #72 on: May 03, 2019, 07:34:26 pm »
It's not so big, just about 300 MiB and as it's text it compresses well: down to around 100 MiB.
Hi hello lucamar,

That then ok. I can make  :)

Thanksy!

@Tausand

If it's to big then open with Notepad and cut out a hundred records of so.
You can do that with all the text file. 
Hi hello JLWest,

Use notepad (i not have, i Linux) then make begin error BOM  ;D

Then make small no good for make good data analyse.

I read specificate "X-Plane APT 1100" and make many question for file "ByAirport.txt". I make new read what want you because think i not understand all. Have example name ariport then specificate write "Text string (up to 40 characters)" and you program MasterAirport make 28   :o

I have question because i look and no see. Were can find apt.dat for download ?
« Last Edit: May 03, 2019, 07:41:15 pm by Thausand »

JLWest

  • Hero Member
  • *****
  • Posts: 545
Re: DataProblems Maybe
« Reply #73 on: May 04, 2019, 01:21:14 am »
@Thausand Link to my GoogleDrive.


https://drive.google.com/open?id=1J4EanhRBD5gYr4gWADyI4BCB5sN1dTeE

@Lucumar It's called:

https://www.splashtop.com/

Works good. Has a free trial and a subscription of $5US month.
JLWEST
Lazuras ver 2.0.2 
 FPC 3.0.4, Lazarus IDE v1.8.2 Windows 10 Pro
 Intel i7 770K CPU 4.2GHz 32702MB Ram
GeForce GTX 1080 Graphics - 8 Gig
3952 GB (1.5 SSD)

JLWest

  • Hero Member
  • *****
  • Posts: 545
Re: DataProblems Maybe
« Reply #74 on: May 04, 2019, 01:30:01 am »
 @Thausand

Notepad or notepad++ can't handle apt.dat very well.

There is a free editor for very large files that wokrs pretty good. It's called Kainet

http://www.kainet.ch/Editor-for-very-large-files.

It is very basic. but fast.

JLWEST
Lazuras ver 2.0.2 
 FPC 3.0.4, Lazarus IDE v1.8.2 Windows 10 Pro
 Intel i7 770K CPU 4.2GHz 32702MB Ram
GeForce GTX 1080 Graphics - 8 Gig
3952 GB (1.5 SSD)