Reading a text file from a web-site.

[J-G]

.. can it be done? 

I'm sure it can but my attempts, so far, have failed.

I can easily execute a program (say PDF Reader, Music Reader) using OpenDocument('URL'+filename) but that doesn't let me read the 'CONTENT' of the file.

What I want to do is read a simple text file and put each line of the text into an array which will later be used as part of a URL to read (and display) other files. This will make the file location dynamic and 'selectable' rather than hard-coded so I can amend the text file as and when required without having to distribute a new version of the main program.

My latest attempt ...

Code: Pascal  [Select][+][-]

1.  
2. Const
3.   LOC_URL : string = 'http//www.*****...';
4. Var
5.     Location : text;
6.     Links : array[1..6] of string[20];
7.     i : byte;
8.  
9. Begin
10.  assign(Location,LOC_URL);
11.   reset(Location);
12.   for i := 1 to 6 do
13.     read(Location,Links[i]);
14.  
15. end;
16.  

... returned a 'Run Error (123)' which usually means 'Bad Filename' but 'LOC_URL', and therefore 'Location', is a straightforward URL with no 'odd' characters or spaces.

[taazz]

file handling routines can not be used to access network resources. You need to use a net library for that and on top of that you need to know and properly implement the base protocol (http, ftp, smtp etc) you are using. In this case I'm assuming that you need an http client library and there are at least 3 libraries you can choose from 1) indy 2) synapse and 3) the one that comes pre installed with fpc. Personally I have used indy and synapse with good results and I can recommend either of them.

[J-G]

I did think that might be the issue Taazz but was hoping my simple approach might just work.

I already have  fphttpclient, sax_html, dom_html, dom and LCLIntf   in the uses clause of my program and use them to both read the filenames on the web-site and download .PDF, .CAP & .MP3 files but that (as yet) doesn't let me see the content of the files within the program. As far as the .PDFs etc. are concerned that doesn't matter, they are displayed perfectly well with 'OpenDocument(...); using their default viewers.

On reflection I suspect that [Saving] the text file - ie. copying it from the web to the local drive - and then open and read it via the normal file handling routines is the way to proceed.

OpenDocument(...) does enable the specified file to be saved but it brings up a 'Save' dialogue and I was trying to get around that. I could check to see if the 'Location' file has been changed - (has a different time-stamp from the existing version) - and only invoke the save routine if that is true . . . . hmmmm

[guest58172]

Using FpWeb it's very easy too: lookat http://wiki.freepascal.org/fphttpclient#Get_body_of_a_web_page_via_HTTP_protocol.

You might also use TProcess + wget (or curl)

[J-G]

@BBasile
Thanks for the pointer but I can't understand that code. It reads (to me) as if it is intended to UPLOAD some data to a web-site, rather than DOWNLOAD the contents of a file already on a site.
Notwithstanding that, I did try to incorporate it as a procedure :

Code: Pascal  [Select][+][-]

1. procedure CheckForNewURLs;
2. Var
3.   NewURLs : TStringStream;
4.   S : String;
5. begin
6.   With TFPHttpClient.Create(Nil) do
7.     try
8.       NewURLs:=TStringStream.Create('');
9.       FileFormPost('http://www.{domain+Filename}','PostFilenameParam',edtSourceFile.Text,NewURLs);
10.       S:=NewURLs.DataString;
11.       NewURLs.Destroy;
12.     finally
13.       Free;
14.     end;
15. end;
16.  

...but this doesn't compile, throwing error  Identifier not found "edtSourceFile" - -  which doesn't really surprise me since I haven't declared that variable anywhere and can't fathom what it's supposed to be or do; nor do I have any idea what 'PostFilenameParam' is for.

I have successfully written code to read the time-stamp from a local file (a pre-cursor to my second approach to the problem) but can't work out how to do the same for a file on the web-site. I've looked through all the components and attributes of a ThtmlDocument but can't see anything relating to the date.

I'd appreciate any advice on how to achieve this as well.

[taazz]

I have successfully written code to read the time-stamp from a local file (a pre-cursor to my second approach to the problem) but can't work out how to do the same for a file on the web-site. I've looked through all the components and attributes of a ThtmlDocument but can't see anything relating to the date.

There is nothing for dates, you upload a small text file which has a line for the version and an other for the date of the data on the site. You download that file and use it to check if the local data need updating.

[lainz]

usually the version is enough

You can do something like
year.month.day.hour.minutes
str1 := 2017.12.01.08:32
webstr := get that file and read that line...

And compare if str1 <> webstr needs to update...

Edit: taking into account that the online one version string is always newer, never older than the one in the PC

[J-G]

There is nothing for dates, you upload a small text file which has a line for the version and an other for the date of the data on the site. You download that file and use it to check if the local data need updating.

??  Whilst I understand the meaning of each 'word' in that message  -  I don't understand the idea

Can you provide a snippet of code which returns the time-stamp of a text file on a web-site?

[lainz]

There is nothing for dates, you upload a small text file which has a line for the version and an other for the date of the data on the site. You download that file and use it to check if the local data need updating.

??  Whilst I understand the meaning of each 'word' in that message  -  I don't understand the idea

Can you provide a snippet of code which returns the time-stamp of a text file on a web-site?

J-G you need to:

- add a txt file in your site
- the txt has a version code on it like

Code: Pascal  [Select][+][-]

1. 2017.12.01

Then you download that file, and check the version (content of the txt) with one you have locally. Is just a string compare.

If these are different you download the file of your choice because needs updating.

You can replace the version with a timestamp if you want, but is the same.

[J-G]

@ lainz
Thanks - but again I lack understanding.

Perhaps if I explain more fully - - -  I put a text file on the web-site (via FTP) - such as 'URLs.TXT' - containing a number of strings such as '/disney/sop', '/disney/tenor'. That file has a date.

Locally there is also a file called 'URLs.TXT' and I want to compare the dates. If they are the same then no action is necessary but if they are different then the local file needs to be over-written; irrespective of whether the web file is earlier or later - the only thing that matters is that the web file is 'current'.

The data in the file is used to determine where certain .MP3 or .PDF files are stored so that the location can be dynamic rather than hard-coded into the program.

The location of 'URLs.TXT' can be static and therefore hard-coded, just as the location of the local version is.

[lainz]

Ok. Let me explain you again.

You need two files
- Urls.txt
- Urls-version.txt

So you download first Urls-version.txt. Inside that file there's a text, any, in this case we choose a number like 20171201 (that is just a date withouth time) is just an example.

You need to have locally another file called Urls-version.txt with the date of latest downloaded Urls-version.txt

For example 20171130.

So you need to compare 20171130 with 20171201. They are different! So if they are different download Urls.txt, if they are the same download nothing.

When you download Urls-version.txt and they are different you must update Urls.txt and also Urls-version.txt locally to make future compare right.

[J-G]

My understanding is improving       but I'm now committed to other tasks so I have to leave this project for a while    -    (I have three choral concerts over the week-end!)

[lainz]

Great! Good luck in these concerts!

[Almir.Bispo]

Look what you can do:
CSV Comp DB on Windows Server (Under Apache) can create csv (files tables),database (folder) interact with desktop application (created with Lazarus) in real time.Look video (portuguese):
https://www.youtube.com/watch?v=7DSTotVI1_I&t=0s

[J-G]

You need two files
- Urls.txt
- Urls-version.txt

So you download first Urls-version.txt. Inside that file there's a text, any, in this case we choose a number like 20171201 (that is just a date without time) is just an example.

You need to have locally another file called Urls-version.txt with the date of latest downloaded Urls-version.txt

For example 20171130.

So you need to compare 20171130 with 20171201. They are different! So if they are different download Urls.txt, if they are the same download nothing.

When you download Urls-version.txt and they are different you must update Urls.txt and also Urls-version.txt locally to make future compare right.

I understand what you are suggesting lainz but don't see how a second file really helps.

I don't want to DOWNLOAD a file (the URL-version.txt you suggest) to then read what it contains. That means that the user has to make a decision as to where to save it and that could be done with the original file. I just need to read the DATE of the file; if I could read the contents of the URLs-Version.txt file I could also read the contents of the URLs.TXT file which would negate the need to read its date.