Recent

Author Topic: simple text web browser help  (Read 8196 times)

wpflum

  • Sr. Member
  • ****
  • Posts: 287
simple text web browser help
« on: August 14, 2011, 03:48:49 pm »
I'm working on a isbn scraper to pull book data off of a web site.  What I need is a simple way to call a site something like 'www.books.com/isbn/isbnnumber' and pull the response into a variable/list/whatever that I can then parse for the info I need.  No other interaction is required.  So its a call, read and close scenario.  It looks like lnet might work but it seems like a lot more that I need, is there anything else I should look at or can some one show me a small example program that does this??   I was going to use perl and Lynx but since I want to roll this into a complete ebook library program I figured I'd be better off doing the entire thing in Lazarus. 

Any ideas/comments/rants???


P.S.  I know I could use Calibre but I have issues with how it handles the Libraries it creates, specifically needing to move all the books to it's own directory instead of leaving them where it finds them.


mica

  • Full Member
  • ***
  • Posts: 196
Re: simple text web browser help
« Reply #1 on: August 14, 2011, 05:51:10 pm »

wpflum

  • Sr. Member
  • ****
  • Posts: 287
Re: simple text web browser help
« Reply #2 on: August 14, 2011, 06:47:36 pm »
The first one

http://www.benibela.de/sources_en.html#internettools

looks interesting and if I can figure it out it might save me some time.

What I'm really looking for right now is some straight forward code to call the web site and receive the page.  I'm going to play with lnet right now but would appreciate any direction.  ;D

Leledumbo

  • Hero Member
  • *****
  • Posts: 8833
  • Programming + Glam Metal + Tae Kwon Do = Me
Re: simple text web browser help
« Reply #3 on: August 14, 2011, 11:38:06 pm »
lNet and Synapse are both capable of doing this. Just use the correct protocol class in lNet, and Synapse' HTTPGetText (+SSL for https) would do the job.

felipemdc

  • Administrator
  • Hero Member
  • *
  • Posts: 3538


wpflum

  • Sr. Member
  • ****
  • Posts: 287
Re: simple text web browser help
« Reply #6 on: August 16, 2011, 02:22:21 pm »
http://www.benibela.de/documentation/internettools/
or
http://www.ararat.cz/synapse/doku.php/public:howto

http://www.ararat.cz/synapse/doku.php/public:howto:httpgetpage

Thank you!!  The last one is exactly what I was looking for.  Right now I'm making a system call to run Lynx and grab its output but now I should be able to use this code and keep more of the program self contained.  :)

 

 

TinyPortal © 2005-2018