Notices
Computer & Technology Related Post here for help and discussion of computing and related technology. Internet, TVs, phones, consoles, computers, tablets and any other gadgets.

Tool for grabbing whole website?

Thread Tools
 
Search this Thread
 
Old 20 November 2007, 04:07 PM
  #1  
Scotsman
Scooby Regular
Thread Starter
iTrader: (3)
 
Scotsman's Avatar
 
Join Date: Apr 2001
Location: Stirling
Posts: 2,168
Likes: 0
Received 0 Likes on 0 Posts
Default Tool for grabbing whole website?

Hi

I've got a website that I'm looking to save to my harddrive - but rather than saving each page, I was wondering if there are recommended tools that I can point at the start page and it will then follow each link and download the arrived at pages. Note that all of the links aren't text links but image map links (click on a picture).

Thanks!
Richard.
Old 20 November 2007, 04:13 PM
  #2  
StiX
Scooby Regular
Support Scoobynet!
iTrader: (4)
 
StiX's Avatar
 
Join Date: Jan 2004
Location: Bridgend, S Wales
Posts: 2,293
Likes: 0
Received 0 Likes on 0 Posts
Default

Can't you just log into wherever you have hosted it and copy all the files?

Or use an ftp client and do the same?
Old 20 November 2007, 04:21 PM
  #3  
Scotsman
Scooby Regular
Thread Starter
iTrader: (3)
 
Scotsman's Avatar
 
Join Date: Apr 2001
Location: Stirling
Posts: 2,168
Likes: 0
Received 0 Likes on 0 Posts
Default

We've paid another company to build the site for us and they are hosting it (we don't have access). We've had some problems with them recently and I wanted to have a local copy of the site prior to having some robust discussions with them. Unfortunately that means I only have access through a browser to the published site.
Old 20 November 2007, 04:22 PM
  #4  
Scotsman
Scooby Regular
Thread Starter
iTrader: (3)
 
Scotsman's Avatar
 
Join Date: Apr 2001
Location: Stirling
Posts: 2,168
Likes: 0
Received 0 Likes on 0 Posts
Default

Just found a selection on Google - such as HTTrack Website Copier - Offline Browser and Download entire website or download web pages with SurfOffline - convenient offline browser with easy-to-use interface.
Old 20 November 2007, 04:24 PM
  #5  
StiX
Scooby Regular
Support Scoobynet!
iTrader: (4)
 
StiX's Avatar
 
Join Date: Jan 2004
Location: Bridgend, S Wales
Posts: 2,293
Likes: 0
Received 0 Likes on 0 Posts
Default

Originally Posted by Scotsman
We've paid another company to build the site for us and they are hosting it (we don't have access). We've had some problems with them recently and I wanted to have a local copy of the site prior to having some robust discussions with them. Unfortunately that means I only have access through a browser to the published site.
Ah - gotcha! I see you have found some now anyway
Old 20 November 2007, 04:25 PM
  #6  
RichB
Scooby Regular
 
RichB's Avatar
 
Join Date: Apr 1999
Location: Bore Knee Muff
Posts: 3,666
Likes: 0
Received 0 Likes on 0 Posts
Default

Ive used HTTrack before and its quite good.
Old 20 November 2007, 04:32 PM
  #7  
Scotsman
Scooby Regular
Thread Starter
iTrader: (3)
 
Scotsman's Avatar
 
Join Date: Apr 2001
Location: Stirling
Posts: 2,168
Likes: 0
Received 0 Likes on 0 Posts
Default

Thanks - I'll give that a shot first.
Old 20 November 2007, 04:58 PM
  #8  
RichB
Scooby Regular
 
RichB's Avatar
 
Join Date: Apr 1999
Location: Bore Knee Muff
Posts: 3,666
Likes: 0
Received 0 Likes on 0 Posts
Default

be aware that obviously any content management system or database content wont be accessible this way, you will just get a flat version of the site.
Also any URLs that pass anything between themselves with the ? or & may not end up quite as you'd hoped.
If its purely static then you'll be fine.
Old 20 November 2007, 05:00 PM
  #9  
Scotsman
Scooby Regular
Thread Starter
iTrader: (3)
 
Scotsman's Avatar
 
Join Date: Apr 2001
Location: Stirling
Posts: 2,168
Likes: 0
Received 0 Likes on 0 Posts
Default

It is a static site (just html files) - but I'm having problems getting any of the website grabbing programmes to follow the image maps links (if that's the right term).
Old 20 November 2007, 05:12 PM
  #10  
RichB
Scooby Regular
 
RichB's Avatar
 
Join Date: Apr 1999
Location: Bore Knee Muff
Posts: 3,666
Likes: 0
Received 0 Likes on 0 Posts
Default

if you want to PM me the site, ill take a look
Old 20 November 2007, 06:27 PM
  #11  
Luminous
Scooby Regular
iTrader: (3)
 
Luminous's Avatar
 
Join Date: Aug 2004
Location: Muppetising life
Posts: 15,449
Likes: 0
Received 0 Likes on 0 Posts
Default

Webzip is pretty good, not used it for years. I used it routinely when I had a dial up account, would let it sync changes for offline browsing later on.

You can tell it how you want it to follow links, how how deep to go etc. For example you may want it to follow all pages that stay within that site, but not follow any links that go to external sites. You can also specify what types of file you want it to look for...not that I ever in my younger years pointed it at a **** site

No idea what the software is like now, take a look and see if it can help.

Last edited by Luminous; 20 November 2007 at 06:30 PM.
Old 20 November 2007, 07:18 PM
  #12  
Scotsman
Scooby Regular
Thread Starter
iTrader: (3)
 
Scotsman's Avatar
 
Join Date: Apr 2001
Location: Stirling
Posts: 2,168
Likes: 0
Received 0 Likes on 0 Posts
Default

Originally Posted by Luminous
Webzip is pretty good, not used it for years. I used it routinely when I had a dial up account, would let it sync changes for offline browsing later on.

You can tell it how you want it to follow links, how how deep to go etc. For example you may want it to follow all pages that stay within that site, but not follow any links that go to external sites. You can also specify what types of file you want it to look for...not that I ever in my younger years pointed it at a **** site

No idea what the software is like now, take a look and see if it can help.
Thanks - I'll give that one a try
Old 20 November 2007, 07:50 PM
  #13  
Scotsman
Scooby Regular
Thread Starter
iTrader: (3)
 
Scotsman's Avatar
 
Join Date: Apr 2001
Location: Stirling
Posts: 2,168
Likes: 0
Received 0 Likes on 0 Posts
Default

Well, Webzip is superb - worked first time and grabed everything it should off. Thanks again for the recommendation.
Old 21 November 2007, 10:47 AM
  #14  
Luminous
Scooby Regular
iTrader: (3)
 
Luminous's Avatar
 
Join Date: Aug 2004
Location: Muppetising life
Posts: 15,449
Likes: 0
Received 0 Likes on 0 Posts
Default

Cool, glad its still as good as it was
Old 21 November 2007, 11:38 AM
  #15  
PeteBrant
Scooby Regular
 
PeteBrant's Avatar
 
Join Date: Sep 2006
Location: Worthing..
Posts: 7,575
Likes: 0
Received 0 Likes on 0 Posts
Default

Can't you set firefox to do something along these lines?
Old 21 November 2007, 12:14 PM
  #16  
Pumpkin
Scooby Senior
 
Pumpkin's Avatar
 
Join Date: Mar 1999
Location: Norwich
Posts: 1,835
Likes: 0
Received 0 Likes on 0 Posts
Default

Webreaper works very well for this sort of job.
Old 21 November 2007, 03:23 PM
  #17  
GazTheHat
Scooby Regular
 
GazTheHat's Avatar
 
Join Date: Aug 2005
Location: 392/361 MY04 STi
Posts: 7,638
Likes: 0
Received 0 Likes on 0 Posts
Default

Blackwidow also does the job.
Old 21 November 2007, 07:20 PM
  #18  
Avi
Scooby Regular
 
Avi's Avatar
 
Join Date: Apr 2001
Location: Manchester
Posts: 5,084
Likes: 0
Received 0 Likes on 0 Posts
Default

Originally Posted by Pumpkin
Webreaper works very well for this sort of job.
I've used webreaper for this too!
Old 22 November 2007, 10:41 AM
  #19  
Pumpkin
Scooby Senior
 
Pumpkin's Avatar
 
Join Date: Mar 1999
Location: Norwich
Posts: 1,835
Likes: 0
Received 0 Likes on 0 Posts
Default

I think someone once used Webreaper to archive Scoobynet and brought the system down!
Related Topics
Thread
Thread Starter
Forum
Replies
Last Post
Little V
Wales
18
09 October 2015 09:45 PM
LostUser
Non Scooby Related
11
29 September 2015 11:00 AM
jobegold@hotmail.co.uk
ScoobyNet General
2
27 September 2015 09:44 PM



Quick Reply: Tool for grabbing whole website?



All times are GMT +1. The time now is 11:06 PM.