Notices
Computer & Technology Related Post here for help and discussion of computing and related technology. Internet, TVs, phones, consoles, computers, tablets and any other gadgets.

Tool for grabbing whole website?

Thread Tools
 
Search this Thread
 
Old Nov 20, 2007 | 04:07 PM
  #1  
Scotsman's Avatar
Scotsman
Thread Starter
Scooby Regular
iTrader: (3)
 
Joined: Apr 2001
Posts: 2,168
Likes: 0
From: Stirling
Default Tool for grabbing whole website?

Hi

I've got a website that I'm looking to save to my harddrive - but rather than saving each page, I was wondering if there are recommended tools that I can point at the start page and it will then follow each link and download the arrived at pages. Note that all of the links aren't text links but image map links (click on a picture).

Thanks!
Richard.
Reply
Old Nov 20, 2007 | 04:13 PM
  #2  
StiX's Avatar
StiX
Scooby Regular
South Wales Scoobys
iTrader: (4)
 
Joined: Jan 2004
Posts: 2,293
Likes: 0
From: Bridgend, S Wales
Default

Can't you just log into wherever you have hosted it and copy all the files?

Or use an ftp client and do the same?
Reply
Old Nov 20, 2007 | 04:21 PM
  #3  
Scotsman's Avatar
Scotsman
Thread Starter
Scooby Regular
iTrader: (3)
 
Joined: Apr 2001
Posts: 2,168
Likes: 0
From: Stirling
Default

We've paid another company to build the site for us and they are hosting it (we don't have access). We've had some problems with them recently and I wanted to have a local copy of the site prior to having some robust discussions with them. Unfortunately that means I only have access through a browser to the published site.
Reply
Old Nov 20, 2007 | 04:22 PM
  #4  
Scotsman's Avatar
Scotsman
Thread Starter
Scooby Regular
iTrader: (3)
 
Joined: Apr 2001
Posts: 2,168
Likes: 0
From: Stirling
Default

Just found a selection on Google - such as HTTrack Website Copier - Offline Browser and Download entire website or download web pages with SurfOffline - convenient offline browser with easy-to-use interface.
Reply
Old Nov 20, 2007 | 04:24 PM
  #5  
StiX's Avatar
StiX
Scooby Regular
South Wales Scoobys
iTrader: (4)
 
Joined: Jan 2004
Posts: 2,293
Likes: 0
From: Bridgend, S Wales
Default

Originally Posted by Scotsman
We've paid another company to build the site for us and they are hosting it (we don't have access). We've had some problems with them recently and I wanted to have a local copy of the site prior to having some robust discussions with them. Unfortunately that means I only have access through a browser to the published site.
Ah - gotcha! I see you have found some now anyway
Reply
Old Nov 20, 2007 | 04:25 PM
  #6  
RichB's Avatar
RichB
Scooby Regular
 
Joined: Apr 1999
Posts: 3,666
Likes: 0
From: Bore Knee Muff
Default

Ive used HTTrack before and its quite good.
Reply
Old Nov 20, 2007 | 04:32 PM
  #7  
Scotsman's Avatar
Scotsman
Thread Starter
Scooby Regular
iTrader: (3)
 
Joined: Apr 2001
Posts: 2,168
Likes: 0
From: Stirling
Default

Thanks - I'll give that a shot first.
Reply
Old Nov 20, 2007 | 04:58 PM
  #8  
RichB's Avatar
RichB
Scooby Regular
 
Joined: Apr 1999
Posts: 3,666
Likes: 0
From: Bore Knee Muff
Default

be aware that obviously any content management system or database content wont be accessible this way, you will just get a flat version of the site.
Also any URLs that pass anything between themselves with the ? or & may not end up quite as you'd hoped.
If its purely static then you'll be fine.
Reply
Old Nov 20, 2007 | 05:00 PM
  #9  
Scotsman's Avatar
Scotsman
Thread Starter
Scooby Regular
iTrader: (3)
 
Joined: Apr 2001
Posts: 2,168
Likes: 0
From: Stirling
Default

It is a static site (just html files) - but I'm having problems getting any of the website grabbing programmes to follow the image maps links (if that's the right term).
Reply
Old Nov 20, 2007 | 05:12 PM
  #10  
RichB's Avatar
RichB
Scooby Regular
 
Joined: Apr 1999
Posts: 3,666
Likes: 0
From: Bore Knee Muff
Default

if you want to PM me the site, ill take a look
Reply
Old Nov 20, 2007 | 06:27 PM
  #11  
Luminous's Avatar
Luminous
Scooby Regular
iTrader: (3)
 
Joined: Aug 2004
Posts: 15,449
Likes: 0
From: Muppetising life
Default

Webzip is pretty good, not used it for years. I used it routinely when I had a dial up account, would let it sync changes for offline browsing later on.

You can tell it how you want it to follow links, how how deep to go etc. For example you may want it to follow all pages that stay within that site, but not follow any links that go to external sites. You can also specify what types of file you want it to look for...not that I ever in my younger years pointed it at a **** site

No idea what the software is like now, take a look and see if it can help.

Last edited by Luminous; Nov 20, 2007 at 06:30 PM.
Reply
Old Nov 20, 2007 | 07:18 PM
  #12  
Scotsman's Avatar
Scotsman
Thread Starter
Scooby Regular
iTrader: (3)
 
Joined: Apr 2001
Posts: 2,168
Likes: 0
From: Stirling
Default

Originally Posted by Luminous
Webzip is pretty good, not used it for years. I used it routinely when I had a dial up account, would let it sync changes for offline browsing later on.

You can tell it how you want it to follow links, how how deep to go etc. For example you may want it to follow all pages that stay within that site, but not follow any links that go to external sites. You can also specify what types of file you want it to look for...not that I ever in my younger years pointed it at a **** site

No idea what the software is like now, take a look and see if it can help.
Thanks - I'll give that one a try
Reply
Old Nov 20, 2007 | 07:50 PM
  #13  
Scotsman's Avatar
Scotsman
Thread Starter
Scooby Regular
iTrader: (3)
 
Joined: Apr 2001
Posts: 2,168
Likes: 0
From: Stirling
Default

Well, Webzip is superb - worked first time and grabed everything it should off. Thanks again for the recommendation.
Reply
Old Nov 21, 2007 | 10:47 AM
  #14  
Luminous's Avatar
Luminous
Scooby Regular
iTrader: (3)
 
Joined: Aug 2004
Posts: 15,449
Likes: 0
From: Muppetising life
Default

Cool, glad its still as good as it was
Reply
Old Nov 21, 2007 | 11:38 AM
  #15  
PeteBrant's Avatar
PeteBrant
Scooby Regular
15 Year Member
 
Joined: Sep 2006
Posts: 7,576
Likes: 0
From: Worthing..
Default

Can't you set firefox to do something along these lines?
Reply
Old Nov 21, 2007 | 12:14 PM
  #16  
Pumpkin's Avatar
Pumpkin
Scooby Senior
 
Joined: Mar 1999
Posts: 1,835
Likes: 0
From: Norwich
Default

Webreaper works very well for this sort of job.
Reply
Old Nov 21, 2007 | 03:23 PM
  #17  
GazTheHat's Avatar
GazTheHat
Scooby Regular
 
Joined: Aug 2005
Posts: 7,638
Likes: 0
From: 392/361 MY04 STi
Default

Blackwidow also does the job.
Reply
Old Nov 21, 2007 | 07:20 PM
  #18  
Avi's Avatar
Avi
Scooby Regular
 
Joined: Apr 2001
Posts: 5,084
Likes: 0
From: Manchester
Default

Originally Posted by Pumpkin
Webreaper works very well for this sort of job.
I've used webreaper for this too!
Reply
Old Nov 22, 2007 | 10:41 AM
  #19  
Pumpkin's Avatar
Pumpkin
Scooby Senior
 
Joined: Mar 1999
Posts: 1,835
Likes: 0
From: Norwich
Default

I think someone once used Webreaper to archive Scoobynet and brought the system down!
Reply
Related Topics
Thread
Thread Starter
Forum
Replies
Last Post
Little V
Wales
18
Oct 9, 2015 09:45 PM
LostUser
Non Scooby Related
11
Sep 29, 2015 11:00 AM
jobegold@hotmail.co.uk
ScoobyNet General
2
Sep 27, 2015 09:44 PM




All times are GMT +1. The time now is 05:42 PM.