[clue-talk] Mirroring a web site?

[Sean LeBlanc]

On 05-17 09:29, Roy Johnston wrote:

> If you want a mirror of a specific page: 
> wget example.com/sample.html
> 
> If you want to get all of a web site - just one level deep (the direct links from index) use: 
> wget example.com
> 
> If you actually do want to grab a mirror of the whole site - use: 
> wget --mirror example.com
> 
> be aware - that you can get more than you expect
> also, it is considered rude to grab a full mirror during business hours - unless you know that they have plenty of bandwidth.

Thanks for the help. I've been re-visiting this again, but haven't been
having much luck with wget and getting it to grab all the site -
specifically, the parts that need logging in via http form auth.

I've tried using the post-data options as described in wget's man page, but
still haven't been having any luck. 

Unfortunately, SpiderZilla seems to come sooo close to making this an easy
thing, but I cannot seem to find form auth options in there either.

-- 
Sean LeBlanc:seanleblanc at comcast.net  
http://sean-leblanc.blogspot.com/
I am an agnostic; I do not pretend to know what many ignorant men are sure of. 
-Clarence Darrow 
(contributed by Chris Johnston)