[CLUE-Tech] Downloading multiple web pages

Kevin Cullis kevincu at orci.com
Sun Apr 29 10:11:49 MDT 2001


Lynn Danielson wrote:
> 
> Kevin Cullis wrote:
> >
> > I'd like to download a complete web site's info without having to do it
> > individually for each web page. How would I go about this?
> 
> Kevin, check out wget.  I've not used it myself, but I believe it will
> do what you want.

Found this on LDPs site:

> > >FYI, I use wget+tidy+xslt to do this (well, the prototype script that is,
> > >as the final version will be in Narval as described in LG#59), so any
> > 
> > I usually do such scripts in wget+sed+awk (+perl when I need it) - KISS
> 
> I KISS a lot and you wouldn't believe how stupid and simple such a script
> can get using wget+tidy+xslt compared to what it'd be using
> wget+{sed,awk,perl,python} !
> 

I'll have to check these out as well. The combination might be just the
thing I'm looking for.

Kevin



More information about the clue-tech mailing list