Download ASP Site

[OwlHoot]

Does anyone know how to download an ASP Website.

I have been using WinHTTRack but it can only go so far. The website is written using ASP and I wish to follow all of the links and get downloaded locally.

Anyone have an ideas?

One word: wget

For example, if you wanted to snaffle a local copy of R F Streater's physics site:

wget -erobots=off --mirror -p -w 2 --convert-links -P C:\streater http://www.mth.kcl.ac.uk/~streater/

In this example the options are:
-erobots=off => ignore robots.txt file settings (naughty!)
-m[irror] => mirror site on local drive (change all site links to "file://" links)
-p => Get all associated components from site like images and CSS files
-w # => pause # seconds between each download (to avoid snowing site under with requests)
-r => recursive (grab sub-links on same site recursively)
-u mozilla => present wget as mozilla browser

I think these options should do what you want, without trying to mirror the whole Internet for example. But it is slightly confusing, as different versions of wget use different combinations to achieve the same end.

But needless to say, it can only get a "connected" set of web pages; so if a site happens to comprise more than one link-disjoint set of pages or a partially ordered set with more than one "maximal" page (for example, downward links that don't each recursively span every page on the site) you'll need to do a wget for for a set of pages that does span the site, if that makes sense.

edit: Sorry, I see several people have already mentioned wget. But as Bunk pointed out, you can get a Windows version.

There's also cURL ; but I haven't used that recently.

[themistry]

You want teleport pro.

Does exactly what you want

TM

[Bunk]

You can download wget for Windows.

[d000hg]

Came across that but dont have Linux setup

Can you install cygwin to give you access to Linux tools on your Windows box?

[Zippy]

In which case can you get him to dump the file structure onto a cd/dvd for you?

Or make it accessable via ftp so you can grab the whole thing that way?

The problem is MF then has to have the server side stuff to use it.

[DaveB]

Came across that but dont have Linux setup.

Surely there is some straight forward software somewhere can recommend.

Basically Im putting together an online catalogue of a supplier of mine for upload into my own website(Im allowed to do this). But Im trying to automate the process and need the site locally to do it.

In which case can you get him to dump the file structure onto a cd/dvd for you?

Or make it accessable via ftp so you can grab the whole thing that way?

[Zippy]

Are there any Firefox tools/plugins that might help? Might be worth a look?

Edit: on the wrong machine at the mo so can't check my own toolkits.

[MarillionFan]

Came across that but dont have Linux setup.

Surely there is some straight forward software somewhere can recommend.

Basically Im putting together an online catalogue of a supplier of mine for upload into my own website(Im allowed to do this). But Im trying to automate the process and need the site locally to do it.

[DaveB]

Does anyone know how to download an ASP Website.

I have been using WinHTTRack but it can only go so far. The website is written using ASP and I wish to follow all of the links and get downloaded locally.

Anyone have an ideas?

Disclaimer : I am not advocating scraping sites you don't own or otherwise have permission to do this to.

If you have access to a Linux box or a nix type command line shell you can get it with wget. There is some tidying up to do afterwards but it should at least get all the pages for you.

Otherwise it's a pain in the arse to scrape asp sites because of the way they tend to be structured and the way the links between pages are constructed.

[MarillionFan]

I want to download the site locally so I can navigate it.

Ideally I would like to go through all the links and download it locally (converting to HTML even).

[Zippy]

Nothing except hacking the site will get you the source code

Was possible with older, unpatched versions of IIS

[Not So Wise]

Nothing except hacking the site will get you the source code

[Zippy]

Does anyone know how to download an ASP Website.

I have been using WinHTTRack but it can only go so far. The website is written using ASP and I wish to follow all of the links and get downloaded locally.

Anyone have an ideas?

Do you want to get the ASP source code?

[MarillionFan]

Does anyone know how to download an ASP Website.

I have been using WinHTTRack but it can only go so far. The website is written using ASP and I wish to follow all of the links and get downloaded locally.

Anyone have an ideas?