I need to get some data from a web site that has a report with a fixed maximum of 20 lines per page. There are about 20k pages.
I need a tool that will download the data into some format - CSV, MySQL, I don't care - and then autmatically follow the link to the next page and repeat.
I could write one, but has anyone used a tool that does this?
Going straight to the database is not an option, I'm afraid. This is the only way to the data.
I need a tool that will download the data into some format - CSV, MySQL, I don't care - and then autmatically follow the link to the next page and repeat.
I could write one, but has anyone used a tool that does this?
Going straight to the database is not an option, I'm afraid. This is the only way to the data.
Comment