- Joined
- Feb 19, 2009
- Messages
- 603
- Likes received
- 54
- Thanks received
- 9
I need to get some data from a web site that has a report with a fixed maximum of 20 lines per page. There are about 20k pages.
I need a tool that will download the data into some format - CSV, MySQL, I don't care - and then autmatically follow the link to the next page and repeat.
I could write one, but has anyone used a tool that does this?
Going straight to the database is not an option, I'm afraid. This is the only way to the data.
I need a tool that will download the data into some format - CSV, MySQL, I don't care - and then autmatically follow the link to the next page and repeat.
I could write one, but has anyone used a tool that does this?
Going straight to the database is not an option, I'm afraid. This is the only way to the data.