Looks like I just worked out the last error, having it ignore robots.txt. It would be nice if this was all stable enough to do a full test run, but everything seems to be in order. Knowing my experience without testing (absolutely every. Possible. Outcome of) things I do relating to computing, there'll probably be issues, but hopefully not. I'll let you know tomorrow (this'll take a while) how it goes.

EDIT: Intermediate tests seem to affirm that this is behaving as expected with no issues. I'd rather it not save images, but it may be doing that. It's worth it working, though.