Last weekend I wanted a break from my usual activities, so I decided to write myself some tools to automate a few tasks. One of these is to pull down QIF data from my bank so that I can import it into money management software (I know, I know: I go wild at weekends). I did a little bit on this a while back but I needed to refresh my memory.
aLink = self.driver.find_element_by_id('lstAccLst:0:lkImageRetail1') aLink.click() aLink = self.driver.find_element_by_id('pnlgrpStatement:conS2:lkoverlay') aLink.click()
This is, to be frank, totally amazing. Forget having to wrangle with obscure form data and ensuring that you look like a browser. If you’re not concerned about performance, then simply use a browser itself via Selenium.
I have found that Selenium is not always as robust as Scrapy. If you start multiple instances from the same script, I’ve had some odd failings. That said, I’m also wrapping my Selenium instance in a virtual displa (using pyvirtualdisplay) so that I don’t see the browser, like this:
self.display = Display(visible=0, size=(800, 600)) self.display.start()