编程语言

你在这里

编程语言

2018-06-20 01:03:53 [twisted] CRITICAL: Traceback (most recent call last): File "/usr/lib/python2.7/dist-packages/twisted/internet/defer.py", line 1299, in _inlineCallbacks result = g.send(result) File "/usr/local/lib/python2.7/dist-packages/scrapy/crawler.py", line 98, in crawl six.reraise(*exc_info) File "/usr/local/lib/python2.7/dist-packages/scrapy/crawler.py", line 79, in crawl self.spi
操作系统Ubuntu17,在scrapy中使用selenium时报错,错误代码如下: File "/usr/local/lib/python2.7/dist-packages/selenium-3.0.0b2-py2.7.egg/selenium/webdriver/firefox/webdriver.py", line 65, in __init__ self.service.start() File "/usr/local/lib/python2.7/dist-packages/selenium-3.0.0b2-py2.7.egg/selenium/webdriver/common/service.py", line 71, in start os.path.basename(self.path), self.start_error_message) selenium.co
利用如下代码从多页爬取,但似乎Request没有调用,或其回调函数没有调用?
def parse(self, response):
    #水平爬取
    next_selector = response.xpath('//*[contains(@class, "house-lst-page-box")]//a[last()]/@href')
    for url in next_selector.extract():
    yield Request(urlparse.urljoin(response.url, url))
    #垂直爬取