Scrapy 多个爬虫顺序启动和间歇式启动

hqs19930 · 发表于 2020-11-5 15:56:29

6666

night_spider · 发表于 2020-11-23 18:09:31

night_spider · 发表于 2020-11-23 18:09:45

pkz · 发表于 2021-3-25 15:04:59

学习下知识,大佬牛逼

lol_olo · 发表于 2021-3-26 10:33:34

看看？

yikuhasaki · 发表于 2021-3-28 21:05:44

不愧是你

jony · 发表于 2021-4-6 17:02:24

chen · 发表于 2021-4-7 11:48:33

6666

qwertyuiop1822 · 发表于 2021-4-8 18:12:32

from twisted.internet import reactor, defer
from scrapy.crawler import CrawlerRunner
from scrapy.utils.log import configure_logging
class MySpider1(scrapy.Spider):
    # Your first spider definition
    ...
class MySpider2(scrapy.Spider):
    # Your second spider definition
    ...
configure_logging()
runner = CrawlerRunner()
@defer.inlineCallbacks
def crawl():
    yield runner.crawl(MySpider1)
    yield runner.crawl(MySpider2)
    reactor.stop()
crawl()
reactor.run()
https://blog.csdn.net/Qwertyuiop2016/article/details/105629980

triangle · 发表于 2021-4-26 10:58:45

Scrapy 多个爬虫 顺序启动 和 间歇式启动

鸢公子 LV

Scrapy 多个爬虫顺序启动和间歇式启动