python – Aiohttp,Asyncio:RuntimeError:事件循环已关闭

我有两个脚本,scraper.py和db_control.py.在scraper.py我有这样的事情: …def scrap(category, field, pages, search, use_proxy, proxy_file): … loop = asyncio.get_event_loop() to_do = [ get_pages(url, para

我有两个脚本,scraper.py和db_control.py.在scraper.py我有这样的事情:

...
def scrap(category,field,pages,search,use_proxy,proxy_file):
    ...
    loop = asyncio.get_event_loop()

    to_do = [ get_pages(url,params,conngen) for url in urls ]
    wait_coro = asyncio.wait(to_do)
    res,_ = loop.run_until_complete(wait_coro)
    ...
    loop.close()

    return [ x.result() for x in res ]

...

在db_control.py中:

from scraper import scrap
...
while new < 15:
    data = scrap(category,proxy_file)
    ...
...

从理论上讲,刮板应该在未知时间开始,直到获得足够的数据.但是当新的不是imidiatelly> 15然后发生此错误:

File "/usr/lib/python3.4/asyncio/base_events.py",line 293,in run_until_complete
self._check_closed()
  File "/usr/lib/python3.4/asyncio/base_events.py",line 265,in _check_closed
raise RuntimeError('Event loop is closed')
RuntimeError: Event loop is closed

但是如果我只运行一次scrap(),脚本就可以了.所以我想重新创建loop = asyncio.get_event_loop()有一些问题,我尝试了this但没有任何改变.我怎么解决这个问题?当然这些只是我的代码的片段,如果您认为问题可以在其他地方,完整代码可用here.

解决方法

方法
run_until_complete,run_forever,run_in_executor,create_task,call_at明确检查

如果它已关闭,则循环并抛出异常.

从文档引用 – BaseEvenLoop.close

This is idempotent and irreversible

除非你有一些(好的)理由,否则你可能只是省略了关闭线:

def scrap(category,proxy_file):
    #...
    loop = asyncio.get_event_loop()

    to_do = [ get_pages(url,_ = loop.run_until_complete(wait_coro)
    #...
    # loop.close()
    return [ x.result() for x in res ]

如果您希望每次都有一个全新的循环,您必须手动创建它并设置为默认值:

def scrap(category,proxy_file):
    #...
    loop = asyncio.new_event_loop()
    asyncio.set_event_loop(loop)    
    to_do = [ get_pages(url,_ = loop.run_until_complete(wait_coro)
    #...
    return [ x.result() for x in res ]

作者: dawei

【声明】:永州站长网内容转载自互联网,其相关言论仅代表作者个人观点绝非权威,不代表本站立场。如您发现内容存在版权问题,请提交相关链接至邮箱:bqsm@foxmail.com,我们将及时予以处理。

为您推荐

联系我们

联系我们

0577-28828765

在线咨询: QQ交谈

邮箱: xwei067@foxmail.com

工作时间:周一至周五,9:00-17:30,节假日休息

返回顶部