“失火” Python异步/等待

问题:“失火” Python异步/等待

有时需要执行一些非关键性的异步操作,但我不想等待它完成。在Tornado的协程实现中,您可以通过简单地省略yield关键字来“触发并忘记”一个异步函数。

我一直在尝试找出如何使用Python 3.5中发布的新语法async/ 来“激发并忘记” await。例如,一个简化的代码片段:

async def async_foo():
    print("Do some stuff asynchronously here...")

def bar():
    async_foo()  # fire and forget "async_foo()"

bar()

但是,发生的事情是bar()永远不会执行,而是收到运行时警告:

RuntimeWarning: coroutine 'async_foo' was never awaited
  async_foo()  # fire and forget "async_foo()"

Sometimes there is some non-critical asynchronous operation that needs to happen but I don’t want to wait for it to complete. In Tornado’s coroutine implementation you can “fire & forget” an asynchronous function by simply ommitting the yield key-word.

I’ve been trying to figure out how to “fire & forget” with the new async/await syntax released in Python 3.5. E.g., a simplified code snippet:

async def async_foo():
    print("Do some stuff asynchronously here...")

def bar():
    async_foo()  # fire and forget "async_foo()"

bar()

What happens though is that bar() never executes and instead we get a runtime warning:

RuntimeWarning: coroutine 'async_foo' was never awaited
  async_foo()  # fire and forget "async_foo()"

回答 0

更新:

如果您使用的是Python> = 3.7,请在任何地方替换asyncio.ensure_futureasyncio.create_task最新的,更好的派生task的方法


asyncio。“解雇”的任务

根据python docs的asyncio.Task说法,有可能启动一些协程以“在后台”执行asyncio.ensure_future 函数创建的任务不会阻止执行(因此函数将立即返回!)。这似乎是您要求的一种“解雇”的方法。

import asyncio


async def async_foo():
    print("async_foo started")
    await asyncio.sleep(1)
    print("async_foo done")


async def main():
    asyncio.ensure_future(async_foo())  # fire and forget async_foo()

    # btw, you can also create tasks inside non-async funcs

    print('Do some actions 1')
    await asyncio.sleep(1)
    print('Do some actions 2')
    await asyncio.sleep(1)
    print('Do some actions 3')


if __name__ == '__main__':
    loop = asyncio.get_event_loop()
    loop.run_until_complete(main())

输出:

Do some actions 1
async_foo started
Do some actions 2
async_foo done
Do some actions 3

如果事件循环完成后正在执行任务怎么办?

请注意,asyncio期望任务将在事件循环完成时完成。因此,如果您更改main()为:

async def main():
    asyncio.ensure_future(async_foo())  # fire and forget

    print('Do some actions 1')
    await asyncio.sleep(0.1)
    print('Do some actions 2')

程序完成后,您会收到以下警告:

Task was destroyed but it is pending!
task: <Task pending coro=<async_foo() running at [...]

为防止这种情况,您可以在事件循环完成后等待所有待处理的任务

async def main():
    asyncio.ensure_future(async_foo())  # fire and forget

    print('Do some actions 1')
    await asyncio.sleep(0.1)
    print('Do some actions 2')


if __name__ == '__main__':
    loop = asyncio.get_event_loop()
    loop.run_until_complete(main())

    # Let's also finish all running tasks:
    pending = asyncio.Task.all_tasks()
    loop.run_until_complete(asyncio.gather(*pending))

杀死任务而不是等待任务

有时,您不想等待任务完成(例如,某些任务可能创建为永久运行)。在这种情况下,您可以只取消()而不是等待它们:

import asyncio
from contextlib import suppress


async def echo_forever():
    while True:
        print("echo")
        await asyncio.sleep(1)


async def main():
    asyncio.ensure_future(echo_forever())  # fire and forget

    print('Do some actions 1')
    await asyncio.sleep(1)
    print('Do some actions 2')
    await asyncio.sleep(1)
    print('Do some actions 3')


if __name__ == '__main__':
    loop = asyncio.get_event_loop()
    loop.run_until_complete(main())

    # Let's also cancel all running tasks:
    pending = asyncio.Task.all_tasks()
    for task in pending:
        task.cancel()
        # Now we should await task to execute it's cancellation.
        # Cancelled task raises asyncio.CancelledError that we can suppress:
        with suppress(asyncio.CancelledError):
            loop.run_until_complete(task)

输出:

Do some actions 1
echo
Do some actions 2
echo
Do some actions 3
echo

Upd:

Replace asyncio.ensure_future with asyncio.create_task everywhere if you’re using Python >= 3.7 It’s newer, nicer way to spawn task.


asyncio.Task to “fire and forget”

According to python docs for asyncio.Task it is possible to start some coroutine to execute “in background”. The task created by asyncio.ensure_future function won’t block the execution (therefore the function will return immediately!). This looks like a way to “fire and forget” as you requested.

import asyncio


async def async_foo():
    print("async_foo started")
    await asyncio.sleep(1)
    print("async_foo done")


async def main():
    asyncio.ensure_future(async_foo())  # fire and forget async_foo()

    # btw, you can also create tasks inside non-async funcs

    print('Do some actions 1')
    await asyncio.sleep(1)
    print('Do some actions 2')
    await asyncio.sleep(1)
    print('Do some actions 3')


if __name__ == '__main__':
    loop = asyncio.get_event_loop()
    loop.run_until_complete(main())

Output:

Do some actions 1
async_foo started
Do some actions 2
async_foo done
Do some actions 3

What if tasks are executing after event loop complete?

Note that asyncio expects task would be completed at the moment event loop completed. So if you’ll change main() to:

async def main():
    asyncio.ensure_future(async_foo())  # fire and forget

    print('Do some actions 1')
    await asyncio.sleep(0.1)
    print('Do some actions 2')

You’ll get this warning after the program finished:

Task was destroyed but it is pending!
task: <Task pending coro=<async_foo() running at [...]

To prevent that you can just await all pending tasks after event loop completed:

async def main():
    asyncio.ensure_future(async_foo())  # fire and forget

    print('Do some actions 1')
    await asyncio.sleep(0.1)
    print('Do some actions 2')


if __name__ == '__main__':
    loop = asyncio.get_event_loop()
    loop.run_until_complete(main())

    # Let's also finish all running tasks:
    pending = asyncio.Task.all_tasks()
    loop.run_until_complete(asyncio.gather(*pending))

Kill tasks instead of awaiting them

Sometimes you don’t want to await tasks to be done (for example, some tasks may be created to run forever). In that case, you can just cancel() them instead of awaiting them:

import asyncio
from contextlib import suppress


async def echo_forever():
    while True:
        print("echo")
        await asyncio.sleep(1)


async def main():
    asyncio.ensure_future(echo_forever())  # fire and forget

    print('Do some actions 1')
    await asyncio.sleep(1)
    print('Do some actions 2')
    await asyncio.sleep(1)
    print('Do some actions 3')


if __name__ == '__main__':
    loop = asyncio.get_event_loop()
    loop.run_until_complete(main())

    # Let's also cancel all running tasks:
    pending = asyncio.Task.all_tasks()
    for task in pending:
        task.cancel()
        # Now we should await task to execute it's cancellation.
        # Cancelled task raises asyncio.CancelledError that we can suppress:
        with suppress(asyncio.CancelledError):
            loop.run_until_complete(task)

Output:

Do some actions 1
echo
Do some actions 2
echo
Do some actions 3
echo

回答 1

谢谢谢尔盖的简洁回答。这是相同的装饰版。

import asyncio
import time

def fire_and_forget(f):
    def wrapped(*args, **kwargs):
        return asyncio.get_event_loop().run_in_executor(None, f, *args, *kwargs)

    return wrapped

@fire_and_forget
def foo():
    time.sleep(1)
    print("foo() completed")

print("Hello")
foo()
print("I didn't wait for foo()")

产生

>>> Hello
>>> foo() started
>>> I didn't wait for foo()
>>> foo() completed

注意:检查我的其他答案,使用普通线程也可以做到这一点。

Thank you Sergey for the succint answer. Here is the decorated version of the same.

import asyncio
import time

def fire_and_forget(f):
    def wrapped(*args, **kwargs):
        return asyncio.get_event_loop().run_in_executor(None, f, *args, *kwargs)

    return wrapped

@fire_and_forget
def foo():
    time.sleep(1)
    print("foo() completed")

print("Hello")
foo()
print("I didn't wait for foo()")

Produces

>>> Hello
>>> foo() started
>>> I didn't wait for foo()
>>> foo() completed

Note: Check my other answer which does the same using plain threads.


回答 2

这不是完全异步执行,但也许run_in_executor()适合您。

def fire_and_forget(task, *args, **kwargs):
    loop = asyncio.get_event_loop()
    if callable(task):
        return loop.run_in_executor(None, task, *args, **kwargs)
    else:    
        raise TypeError('Task must be a callable')

def foo():
    #asynchronous stuff here


fire_and_forget(foo)

This is not entirely asynchronous execution, but maybe run_in_executor() is suitable for you.

def fire_and_forget(task, *args, **kwargs):
    loop = asyncio.get_event_loop()
    if callable(task):
        return loop.run_in_executor(None, task, *args, **kwargs)
    else:    
        raise TypeError('Task must be a callable')

def foo():
    #asynchronous stuff here


fire_and_forget(foo)

回答 3

由于某种原因,如果您无法使用,asyncio那么这里是使用普通线程的实现。检查我的其他答案和谢尔盖的答案。

import threading

def fire_and_forget(f):
    def wrapped():
        threading.Thread(target=f).start()

    return wrapped

@fire_and_forget
def foo():
    time.sleep(1)
    print("foo() completed")

print("Hello")
foo()
print("I didn't wait for foo()")

For some reason if you are unable to use asyncio then here is the implementation using plain threads. Check my other answers and Sergey’s answer too.

import threading

def fire_and_forget(f):
    def wrapped():
        threading.Thread(target=f).start()

    return wrapped

@fire_and_forget
def foo():
    time.sleep(1)
    print("foo() completed")

print("Hello")
foo()
print("I didn't wait for foo()")