标签归档:reload

更改源代码时自动加载gunicorn

问题:更改源代码时自动加载gunicorn

最后,我将开发环境从runserver迁移到gunicorn / nginx。

将runserver的自动重载功能复制到gunicorn会很方便,因此当源更改时,服务器会自动重新启动。否则,我必须使用手动重新启动服务器kill -HUP

有什么方法可以避免手动重启?

Finally I migrated my development env from runserver to gunicorn/nginx.

It’d be convenient to replicate the autoreload feature of runserver to gunicorn, so the server automatically restarts when source changes. Otherwise I have to restart the server manually with kill -HUP.

Any way to avoid the manual restart?


回答 0

尽管这是个老问题,但仅出于一致性考虑-因为19.0版本的gunicorn可以--reload选择。因此,不再需要第三方工具。

While this is old question you need to know that ever since version 19.0 gunicorn has had the --reload option. So now no third party tools are needed.


回答 1

一种选择是使用–max-requests通过添加--max-requests 1到启动选项来将每个生成的进程限制为仅服务一个请求。每个新产生的进程都应该看到您的代码更改,并且在开发环境中,每个请求的额外启动时间可以忽略不计。

One option would be to use the –max-requests to limit each spawned process to serving only one request by adding --max-requests 1 to the startup options. Every newly spawned process should see your code changes and in a development environment the extra startup time per request should be negligible.


回答 2

Bryan Helmig提出了这个建议,我对其进行了修改,以使其直接使用run_gunicorn而不是gunicorn直接启动,以便可以将这3个命令剪切并粘贴到django项目根文件夹中的shell中(激活了virtualenv):

pip install watchdog -U
watchmedo shell-command --patterns="*.py;*.html;*.css;*.js" --recursive --command='echo "${watch_src_path}" && kill -HUP `cat gunicorn.pid`' . &
python manage.py run_gunicorn 127.0.0.1:80 --pid=gunicorn.pid

Bryan Helmig came up with this and I modified it to use run_gunicorn instead of launching gunicorn directly, to make it possible to just cut and paste these 3 commands into a shell in your django project root folder (with your virtualenv activated):

pip install watchdog -U
watchmedo shell-command --patterns="*.py;*.html;*.css;*.js" --recursive --command='echo "${watch_src_path}" && kill -HUP `cat gunicorn.pid`' . &
python manage.py run_gunicorn 127.0.0.1:80 --pid=gunicorn.pid

回答 3

我使用git push部署到生产环境,并设置git挂钩来运行脚本。这种方法的优点是您还可以同时进行迁移和软件包安装。 https://mikeeverhart.net/2013/01/using-git-to-deploy-code/

mkdir -p /home/git/project_name.git
cd /home/git/project_name.git
git init --bare

然后创建一个脚本/home/git/project_name.git/hooks/post-receive

#!/bin/bash
GIT_WORK_TREE=/path/to/project git checkout -f
source /path/to/virtualenv/activate
pip install -r /path/to/project/requirements.txt
python /path/to/project/manage.py migrate
sudo supervisorctl restart project_name

确保为chmod u+x post-receive,并将用户添加到sudoers。允许它sudo supervisorctl不带密码运行。 https://www.cyberciti.biz/faq/linux-unix-running-sudo-command-without-a-password/

在本地/开发服务器上,我进行了设置,git remote从而可以推送到生产服务器

git remote add production ssh://user_name@production-server/home/git/project_name.git

# initial push
git push production +master:refs/heads/master

# subsequent push
git push production master

另外,在脚本运行时,您将看到所有提示。因此,您将看到迁移/软件包安装/主管重启是否存在任何问题。

I use git push to deploy to production and set up git hooks to run a script. The advantage of this approach is you can also do your migration and package installation at the same time. https://mikeeverhart.net/2013/01/using-git-to-deploy-code/

mkdir -p /home/git/project_name.git
cd /home/git/project_name.git
git init --bare

Then create a script /home/git/project_name.git/hooks/post-receive.

#!/bin/bash
GIT_WORK_TREE=/path/to/project git checkout -f
source /path/to/virtualenv/activate
pip install -r /path/to/project/requirements.txt
python /path/to/project/manage.py migrate
sudo supervisorctl restart project_name

Make sure to chmod u+x post-receive, and add user to sudoers. Allow it to run sudo supervisorctl without password. https://www.cyberciti.biz/faq/linux-unix-running-sudo-command-without-a-password/

From my local / development server, I set up git remote that allows me to push to the production server

git remote add production ssh://user_name@production-server/home/git/project_name.git

# initial push
git push production +master:refs/heads/master

# subsequent push
git push production master

As a bonus, you will get to see all the prompts as the script is running. So you will see if there is any issue with the migration/package installation/supervisor restart.


如何卸载(重新加载)模块?

问题:如何卸载(重新加载)模块?

我有一台运行时间较长的Python服务器,并且希望能够在不重新启动服务器的情况下升级服务。最好的方法是什么?

if foo.py has changed:
    unimport foo  <-- How do I do this?
    import foo
    myfoo = foo.Foo()

I have a long-running Python server and would like to be able to upgrade a service without restarting the server. What’s the best way do do this?

if foo.py has changed:
    unimport foo  <-- How do I do this?
    import foo
    myfoo = foo.Foo()

回答 0

您可以使用reload内置函数(仅适用于Python 3.4+)重新导入已导入的模块:

from importlib import reload  
import foo

while True:
    # Do some things.
    if is_changed(foo):
        foo = reload(foo)

在Python 3中,reload已移至imp模块。在3.4中,imp不推荐使用importlib,而reload在中添加了。当定位到3或更高版本时,在调用reload或导入它时参考相应的模块。

我认为这就是您想要的。诸如Django开发服务器之类的Web服务器都使用此服务器,这样您就可以查看代码更改的效果,而无需重新启动服务器进程本身。

引用文档:

重新编译Python模块的代码并重新执行模块级代码,从而定义了一组新对象,这些对象绑定到模块字典中的名称。扩展模块的init函数不会被第二次调用。与Python中的所有其他对象一样,旧对象仅在其引用计数降至零后才被回收。模块命名空间中的名称将更新为指向任何新的或更改的对象。对旧对象的其他引用(例如模块外部的名称)不会反弹以引用新对象,并且如果需要的话,必须在出现它们的每个命名空间中进行更新。

正如您在问题中指出的那样,Foo如果Foo类驻留在foo模块中,则必须重构对象。

You can reload a module when it has already been imported by using the reload builtin function (Python 3.4+ only):

from importlib import reload  
import foo

while True:
    # Do some things.
    if is_changed(foo):
        foo = reload(foo)

In Python 3, reload was moved to the imp module. In 3.4, imp was deprecated in favor of importlib, and reload was added to the latter. When targeting 3 or later, either reference the appropriate module when calling reload or import it.

I think that this is what you want. Web servers like Django’s development server use this so that you can see the effects of your code changes without restarting the server process itself.

To quote from the docs:

Python modules’ code is recompiled and the module-level code reexecuted, defining a new set of objects which are bound to names in the module’s dictionary. The init function of extension modules is not called a second time. As with all other objects in Python the old objects are only reclaimed after their reference counts drop to zero. The names in the module namespace are updated to point to any new or changed objects. Other references to the old objects (such as names external to the module) are not rebound to refer to the new objects and must be updated in each namespace where they occur if that is desired.

As you noted in your question, you’ll have to reconstruct Foo objects if the Foo class resides in the foo module.


回答 1

在Python 3.0–3.3中,您将使用: imp.reload(module)

BDFL已经回答了这个问题。

但是,imp在3.4中已弃用,importlib改为(感谢@Stefan!)。

因此,importlib.reload(module)尽管我不确定,但您现在应该使用。

In Python 3.0–3.3 you would use: imp.reload(module)

The BDFL has answered this question.

However, imp was deprecated in 3.4, in favour of importlib (thanks @Stefan!).

I think, therefore, you’d now use importlib.reload(module), although I’m not sure.


回答 2

如果模块不是纯Python,则删除模块可能会特别困难。

以下是一些信息:我如何真正删除导入的模块?

您可以使用sys.getrefcount()来查找实际的引用数。

>>> import sys, empty, os
>>> sys.getrefcount(sys)
9
>>> sys.getrefcount(os)
6
>>> sys.getrefcount(empty)
3

大于3的数字表示很难摆脱该模块。本地的“空”(不包含任何内容)模块应在之后收集垃圾

>>> del sys.modules["empty"]
>>> del empty

作为第三个引用是getrefcount()函数的构件。

It can be especially difficult to delete a module if it is not pure Python.

Here is some information from: How do I really delete an imported module?

You can use sys.getrefcount() to find out the actual number of references.

>>> import sys, empty, os
>>> sys.getrefcount(sys)
9
>>> sys.getrefcount(os)
6
>>> sys.getrefcount(empty)
3

Numbers greater than 3 indicate that it will be hard to get rid of the module. The homegrown “empty” (containing nothing) module should be garbage collected after

>>> del sys.modules["empty"]
>>> del empty

as the third reference is an artifact of the getrefcount() function.


回答 3

reload(module),但前提是它是完全独立的。如果还有其他引用该模块(或属于该模块的任何对象)的引用,则您将得到细微而奇怪的错误,这些错误是由于旧代码的停留时间超出您的预期而导致的,并且isinstance无法在不同版本的相同的代码。

如果您具有单向依赖关系,则还必须重新加载所有依赖于重新加载的模块的模块,以摆脱对旧代码的所有引用。然后递归依赖于重新加载的模块重新加载模块。

如果您有循环依赖关系(例如在处理重新加载程序包时非常常见),则必须一次性卸载组中的所有模块。您无法执行此操作,reload()因为它将在刷新依赖关系之前重新导入每个模块,从而允许旧引用爬入新模块。

在这种情况下,唯一的方法是hack sys.modules,这是不受支持的。您必须仔细检查并删除sys.modules要在下次导入时重新加载的每个条目,还必须删除其值None用于处理实现问题的条目,以缓存失败的相对导入。它不是很好,但是只要您有一套完全独立的依赖项,并且不会将引用保留在其代码库之外,那么它就是可行的。

最好重新启动服务器。:-)

reload(module), but only if it’s completely stand-alone. If anything else has a reference to the module (or any object belonging to the module), then you’ll get subtle and curious errors caused by the old code hanging around longer than you expected, and things like isinstance not working across different versions of the same code.

If you have one-way dependencies, you must also reload all modules that depend on the the reloaded module to get rid of all the references to the old code. And then reload modules that depend on the reloaded modules, recursively.

If you have circular dependencies, which is very common for example when you are dealing with reloading a package, you must unload all the modules in the group in one go. You can’t do this with reload() because it will re-import each module before its dependencies have been refreshed, allowing old references to creep into new modules.

The only way to do it in this case is to hack sys.modules, which is kind of unsupported. You’d have to go through and delete each sys.modules entry you wanted to be reloaded on next import, and also delete entries whose values are None to deal with an implementation issue to do with caching failed relative imports. It’s not terribly nice but as long as you have a fully self-contained set of dependencies that doesn’t leave references outside its codebase, it’s workable.

It’s probably best to restart the server. :-)


回答 4

if 'myModule' in sys.modules:  
    del sys.modules["myModule"]
if 'myModule' in sys.modules:  
    del sys.modules["myModule"]

回答 5

对于Python 2,请使用内置函数reload()

reload(module)

对于Python 2和3.2–3.3,请使用从模块imp重新加载

import imp
imp.reload(module)

但是从3.4版开始imp 不推荐使用importlib,所以请使用:

import importlib
importlib.reload(module)

要么

from importlib import reload
reload(module)

For Python 2 use built-in function reload():

reload(module)

For Python 2 and 3.2–3.3 use reload from module imp:

import imp
imp.reload(module)

But imp is deprecated since version 3.4 in favor of importlib, so use:

import importlib
importlib.reload(module)

or

from importlib import reload
reload(module)

回答 6

以下代码允许您与Python 2/3兼容:

try:
    reload
except NameError:
    # Python 3
    from imp import reload

您可以reload()在两个版本中都使用它,这使事情变得更简单。

The following code allows you Python 2/3 compatibility:

try:
    reload
except NameError:
    # Python 3
    from imp import reload

The you can use it as reload() in both versions which makes things simpler.


回答 7

接受的答案不处理from X import Y的情况。这段代码可以处理它以及标准的导入情况:

def importOrReload(module_name, *names):
    import sys

    if module_name in sys.modules:
        reload(sys.modules[module_name])
    else:
        __import__(module_name, fromlist=names)

    for name in names:
        globals()[name] = getattr(sys.modules[module_name], name)

# use instead of: from dfly_parser import parseMessages
importOrReload("dfly_parser", "parseMessages")

在重载的情况下,我们将顶级名称重新分配给新重载的模块中存储的值,从而更新它们。

The accepted answer doesn’t handle the from X import Y case. This code handles it and the standard import case as well:

def importOrReload(module_name, *names):
    import sys

    if module_name in sys.modules:
        reload(sys.modules[module_name])
    else:
        __import__(module_name, fromlist=names)

    for name in names:
        globals()[name] = getattr(sys.modules[module_name], name)

# use instead of: from dfly_parser import parseMessages
importOrReload("dfly_parser", "parseMessages")

In the reloading case, we reassign the top level names to the values stored in the newly reloaded module, which updates them.


回答 8

这是重新加载模块的现代方法:

from importlib import reload

如果要支持3.5之前的Python版本,请尝试以下操作:

from sys import version_info
if version_info[0] < 3:
    pass # Python 2 has built in reload
elif version_info[0] == 3 and version_info[1] <= 4:
    from imp import reload # Python 3.0 - 3.4 
else:
    from importlib import reload # Python 3.5+

要使用它,请运行reload(MODULE),并替换MODULE为要重新加载的模块。

例如,reload(math)将重新加载math模块。

This is the modern way of reloading a module:

from importlib import reload

If you want to support versions of Python older than 3.5, try this:

from sys import version_info
if version_info[0] < 3:
    pass # Python 2 has built in reload
elif version_info[0] == 3 and version_info[1] <= 4:
    from imp import reload # Python 3.0 - 3.4 
else:
    from importlib import reload # Python 3.5+

To use it, run reload(MODULE), replacing MODULE with the module you want to reload.

For example, reload(math) will reload the math module.


回答 9

如果您不在服务器中,但是正在开发并且需要经常重新加载模块,那么这里是个不错的提示。

首先,请确保您使用的是Jupyter Notebook项目中出色的IPython shell。安装Jupyter后,你可以启动它ipython,或者jupyter console,甚至更好,jupyter qtconsole,这将为您提供一个漂亮的彩色控制台,并在任何OS中均具有代码完成功能。

现在在您的外壳中,键入:

%load_ext autoreload
%autoreload 2

现在,每次您运行脚本时,模块都会重新加载。

除了2,自动重载魔术还有其他选择

%autoreload
Reload all modules (except those excluded by %aimport) automatically now.

%autoreload 0
Disable automatic reloading.

%autoreload 1
Reload all modules imported with %aimport every time before executing the Python code typed.

%autoreload 2
Reload all modules (except those excluded by %aimport) every time before
executing the Python code typed.

If you are not in a server, but developing and need to frequently reload a module, here’s a nice tip.

First, make sure you are using the excellent IPython shell, from the Jupyter Notebook project. After installing Jupyter, you can start it with ipython, or jupyter console, or even better, jupyter qtconsole, which will give you a nice colorized console with code completion in any OS.

Now in your shell, type:

%load_ext autoreload
%autoreload 2

Now, every time you run your script, your modules will be reloaded.

Beyond the 2, there are other options of the autoreload magic:

%autoreload
Reload all modules (except those excluded by %aimport) automatically now.

%autoreload 0
Disable automatic reloading.

%autoreload 1
Reload all modules imported with %aimport every time before executing the Python code typed.

%autoreload 2
Reload all modules (except those excluded by %aimport) every time before
executing the Python code typed.

回答 10

对于那些想要卸载所有模块的人(在Emacs下的Python解释器中运行时):

   for mod in sys.modules.values():
      reload(mod)

有关更多信息,请参见重新加载Python模块

For those like me who want to unload all modules (when running in the Python interpreter under Emacs):

   for mod in sys.modules.values():
      reload(mod)

More information is in Reloading Python modules.


回答 11

追求特质有一个可以很好地完成此任务的模块。https://traits.readthedocs.org/zh/4.3.0/_modules/traits/util/refresh.html

它将重新加载已更改的所有模块,并更新正在使用该模块的其他模块和实例对象。大多数情况下它不起作用__very_private__方法使用,并且可能会阻塞类继承,但是它为我节省了编写PyQt guis或在Maya或Nuke等程序中运行的东西时不必重新启动主机应用程序的疯狂时间。它可能在20%到30%的时间内无效,但是仍然非常有用。

Enthought的软件包不会在文件更改时立即重新加载文件-您必须明确地调用它-但是如果您真的需要它,那么实现起来应该不那么困难

Enthought Traits has a module that works fairly well for this. https://traits.readthedocs.org/en/4.3.0/_modules/traits/util/refresh.html

It will reload any module that has been changed, and update other modules and instanced objects that are using it. It does not work most of the time with __very_private__ methods, and can choke on class inheritance, but it saves me crazy amounts of time from having to restart the host application when writing PyQt guis, or stuff that runs inside programs such as Maya or Nuke. It doesn’t work maybe 20-30 % of the time, but it’s still incredibly helpful.

Enthought’s package doesn’t reload files the moment they change – you have to call it explicitely – but that shouldn’t be all that hard to implement if you really need it


回答 12

那些正在使用python 3并从importlib重新加载的人。

如果您遇到问题,例如似乎模块无法重新加载…那是因为它需要一些时间来重新编译pyc(最多60秒)。我写此提示只是想知道您是否遇到过此类问题。

Those who are using python 3 and reload from importlib.

If you have problems like it seems that module doesn’t reload… That is because it needs some time to recompile pyc (up to 60 sec).I writing this hint just that you know if you have experienced this kind of problem.


回答 13

2018-02-01

  1. foo必须提前成功导入模块。
  2. from importlib import reloadreload(foo)

31.5。importlib —导入的实现— Python 3.6.4文档

2018-02-01

  1. module foo must be imported successfully in advance.
  2. from importlib import reload, reload(foo)

31.5. importlib — The implementation of import — Python 3.6.4 documentation


回答 14

其他选择。看到Python默认值importlib.reload将只是重新导入作为参数传递的库。它不会重新加载您的lib导入的库。如果您更改了很多文件并且要导入的包有些复杂,则必须进行一次深度重载

如果您安装了IPythonJupyter,则可以使用一个函数来深度重新加载所有库:

from IPython.lib.deepreload import reload as dreload
dreload(foo)

如果您没有Jupyter,请在外壳程序中使用以下命令将其安装:

pip3 install jupyter

Other option. See that Python default importlib.reload will just reimport the library passed as an argument. It won’t reload the libraries that your lib import. If you changed a lot of files and have a somewhat complex package to import, you must do a deep reload.

If you have IPython or Jupyter installed, you can use a function to deep reload all libs:

from IPython.lib.deepreload import reload as dreload
dreload(foo)

If you don’t have Jupyter, install it with this command in your shell:

pip3 install jupyter

回答 15

编辑(答案V2)

之前的解决方案仅适用于获取重置信息,但是它不会更改所有引用(超出reload但少于要求)。为了实际设置所有引用,我必须进入垃圾收集器,并在那里重写引用。现在它就像一种魅力!

请注意,这不会如果GC已关闭,或者重新加载了不受GC监视的数据,则。如果您不想弄乱GC,那么原始答案可能就足够了。

新代码:

import importlib
import inspect
import gc
from weakref import ref


def reset_module(module, inner_modules_also=True):
    """
    This function is a stronger form of importlib's `reload` function. What it does, is that aside from reloading a
    module, it goes to the old instance of the module, and sets all the (not read-only) attributes, functions and classes
    to be the reloaded-module's
    :param module: The module to reload (module reference, not the name)
    :param inner_modules_also: Whether to treat ths module as a package as well, and reload all the modules within it.
    """

    # For the case when the module is actually a package
    if inner_modules_also:
        submods = {submod for _, submod in inspect.getmembers(module)
                   if (type(submod).__name__ == 'module') and (submod.__package__.startswith(module.__name__))}
        for submod in submods:
            reset_module(submod, True)

    # First, log all the references before reloading (because some references may be changed by the reload operation).
    module_tree = _get_tree_references_to_reset_recursively(module, module.__name__)

    new_module = importlib.reload(module)
    _reset_item_recursively(module, module_tree, new_module)


def _update_referrers(item, new_item):
    refs = gc.get_referrers(item)

    weak_ref_item = ref(item)
    for coll in refs:
        if type(coll) == dict:
            enumerator = coll.keys()
        elif type(coll) == list:
            enumerator = range(len(coll))
        else:
            continue

        for key in enumerator:

            if weak_ref_item() is None:
                # No refs are left in the GC
                return

            if coll[key] is weak_ref_item():
                coll[key] = new_item

def _get_tree_references_to_reset_recursively(item, module_name, grayed_out_item_ids = None):
    if grayed_out_item_ids is None:
        grayed_out_item_ids = set()

    item_tree = dict()
    attr_names = set(dir(item)) - _readonly_attrs
    for sub_item_name in attr_names:

        sub_item = getattr(item, sub_item_name)
        item_tree[sub_item_name] = [sub_item, None]

        try:
            # Will work for classes and functions defined in that module.
            mod_name = sub_item.__module__
        except AttributeError:
            mod_name = None

        # If this item was defined within this module, deep-reset
        if (mod_name is None) or (mod_name != module_name) or (id(sub_item) in grayed_out_item_ids) \
                or isinstance(sub_item, EnumMeta):
            continue

        grayed_out_item_ids.add(id(sub_item))
        item_tree[sub_item_name][1] = \
            _get_tree_references_to_reset_recursively(sub_item, module_name, grayed_out_item_ids)

    return item_tree


def _reset_item_recursively(item, item_subtree, new_item):

    # Set children first so we don't lose the current references.
    if item_subtree is not None:
        for sub_item_name, (sub_item, sub_item_tree) in item_subtree.items():

            try:
                new_sub_item = getattr(new_item, sub_item_name)
            except AttributeError:
                # The item doesn't exist in the reloaded module. Ignore.
                continue

            try:
                # Set the item
                _reset_item_recursively(sub_item, sub_item_tree, new_sub_item)
            except Exception as ex:
                pass

    _update_referrers(item, new_item)

原始答案

就像@bobince的答案中所写,如果另一个模块中已经存在对该模块的引用(特别是如果它是使用as诸如import numpy as np),则该实例将不会被覆盖。

在应用要求配置模块处于“干净状态”状态的测试时,这对我来说是相当麻烦的,因此我编写了一个名为的函数,该函数reset_module使用importlibreload函数并递归覆盖所有声明的模块的属性。已通过Python 3.6版进行了测试。

import importlib
import inspect
from enum import EnumMeta

_readonly_attrs = {'__annotations__', '__call__', '__class__', '__closure__', '__code__', '__defaults__', '__delattr__',
               '__dict__', '__dir__', '__doc__', '__eq__', '__format__', '__func__', '__ge__', '__get__',
               '__getattribute__', '__globals__', '__gt__', '__hash__', '__init__', '__init_subclass__',
               '__kwdefaults__', '__le__', '__lt__', '__module__', '__name__', '__ne__', '__new__', '__qualname__',
               '__reduce__', '__reduce_ex__', '__repr__', '__self__', '__setattr__', '__sizeof__', '__str__',
               '__subclasshook__', '__weakref__', '__members__', '__mro__', '__itemsize__', '__isabstractmethod__',
               '__basicsize__', '__base__'}


def reset_module(module, inner_modules_also=True):
    """
    This function is a stronger form of importlib's `reload` function. What it does, is that aside from reloading a
    module, it goes to the old instance of the module, and sets all the (not read-only) attributes, functions and classes
    to be the reloaded-module's
    :param module: The module to reload (module reference, not the name)
    :param inner_modules_also: Whether to treat ths module as a package as well, and reload all the modules within it.
    """

    new_module = importlib.reload(module)

    reset_items = set()

    # For the case when the module is actually a package
    if inner_modules_also:
        submods = {submod for _, submod in inspect.getmembers(module)
                   if (type(submod).__name__ == 'module') and (submod.__package__.startswith(module.__name__))}
        for submod in submods:
            reset_module(submod, True)

    _reset_item_recursively(module, new_module, module.__name__, reset_items)


def _reset_item_recursively(item, new_item, module_name, reset_items=None):
    if reset_items is None:
        reset_items = set()

    attr_names = set(dir(item)) - _readonly_attrs

    for sitem_name in attr_names:

        sitem = getattr(item, sitem_name)
        new_sitem = getattr(new_item, sitem_name)

        try:
            # Set the item
            setattr(item, sitem_name, new_sitem)

            try:
                # Will work for classes and functions defined in that module.
                mod_name = sitem.__module__
            except AttributeError:
                mod_name = None

            # If this item was defined within this module, deep-reset
            if (mod_name is None) or (mod_name != module_name) or (id(sitem) in reset_items) \
                    or isinstance(sitem, EnumMeta):  # Deal with enums
                continue

            reset_items.add(id(sitem))
            _reset_item_recursively(sitem, new_sitem, module_name, reset_items)
        except Exception as ex:
            raise Exception(sitem_name) from ex

注意:小心使用!在非外围模块(例如,定义外部使用的类的模块)上使用它们可能会导致Python内部发生问题(例如,酸洗/不酸洗问题)。

Edit (Answer V2)

The solution from before is good for just getting the reset information, but it will not change all the references (more than reload but less then required). To actually set all the references as well, I had to go into the garbage collector, and rewrite the references there. Now it works like a charm!

Note that this will not work if the GC is turned off, or if reloading data that’s not monitored by the GC. If you don’t want to mess with the GC, the original answer might be enough for you.

New code:

import importlib
import inspect
import gc
from weakref import ref


def reset_module(module, inner_modules_also=True):
    """
    This function is a stronger form of importlib's `reload` function. What it does, is that aside from reloading a
    module, it goes to the old instance of the module, and sets all the (not read-only) attributes, functions and classes
    to be the reloaded-module's
    :param module: The module to reload (module reference, not the name)
    :param inner_modules_also: Whether to treat ths module as a package as well, and reload all the modules within it.
    """

    # For the case when the module is actually a package
    if inner_modules_also:
        submods = {submod for _, submod in inspect.getmembers(module)
                   if (type(submod).__name__ == 'module') and (submod.__package__.startswith(module.__name__))}
        for submod in submods:
            reset_module(submod, True)

    # First, log all the references before reloading (because some references may be changed by the reload operation).
    module_tree = _get_tree_references_to_reset_recursively(module, module.__name__)

    new_module = importlib.reload(module)
    _reset_item_recursively(module, module_tree, new_module)


def _update_referrers(item, new_item):
    refs = gc.get_referrers(item)

    weak_ref_item = ref(item)
    for coll in refs:
        if type(coll) == dict:
            enumerator = coll.keys()
        elif type(coll) == list:
            enumerator = range(len(coll))
        else:
            continue

        for key in enumerator:

            if weak_ref_item() is None:
                # No refs are left in the GC
                return

            if coll[key] is weak_ref_item():
                coll[key] = new_item

def _get_tree_references_to_reset_recursively(item, module_name, grayed_out_item_ids = None):
    if grayed_out_item_ids is None:
        grayed_out_item_ids = set()

    item_tree = dict()
    attr_names = set(dir(item)) - _readonly_attrs
    for sub_item_name in attr_names:

        sub_item = getattr(item, sub_item_name)
        item_tree[sub_item_name] = [sub_item, None]

        try:
            # Will work for classes and functions defined in that module.
            mod_name = sub_item.__module__
        except AttributeError:
            mod_name = None

        # If this item was defined within this module, deep-reset
        if (mod_name is None) or (mod_name != module_name) or (id(sub_item) in grayed_out_item_ids) \
                or isinstance(sub_item, EnumMeta):
            continue

        grayed_out_item_ids.add(id(sub_item))
        item_tree[sub_item_name][1] = \
            _get_tree_references_to_reset_recursively(sub_item, module_name, grayed_out_item_ids)

    return item_tree


def _reset_item_recursively(item, item_subtree, new_item):

    # Set children first so we don't lose the current references.
    if item_subtree is not None:
        for sub_item_name, (sub_item, sub_item_tree) in item_subtree.items():

            try:
                new_sub_item = getattr(new_item, sub_item_name)
            except AttributeError:
                # The item doesn't exist in the reloaded module. Ignore.
                continue

            try:
                # Set the item
                _reset_item_recursively(sub_item, sub_item_tree, new_sub_item)
            except Exception as ex:
                pass

    _update_referrers(item, new_item)

Original Answer

As written in @bobince’s answer, if there’s already a reference to that module in another module (especially if it was imported with the as keyword like import numpy as np), that instance will not be overwritten.

This proved quite problematic to me when applying tests that required a “clean-slate” state of the configuration modules, so I’ve written a function named reset_module that uses importlib‘s reload function and recursively overwrites all the declared module’s attributes. It has been tested with Python version 3.6.

import importlib
import inspect
from enum import EnumMeta

_readonly_attrs = {'__annotations__', '__call__', '__class__', '__closure__', '__code__', '__defaults__', '__delattr__',
               '__dict__', '__dir__', '__doc__', '__eq__', '__format__', '__func__', '__ge__', '__get__',
               '__getattribute__', '__globals__', '__gt__', '__hash__', '__init__', '__init_subclass__',
               '__kwdefaults__', '__le__', '__lt__', '__module__', '__name__', '__ne__', '__new__', '__qualname__',
               '__reduce__', '__reduce_ex__', '__repr__', '__self__', '__setattr__', '__sizeof__', '__str__',
               '__subclasshook__', '__weakref__', '__members__', '__mro__', '__itemsize__', '__isabstractmethod__',
               '__basicsize__', '__base__'}


def reset_module(module, inner_modules_also=True):
    """
    This function is a stronger form of importlib's `reload` function. What it does, is that aside from reloading a
    module, it goes to the old instance of the module, and sets all the (not read-only) attributes, functions and classes
    to be the reloaded-module's
    :param module: The module to reload (module reference, not the name)
    :param inner_modules_also: Whether to treat ths module as a package as well, and reload all the modules within it.
    """

    new_module = importlib.reload(module)

    reset_items = set()

    # For the case when the module is actually a package
    if inner_modules_also:
        submods = {submod for _, submod in inspect.getmembers(module)
                   if (type(submod).__name__ == 'module') and (submod.__package__.startswith(module.__name__))}
        for submod in submods:
            reset_module(submod, True)

    _reset_item_recursively(module, new_module, module.__name__, reset_items)


def _reset_item_recursively(item, new_item, module_name, reset_items=None):
    if reset_items is None:
        reset_items = set()

    attr_names = set(dir(item)) - _readonly_attrs

    for sitem_name in attr_names:

        sitem = getattr(item, sitem_name)
        new_sitem = getattr(new_item, sitem_name)

        try:
            # Set the item
            setattr(item, sitem_name, new_sitem)

            try:
                # Will work for classes and functions defined in that module.
                mod_name = sitem.__module__
            except AttributeError:
                mod_name = None

            # If this item was defined within this module, deep-reset
            if (mod_name is None) or (mod_name != module_name) or (id(sitem) in reset_items) \
                    or isinstance(sitem, EnumMeta):  # Deal with enums
                continue

            reset_items.add(id(sitem))
            _reset_item_recursively(sitem, new_sitem, module_name, reset_items)
        except Exception as ex:
            raise Exception(sitem_name) from ex

Note: Use with care! Using these on non-peripheral modules (modules that define externally-used classes, for example) might lead to internal problems in Python (such as pickling/un-pickling issues).


回答 16

对我而言,Abaqus就是这种方式。假设您的文件是Class_VerticesEdges.py

sys.path.append('D:\...\My Pythons')
if 'Class_VerticesEdges' in sys.modules:  
    del sys.modules['Class_VerticesEdges']
    print 'old module Class_VerticesEdges deleted'
from Class_VerticesEdges import *
reload(sys.modules['Class_VerticesEdges'])

for me for case of Abaqus it is the way it works. Imagine your file is Class_VerticesEdges.py

sys.path.append('D:\...\My Pythons')
if 'Class_VerticesEdges' in sys.modules:  
    del sys.modules['Class_VerticesEdges']
    print 'old module Class_VerticesEdges deleted'
from Class_VerticesEdges import *
reload(sys.modules['Class_VerticesEdges'])

回答 17

尝试在Sublime Text中重新加载某些内容时遇到了很多麻烦,但最终我可以编写此实用程序,根据代码在Sublime Text上重新加载模块 sublime_plugin.py用于重新加载模块重新加载模块。

下面的内容允许您从路径上带有空格的模块中重新加载模块,然后在重新加载之后,您可以照常导入。

def reload_module(full_module_name):
    """
        Assuming the folder `full_module_name` is a folder inside some
        folder on the python sys.path, for example, sys.path as `C:/`, and
        you are inside the folder `C:/Path With Spaces` on the file 
        `C:/Path With Spaces/main.py` and want to re-import some files on
        the folder `C:/Path With Spaces/tests`

        @param full_module_name   the relative full path to the module file
                                  you want to reload from a folder on the
                                  python `sys.path`
    """
    import imp
    import sys
    import importlib

    if full_module_name in sys.modules:
        module_object = sys.modules[full_module_name]
        module_object = imp.reload( module_object )

    else:
        importlib.import_module( full_module_name )

def run_tests():
    print( "\n\n" )
    reload_module( "Path With Spaces.tests.semantic_linefeed_unit_tests" )
    reload_module( "Path With Spaces.tests.semantic_linefeed_manual_tests" )

    from .tests import semantic_linefeed_unit_tests
    from .tests import semantic_linefeed_manual_tests

    semantic_linefeed_unit_tests.run_unit_tests()
    semantic_linefeed_manual_tests.run_manual_tests()

if __name__ == "__main__":
    run_tests()

如果是第一次运行,则应该加载该模块,但是如果以后可以再次使用该方法/功能run_tests(),它将重新加载测试文件。使用Sublime Text(Python 3.3.6)会发生很多事情,因为它的解释器永远不会关闭(除非您重新启动Sublime Text,即Python3.3解释器)。

I got a lot of trouble trying to reload something inside Sublime Text, but finally I could wrote this utility to reload modules on Sublime Text based on the code sublime_plugin.py uses to reload modules.

This below accepts you to reload modules from paths with spaces on their names, then later after reloading you can just import as you usually do.

def reload_module(full_module_name):
    """
        Assuming the folder `full_module_name` is a folder inside some
        folder on the python sys.path, for example, sys.path as `C:/`, and
        you are inside the folder `C:/Path With Spaces` on the file 
        `C:/Path With Spaces/main.py` and want to re-import some files on
        the folder `C:/Path With Spaces/tests`

        @param full_module_name   the relative full path to the module file
                                  you want to reload from a folder on the
                                  python `sys.path`
    """
    import imp
    import sys
    import importlib

    if full_module_name in sys.modules:
        module_object = sys.modules[full_module_name]
        module_object = imp.reload( module_object )

    else:
        importlib.import_module( full_module_name )

def run_tests():
    print( "\n\n" )
    reload_module( "Path With Spaces.tests.semantic_linefeed_unit_tests" )
    reload_module( "Path With Spaces.tests.semantic_linefeed_manual_tests" )

    from .tests import semantic_linefeed_unit_tests
    from .tests import semantic_linefeed_manual_tests

    semantic_linefeed_unit_tests.run_unit_tests()
    semantic_linefeed_manual_tests.run_manual_tests()

if __name__ == "__main__":
    run_tests()

If you run for the first time, this should load the module, but if later you can again the method/function run_tests() it will reload the tests files. With Sublime Text (Python 3.3.6) this happens a lot because its interpreter never closes (unless you restart Sublime Text, i.e., the Python3.3 interpreter).


回答 18

另一种方法是将模块导入功能中。这样,当函数完成时,模块将收集垃圾。

Another way could be to import the module in a function. This way when the function completes the module gets garbage collected.