在IPython中重新加载子模块

问题:在IPython中重新加载子模块

目前,我正在处理一个包含子模块并使用numpy / scipy的python项目。Ipython用作交互式控制台。不幸的是,我对现在使用的工作流程不是很满意,请多多指教。

在IPython中,该框架是通过一个简单的import命令加载的。但是,通常有必要在框架的子模块之一中更改代码。至此,已经加载了一个模型,并且我使用IPython与之交互。

现在,该框架包含许多相互依赖的模块,即,在最初加载该框架时,主模块正在导入和配置子模块。仅当使用重新加载模块时,才执行对代码的更改reload(main_mod.sub_mod)。这很麻烦,因为我需要使用完整路径分别重新加载所有更改的模块。如果reload(main_module)还重新加载所有子模块,但又不重新加载numpy / scipy ,将非常方便。

Currently I am working on a python project that contains sub modules and uses numpy/scipy. Ipython is used as interactive console. Unfortunately I am not very happy with workflow that I am using right now, I would appreciate some advice.

In IPython, the framework is loaded by a simple import command. However, it is often necessary to change code in one of the submodules of the framework. At this point a model is already loaded and I use IPython to interact with it.

Now, the framework contains many modules that depend on each other, i.e. when the framework is initially loaded the main module is importing and configuring the submodules. The changes to the code are only executed if the module is reloaded using reload(main_mod.sub_mod). This is cumbersome as I need to reload all changed modules individually using the full path. It would be very convenient if reload(main_module) would also reload all sub modules, but without reloading numpy/scipy..


回答 0

IPython带有一些自动重装魔术:

%load_ext autoreload
%autoreload 2

每次执行新行之前,它将重新加载所有更改的模块。它的工作方式与稍有不同dreload。有一些警告,请键入%autoreload?以查看可能出问题的地方。


如果要始终启用此设置,请修改IPython配置文件~/.ipython/profile_default/ipython_config.py[1]并附加:

c.InteractiveShellApp.extensions = ['autoreload']     
c.InteractiveShellApp.exec_lines = ['%autoreload 2']

通过下面的评论归功于@Kos。

[1]如果您没有该文件~/.ipython/profile_default/ipython_config.py,则需要先调用ipython profile create。或者文件可能位于$IPYTHONDIR

IPython comes with some automatic reloading magic:

%load_ext autoreload
%autoreload 2

It will reload all changed modules every time before executing a new line. The way this works is slightly different than dreload. Some caveats apply, type %autoreload? to see what can go wrong.


If you want to always enable this settings, modify your IPython configuration file ~/.ipython/profile_default/ipython_config.py[1] and appending:

c.InteractiveShellApp.extensions = ['autoreload']     
c.InteractiveShellApp.exec_lines = ['%autoreload 2']

Credit to @Kos via a comment below.

[1] If you don’t have the file ~/.ipython/profile_default/ipython_config.py, you need to call ipython profile create first. Or the file may be located at $IPYTHONDIR.


回答 1

在IPython 0.12(可能更早)中,您可以使用以下命令:

%load_ext autoreload
%autoreload 2

这与pv的答案基本相同,除了扩展名已重命名并现在使用加载%load_ext

In IPython 0.12 (and possibly earlier), you can use this:

%load_ext autoreload
%autoreload 2

This is essentially the same as the answer by pv., except that the extension has been renamed and is now loaded using %load_ext.


回答 2

由于某种原因,当您将代码从一个笔记本导入另一个笔记本时,这两种方法都不起作用%autoreload,也dreload似乎不起作用。只有普通的Python 可以工作:reload

reload(module)

基于[1]

For some reason, neither %autoreload, nor dreload seem to work for the situation when you import code from one notebook to another. Only plain Python reload works:

reload(module)

Based on [1].


回答 3

IPython提供dreload()了递归方式重新加载所有子模块。就个人而言,我更喜欢使用%run()magic命令(尽管它不会执行深度重新加载,正如John Salvatier在评论中指出的那样)。

IPython offers dreload() to recursively reload all submodules. Personally, I prefer to use the %run() magic command (though it does not perform a deep reload, as pointed out by John Salvatier in the comments).


回答 4

名为importliballow的模块可以访问导入内部构件。特别是,它提供功能importlib.reload()

import importlib
importlib.reload(my_module)

与此相反%autoreloadimportlib.reload()还重置模块中设置的全局变量。在大多数情况下,这就是您想要的。

importlib仅从Python 3.1开始可用。对于旧版本,您必须使用module imp

Module named importlib allow to access to import internals. Especially, it provide function importlib.reload():

import importlib
importlib.reload(my_module)

In contrary of %autoreload, importlib.reload() also reset global variables set in module. In most cases, it is what you want.

importlib is only available since Python 3.1. For older version, you have to use module imp.


回答 5

http://shawnleezx.github.io/blog/2015/08/03/some-notes-on-ipython-startup-script/

为了避免一遍又一遍地输入这些魔术函数,可以将它们放在ipython启动脚本中(用.ipython / profile_default / startup下的.py后缀命名。该文件夹下的所有python脚本将按照词法顺序加载),看起来如下:

from IPython import get_ipython
ipython = get_ipython()

ipython.magic("pylab")
ipython.magic("load_ext autoreload")
ipython.magic("autoreload 2")

http://shawnleezx.github.io/blog/2015/08/03/some-notes-on-ipython-startup-script/

To avoid typing those magic function again and again, they could be put in the ipython startup script(Name it with .py suffix under .ipython/profile_default/startup. All python scripts under that folder will be loaded according to lexical order), which looks like the following:

from IPython import get_ipython
ipython = get_ipython()

ipython.magic("pylab")
ipython.magic("load_ext autoreload")
ipython.magic("autoreload 2")

回答 6

这个怎么样:

import inspect

# needs to be primed with an empty set for loaded
def recursively_reload_all_submodules(module, loaded=None):
    for name in dir(module):
        member = getattr(module, name)
        if inspect.ismodule(member) and member not in loaded:
            recursively_reload_all_submodules(member, loaded)
    loaded.add(module)
    reload(module)

import mymodule
recursively_reload_all_submodules(mymodule, set())

这样可以有效地重新加载您为其提供的整个模块树和子模块树。您也可以将此函数放在.ipythonrc中(我认为),以便每次启动解释器时都将其加载。

How about this:

import inspect

# needs to be primed with an empty set for loaded
def recursively_reload_all_submodules(module, loaded=None):
    for name in dir(module):
        member = getattr(module, name)
        if inspect.ismodule(member) and member not in loaded:
            recursively_reload_all_submodules(member, loaded)
    loaded.add(module)
    reload(module)

import mymodule
recursively_reload_all_submodules(mymodule, set())

This should effectively reload the entire tree of modules and submodules you give it. You can also put this function in your .ipythonrc (I think) so it is loaded every time you start the interpreter.


回答 7

另外一个选项:

$ cat << EOF > ~/.ipython/profile_default/startup/50-autoreload.ipy
%load_ext autoreload
%autoreload 2
EOF

在Ubuntu 14.04上的ipython和ipython3 v5.1.0上进行了验证。

Another option:

$ cat << EOF > ~/.ipython/profile_default/startup/50-autoreload.ipy
%load_ext autoreload
%autoreload 2
EOF

Verified on ipython and ipython3 v5.1.0 on Ubuntu 14.04.


回答 8

我的重载标准做法是在首次打开时将两种方法结合起来IPython

from IPython.lib.deepreload import reload
%load_ext autoreload
%autoreload 2

在执行此操作之前加载模块将导致即使使用手册也无法重新加载它们reload(module_name)。我仍然很少遇到类方法无法重装的莫名其妙的问题,而我尚未研究过。

My standard practice for reloading is to combine both methods following first opening of IPython:

from IPython.lib.deepreload import reload
%load_ext autoreload
%autoreload 2

Loading modules before doing this will cause them not to be reloaded, even with a manual reload(module_name). I still, very rarely, get inexplicable problems with class methods not reloading that I’ve not yet looked into.


回答 9

请注意,autoreload如果您手动保存更改的文件(例如,使用ctrl + s或cmd + s),则上述内容仅在IntelliJ中有效。自动保存似乎不起作用。

Note that the above mentioned autoreload only works in IntelliJ if you manually save the changed file (e.g. using ctrl+s or cmd+s). It doesn’t seem to work with auto-saving.


回答 10

在Anaconda的Jupyter笔记本上,执行以下操作:

%load_ext autoreload
%autoreload 2

产生了消息:

autoreload扩展程序已加载。要重新加载它,请使用: %reload_ext autoreload

看起来最好这样做:

%reload_ext autoreload
%autoreload 2

版本信息:

笔记本服务器的版本为5.0.0,并且运行在:Python 3.6.2 | Anaconda,Inc. | (默认值,2017年9月20日,13:35:58)[MSC v.1900 32位(Intel)]

On Jupyter Notebooks on Anaconda, doing this:

%load_ext autoreload
%autoreload 2

produced the message:

The autoreload extension is already loaded. To reload it, use: %reload_ext autoreload

It looks like it’s preferable to do:

%reload_ext autoreload
%autoreload 2

Version information:

The version of the notebook server is 5.0.0 and is running on: Python 3.6.2 |Anaconda, Inc.| (default, Sep 20 2017, 13:35:58) [MSC v.1900 32 bit (Intel)]


回答 11

此操作将不会重新加载任何子对象,我相信您必须为此使用IPython的deepreload。

Any subobjects will not be reloaded by this, I believe you have to use IPython’s deepreload for that.