标签归档:siblings

兄弟包进口

问题:兄弟包进口

我已经尝试阅读有关同级导入甚至 包文档的问题,但是我还没有找到答案。

具有以下结构:

├── LICENSE.md
├── README.md
├── api
   ├── __init__.py
   ├── api.py
   └── api_key.py
├── examples
   ├── __init__.py
   ├── example_one.py
   └── example_two.py
└── tests
   ├── __init__.py
   └── test_one.py

examplestests目录中的脚本如何 从api模块导入 并从命令行运行?

另外,我想避免sys.path.insert对每个文件进行难看的修改。当然可以在Python中完成,对吗?

I’ve tried reading through questions about sibling imports and even the package documentation, but I’ve yet to find an answer.

With the following structure:

├── LICENSE.md
├── README.md
├── api
│   ├── __init__.py
│   ├── api.py
│   └── api_key.py
├── examples
│   ├── __init__.py
│   ├── example_one.py
│   └── example_two.py
└── tests
│   ├── __init__.py
│   └── test_one.py

How can the scripts in the examples and tests directories import from the api module and be run from the commandline?

Also, I’d like to avoid the ugly sys.path.insert hack for every file. Surely this can be done in Python, right?


回答 0

七年后

自从我在下面写下答案以来,修改sys.path仍然是一种快速技巧,对于私有脚本来说效果很好,但是已经有了一些改进

  • 安装该软件包(无论是否在virtualenv中)都将为您提供所需的内容,尽管我建议使用pip进行操作,而不是直接使用setuptools(并setup.cfg用于存储元数据)
  • 使用该-m标志并作为软件包运行也可以(但是如果要将工作目录转换为可安装的软件包,将会有些尴尬)。
  • 对于测试,特别是pytest能够在这种情况下找到api程序包,并sys.path为您解决问题

因此,这实际上取决于您要做什么。但是,就您而言,既然您的目标似乎是在某个时候制作一个合适的程序包,那么通过安装pip -e可能是您最好的选择,即使它尚不完美。

旧答案

正如其他地方已经说过的那样,可怕的事实是,您必须进行丑陋的修改才能允许从同级模块中导入数据或从该__main__模块中的父级程序包中进行导入。PEP 366中详细介绍了该问题。PEP 3122试图以更合理的方式处理进口,但圭多拒绝了它的一项

唯一的用例似乎是正在运行的脚本,它们恰好位于模块的目录中,我一直将其视为反模式。

这里

不过,我会定期使用这种模式

# Ugly hack to allow absolute import from the root folder
# whatever its name is. Please forgive the heresy.
if __name__ == "__main__" and __package__ is None:
    from sys import path
    from os.path import dirname as dir

    path.append(dir(path[0]))
    __package__ = "examples"

import api

path[0]是运行脚本的父文件夹和dir(path[0])顶级文件夹。

虽然我仍然不能使用相对导入,但是它确实允许从顶层(在您的示例api的父文件夹中)进行绝对导入。

Seven years after

Since I wrote the answer below, modifying sys.path is still a quick-and-dirty trick that works well for private scripts, but there has been several improvements

  • Installing the package (in a virtualenv or not) will give you what you want, though I would suggest using pip to do it rather than using setuptools directly (and using setup.cfg to store the metadata)
  • Using the -m flag and running as a package works too (but will turn out a bit awkward if you want to convert your working directory into an installable package).
  • For the tests, specifically, pytest is able to find the api package in this situation and takes care of the sys.path hacks for you

So it really depends on what you want to do. In your case, though, since it seems that your goal is to make a proper package at some point, installing through pip -e is probably your best bet, even if it is not perfect yet.

Old answer

As already stated elsewhere, the awful truth is that you have to do ugly hacks to allow imports from siblings modules or parents package from a __main__ module. The issue is detailed in PEP 366. PEP 3122 attempted to handle imports in a more rational way but Guido has rejected it one the account of

The only use case seems to be running scripts that happen to be living inside a module’s directory, which I’ve always seen as an antipattern.

(here)

Though, I use this pattern on a regular basis with

# Ugly hack to allow absolute import from the root folder
# whatever its name is. Please forgive the heresy.
if __name__ == "__main__" and __package__ is None:
    from sys import path
    from os.path import dirname as dir

    path.append(dir(path[0]))
    __package__ = "examples"

import api

Here path[0] is your running script’s parent folder and dir(path[0]) your top level folder.

I have still not been able to use relative imports with this, though, but it does allow absolute imports from the top level (in your example api‘s parent folder).


回答 1

厌倦了sys.path hacks?

有大量的sys.path.append-hacks,但是我找到了另一种解决问题的方法。

摘要

  • 将代码包装到一个文件夹中(例如packaged_stuff
  • setup.py在使用setuptools.setup()的地方使用创建脚本。
  • 使用以下命令以可编辑状态安装软件包 pip install -e <myproject_folder>
  • 导入使用 from packaged_stuff.modulename import function_name

建立

起点是您提供的文件结构,包装在名为的文件夹中myproject

.
└── myproject
    ├── api
       ├── api_key.py
       ├── api.py
       └── __init__.py
    ├── examples
       ├── example_one.py
       ├── example_two.py
       └── __init__.py
    ├── LICENCE.md
    ├── README.md
    └── tests
        ├── __init__.py
        └── test_one.py

我将调用.根文件夹,在本例中,它位于C:\tmp\test_imports\

api.py

作为测试用例,让我们使用以下./api/api.py

def function_from_api():
    return 'I am the return value from api.api!'

test_one.py

from api.api import function_from_api

def test_function():
    print(function_from_api())

if __name__ == '__main__':
    test_function()

尝试运行test_one:

PS C:\tmp\test_imports> python .\myproject\tests\test_one.py
Traceback (most recent call last):
  File ".\myproject\tests\test_one.py", line 1, in <module>
    from api.api import function_from_api
ModuleNotFoundError: No module named 'api'

还尝试相对进口将无法正常工作:

使用from ..api.api import function_from_api会导致

PS C:\tmp\test_imports> python .\myproject\tests\test_one.py
Traceback (most recent call last):
  File ".\tests\test_one.py", line 1, in <module>
    from ..api.api import function_from_api
ValueError: attempted relative import beyond top-level package

脚步

  1. 将setup.py文件创建到根目录

的内容为setup.py*

from setuptools import setup, find_packages

setup(name='myproject', version='1.0', packages=find_packages())
  1. 使用虚拟环境

如果您熟悉虚拟环境,请激活一个,然后跳到下一步。虚拟环境的使用不是绝对必需的,但从长远来看(当您正在进行多个项目时),它们确实可以帮助您。最基本的步骤是(在根文件夹中运行)

  • 创建虚拟环境
    • python -m venv venv
  • 激活虚拟环境
    • source ./venv/bin/activate(Linux,macOS)或./venv/Scripts/activate(Win)

要了解更多信息,只需在Google上搜索“ python虚拟环境教程”或类似内容即可。除了创建,激活和停用之外,您可能根本不需要任何其他命令。

创建并激活虚拟环境后,控制台应在括号中提供虚拟环境的名称。

PS C:\tmp\test_imports> python -m venv venv
PS C:\tmp\test_imports> .\venv\Scripts\activate
(venv) PS C:\tmp\test_imports>

您的文件夹树应如下所示**

.
├── myproject
   ├── api
      ├── api_key.py
      ├── api.py
      └── __init__.py
   ├── examples
      ├── example_one.py
      ├── example_two.py
      └── __init__.py
   ├── LICENCE.md
   ├── README.md
   └── tests
       ├── __init__.py
       └── test_one.py
├── setup.py
└── venv
    ├── Include
    ├── Lib
    ├── pyvenv.cfg
    └── Scripts [87 entries exceeds filelimit, not opening dir]
  1. pip以可编辑状态安装项目

安装您的顶级包myproject使用pip。诀窍是-e在执行安装时使用标志。这样,它以可编辑状态安装,并且对.py文件所做的所有编辑将自动包含在已安装的软件包中。

在根目录中,运行

pip install -e . (注意点,它代表“当前目录”)

您还可以看到它是通过使用安装的 pip freeze

(venv) PS C:\tmp\test_imports> pip install -e .
Obtaining file:///C:/tmp/test_imports
Installing collected packages: myproject
  Running setup.py develop for myproject
Successfully installed myproject
(venv) PS C:\tmp\test_imports> pip freeze
myproject==1.0
  1. 添加myproject.到您的进口中

请注意,您将只需要添加myproject.导入否则将无法正常工作。不能使用setup.py&导入的导入pip install仍然可以正常工作。请参见下面的示例。


测试解决方案

现在,让我们使用api.py上面test_one.py定义的和下面定义的测试解决方案。

test_one.py

from myproject.api.api import function_from_api

def test_function():
    print(function_from_api())

if __name__ == '__main__':
    test_function()

运行测试

(venv) PS C:\tmp\test_imports> python .\myproject\tests\test_one.py
I am the return value from api.api!

*有关更多详细的setup.py示例,请参阅setuptools文档

**实际上,您可以将虚拟环境放在硬盘上的任何位置。

Tired of sys.path hacks?

There are plenty of sys.path.append -hacks available, but I found an alternative way of solving the problem in hand.

Summary

  • Wrap the code into one folder (e.g. packaged_stuff)
  • Use create setup.py script where you use setuptools.setup().
  • Pip install the package in editable state with pip install -e <myproject_folder>
  • Import using from packaged_stuff.modulename import function_name

Setup

The starting point is the file structure you have provided, wrapped in a folder called myproject.

.
└── myproject
    ├── api
    │   ├── api_key.py
    │   ├── api.py
    │   └── __init__.py
    ├── examples
    │   ├── example_one.py
    │   ├── example_two.py
    │   └── __init__.py
    ├── LICENCE.md
    ├── README.md
    └── tests
        ├── __init__.py
        └── test_one.py

I will call the . the root folder, and in my example case it is located at C:\tmp\test_imports\.

api.py

As a test case, let’s use the following ./api/api.py

def function_from_api():
    return 'I am the return value from api.api!'

test_one.py

from api.api import function_from_api

def test_function():
    print(function_from_api())

if __name__ == '__main__':
    test_function()

Try to run test_one:

PS C:\tmp\test_imports> python .\myproject\tests\test_one.py
Traceback (most recent call last):
  File ".\myproject\tests\test_one.py", line 1, in <module>
    from api.api import function_from_api
ModuleNotFoundError: No module named 'api'

Also trying relative imports wont work:

Using from ..api.api import function_from_api would result into

PS C:\tmp\test_imports> python .\myproject\tests\test_one.py
Traceback (most recent call last):
  File ".\tests\test_one.py", line 1, in <module>
    from ..api.api import function_from_api
ValueError: attempted relative import beyond top-level package

Steps

  1. Make a setup.py file to the root level directory

The contents for the setup.py would be*

from setuptools import setup, find_packages

setup(name='myproject', version='1.0', packages=find_packages())
  1. Use a virtual environment

If you are familiar with virtual environments, activate one, and skip to the next step. Usage of virtual environments are not absolutely required, but they will really help you out in the long run (when you have more than 1 project ongoing..). The most basic steps are (run in the root folder)

  • Create virtual env
    • python -m venv venv
  • Activate virtual env
    • source ./venv/bin/activate (Linux, macOS) or ./venv/Scripts/activate (Win)

To learn more about this, just Google out “python virtual env tutorial” or similar. You probably never need any other commands than creating, activating and deactivating.

Once you have made and activated a virtual environment, your console should give the name of the virtual environment in parenthesis

PS C:\tmp\test_imports> python -m venv venv
PS C:\tmp\test_imports> .\venv\Scripts\activate
(venv) PS C:\tmp\test_imports>

and your folder tree should look like this**

.
├── myproject
│   ├── api
│   │   ├── api_key.py
│   │   ├── api.py
│   │   └── __init__.py
│   ├── examples
│   │   ├── example_one.py
│   │   ├── example_two.py
│   │   └── __init__.py
│   ├── LICENCE.md
│   ├── README.md
│   └── tests
│       ├── __init__.py
│       └── test_one.py
├── setup.py
└── venv
    ├── Include
    ├── Lib
    ├── pyvenv.cfg
    └── Scripts [87 entries exceeds filelimit, not opening dir]
  1. pip install your project in editable state

Install your top level package myproject using pip. The trick is to use the -e flag when doing the install. This way it is installed in an editable state, and all the edits made to the .py files will be automatically included in the installed package.

In the root directory, run

pip install -e . (note the dot, it stands for “current directory”)

You can also see that it is installed by using pip freeze

(venv) PS C:\tmp\test_imports> pip install -e .
Obtaining file:///C:/tmp/test_imports
Installing collected packages: myproject
  Running setup.py develop for myproject
Successfully installed myproject
(venv) PS C:\tmp\test_imports> pip freeze
myproject==1.0
  1. Add myproject. into your imports

Note that you will have to add myproject. only into imports that would not work otherwise. Imports that worked without the setup.py & pip install will work still work fine. See an example below.


Test the solution

Now, let’s test the solution using api.py defined above, and test_one.py defined below.

test_one.py

from myproject.api.api import function_from_api

def test_function():
    print(function_from_api())

if __name__ == '__main__':
    test_function()

running the test

(venv) PS C:\tmp\test_imports> python .\myproject\tests\test_one.py
I am the return value from api.api!

* See the setuptools docs for more verbose setup.py examples.

** In reality, you could put your virtual environment anywhere on your hard disk.


回答 2

这是我在文件tests夹中的Python文件顶部插入的另一种选择:

# Path hack.
import sys, os
sys.path.insert(0, os.path.abspath('..'))

Here is another alternative that I insert at top of the Python files in tests folder:

# Path hack.
import sys, os
sys.path.insert(0, os.path.abspath('..'))

回答 3

sys.path除非有必要,否则您不需要,也不应hack ,在这种情况下则没有必要。用:

import api.api_key # in tests, examples

从项目目录运行:python -m tests.test_one

您可能应该tests在内部移动(如果它们是api的unittests)api并运行python -m api.test以运行所有测试(假设存在__main__.py)或python -m api.test.test_one改为运行test_one

您也可以__init__.pyexamples(不是Python软件包)中删除该示例,并在api安装了virtualenv的示例中运行示例,例如,如果您具有适当的条件,则pip install -e .在virtualenv中将安装就地api软件包setup.py

You don’t need and shouldn’t hack sys.path unless it is necessary and in this case it is not. Use:

import api.api_key # in tests, examples

Run from the project directory: python -m tests.test_one.

You should probably move tests (if they are api’s unittests) inside api and run python -m api.test to run all tests (assuming there is __main__.py) or python -m api.test.test_one to run test_one instead.

You could also remove __init__.py from examples (it is not a Python package) and run the examples in a virtualenv where api is installed e.g., pip install -e . in a virtualenv would install inplace api package if you have proper setup.py.


回答 4

我还没有对Python的理解,没有看到在没有同级/相对导入hack的情况下在不相关的项目之间共享代码的预期方式所必需的。直到那天,这是我的解决方案。对于examplestests从中导入东西..\api,它看起来像:

import sys.path
import os.path
# Import from sibling directory ..\api
sys.path.append(os.path.dirname(os.path.abspath(__file__)) + "/..")
import api.api
import api.api_key

I don’t yet have the comprehension of Pythonology necessary to see the intended way of sharing code amongst unrelated projects without a sibling/relative import hack. Until that day, this is my solution. For examples or tests to import stuff from ..\api, it would look like:

import sys.path
import os.path
# Import from sibling directory ..\api
sys.path.append(os.path.dirname(os.path.abspath(__file__)) + "/..")
import api.api
import api.api_key

回答 5

对于同级包导入,可以使用[sys.path] [2]模块的insertappend方法:

if __name__ == '__main__' and if __package__ is None:
    import sys
    from os import path
    sys.path.append( path.dirname( path.dirname( path.abspath(__file__) ) ) )
    import api

如果您按以下方式启动脚本,这将起作用:

python examples/example_one.py
python tests/test_one.py

另一方面,您也可以使用相对导入:

if __name__ == '__main__' and if __package__ is not None:
    import ..api.api

在这种情况下,您将必须使用‘-m’参数启动脚本(请注意,在这种情况下,您不得使用‘.py’扩展名):

python -m packageName.examples.example_one
python -m packageName.tests.test_one

当然,您可以将两种方法混合使用,以便您的脚本无论如何调用都可以工作:

if __name__ == '__main__':
    if __package__ is None:
        import sys
        from os import path
        sys.path.append( path.dirname( path.dirname( path.abspath(__file__) ) ) )
        import api
    else:
        import ..api.api

For siblings package imports, you can use either the insert or the append method of the [sys.path][2] module:

if __name__ == '__main__' and if __package__ is None:
    import sys
    from os import path
    sys.path.append( path.dirname( path.dirname( path.abspath(__file__) ) ) )
    import api

This will work if you are launching your scripts as follows:

python examples/example_one.py
python tests/test_one.py

On the other hand, you can also use the relative import:

if __name__ == '__main__' and if __package__ is not None:
    import ..api.api

In this case you will have to launch your script with the ‘-m’ argument (note that, in this case, you must not give the ‘.py’ extension):

python -m packageName.examples.example_one
python -m packageName.tests.test_one

Of course, you can mix the two approaches, so that your script will work no matter how it is called:

if __name__ == '__main__':
    if __package__ is None:
        import sys
        from os import path
        sys.path.append( path.dirname( path.dirname( path.abspath(__file__) ) ) )
        import api
    else:
        import ..api.api

回答 6

TLDR

此方法不需要setuptools,path hacks,其他命令行参数或在项目的每个文件中指定软件包的顶层。

只需在要调用的父目录中创建一个脚本,__main__然后从那里运行所有内容即可。有关进一步的说明,请继续阅读。

说明

这可以实现,而无需一起寻找新路径,使用额外的命令行参数或向您的每个程序添加代码以识别其兄弟姐妹。

我相信之前提到的失败的原因是被调用的程序将其__name__设置为__main__。发生这种情况时,被调用的脚本会接受自身位于程序包的顶层,并拒绝识别兄弟目录中的脚本。

但是,目录顶层下的所有内容仍然可以识别顶层下的任何内容。这意味着要使同级目录中的文件相互识别/利用,您只需要做的就是从其父目录中的脚本中调用它们。

概念证明 在具有以下结构的目录中:

.
|__Main.py
|
|__Siblings
   |
   |___sib1
   |   |
   |   |__call.py
   |
   |___sib2
       |
       |__callsib.py

Main.py 包含以下代码:

import sib1.call as call


def main():
    call.Call()


if __name__ == '__main__':
    main()

sib1 / call.py包含:

import sib2.callsib as callsib


def Call():
    callsib.CallSib()


if __name__ == '__main__':
    Call()

sib2 / callsib.py包含:

def CallSib():
    print("Got Called")

if __name__ == '__main__':
    CallSib()

如果重现此示例,您将注意到, 即使通过Main.py调用,调用也会导致按定义打印“ Got Called” 。但是,如果要直接调用(在对导入进行适当更改之后),则会引发异常。即使它由其父目录中的脚本调用时可以工作,但是如果它认为自己位于程序包的顶层,则它将无法工作。sib2/callsib.pysib2/callsib.pysib1/call.pysib1/call.py

TLDR

This method does not require setuptools, path hacks, additional command line arguments, or specifying the top level of the package in every single file of your project.

Just make a script in the parent directory of whatever your are calling to be your __main__ and run everything from there. For further explanation continue reading.

Explanation

This can be accomplished without hacking a new path together, extra command line args, or adding code to each of your programs to recognize its siblings.

The reason this fails as I believe was mentioned before is the programs being called have their __name__ set as __main__. When this occurs the script being called accepts itself to be on the top level of the package and refuses to recognize scripts in sibling directories.

However, everything under the top level of the directory will still recognize ANYTHING ELSE under the top level. This means the ONLY thing you have to do to get files in sibling directories to recognize/utilize each other is to call them from a script in their parent directory.

Proof of Concept In a dir with the following structure:

.
|__Main.py
|
|__Siblings
   |
   |___sib1
   |   |
   |   |__call.py
   |
   |___sib2
       |
       |__callsib.py

Main.py contains the following code:

import sib1.call as call


def main():
    call.Call()


if __name__ == '__main__':
    main()

sib1/call.py contains:

import sib2.callsib as callsib


def Call():
    callsib.CallSib()


if __name__ == '__main__':
    Call()

and sib2/callsib.py contains:

def CallSib():
    print("Got Called")

if __name__ == '__main__':
    CallSib()

If you reproduce this example you will notice that calling Main.py will result in “Got Called” being printed as is defined in sib2/callsib.py even though sib2/callsib.py got called through sib1/call.py. However if one were to directly call sib1/call.py (after making appropriate changes to the imports) it throws an exception. Even though it worked when called by the script in its parent directory, it will not work if it believes itself to be on the top level of the package.


回答 7

我制作了一个示例项目来演示如何处理此问题,这确实是如上所述的另一个sys.path hack。Python Sibling Import Example,它依赖于:

if __name__ == '__main__': import os import sys sys.path.append(os.getcwd())

只要您的工作目录位于Python项目的根目录下,这似乎就非常有效。如果有人将其部署在实际的生产环境中,那么很高兴听到它是否也可以在此环境中工作。

I made a sample project to demonstrate how I handled this, which is indeed another sys.path hack as indicated above. Python Sibling Import Example, which relies on:

if __name__ == '__main__': import os import sys sys.path.append(os.getcwd())

This seems to be pretty effective so long as your working directory remains at the root of the Python project. If anyone deploys this in a real production environment it’d be great to hear if it works there as well.


回答 8

您需要查看如何在相关代码中编写导入语句。如果examples/example_one.py使用以下导入语句:

import api.api

…然后,它希望项目的根目录位于系统路径中。

最简单的方法来支持此操作(如您所说),就是从顶层目录运行示例,如下所示:

PYTHONPATH=$PYTHONPATH:. python examples/example_one.py 

You need to look to see how the import statements are written in the related code. If examples/example_one.py uses the following import statement:

import api.api

…then it expects the root directory of the project to be in the system path.

The easiest way to support this without any hacks (as you put it) would be to run the examples from the top level directory, like this:

PYTHONPATH=$PYTHONPATH:. python examples/example_one.py 

回答 9

以防万一有人在Eclipse上使用Pydev的情况出现在这里:您可以使用Project-> Properties并在左侧菜单Pydev-PYTHONPATH下设置External Libraries,将同级的父路径(以及调用模块的父路径)添加为外部库文件夹。然后,您可以从同级导入,例如。from sibling import some_class

Just in case someone using Pydev on Eclipse end up here: you can add the sibling’s parent path (and thus the calling module’s parent) as an external library folder using Project->Properties and setting External Libraries under the left menu Pydev-PYTHONPATH. Then you can import from your sibling, e. g. from sibling import some_class.


回答 10

首先,应避免使用与模块本身同名的文件。它可能会破坏其他进口。

导入文件时,首先解释器检查当前目录,然后搜索全局目录。

里面examples或者tests您可以拨打:

from ..api import api

First, you should avoid having files with the same name as the module itself. It may break other imports.

When you import a file, first the interpreter checks the current directory and then searchs global directories.

Inside examples or tests you can call:

from ..api import api