将Python搜索路径扩展到其他来源

问题:将Python搜索路径扩展到其他来源

我刚刚加入了一个具有相当大的现有代码库的项目。我们用linux开发,不使用和IDE。我们通过命令行运行。我试图弄清楚如何在运行项目模块时让python搜索正确的路径。例如,当我运行类似的内容时:

python someprojectfile.py

我懂了

ImportError: no module named core.'somemodule'

我认为所有导入的内容都是这样,我认为这是路径问题。

TLDR:

如何~/codez/project/在导入语句期间让Python搜索* .py文件以及所有文件和文件夹。

I have just joined a project with a rather large existing code base. We develop in linux and do not use and IDE. We run through the command line. I’m trying to figure out how to get python to search for the right path when I run project modules. For instance, when I run something like:

python someprojectfile.py

I get

ImportError: no module named core.'somemodule'

I get this for all of my imports to I assume it’s an issue with the path.

TLDR:

How do I get Python to search ~/codez/project/ and all the files and folders for *.py files during import statements.


回答 0

有几种方法可以做到这一点:

  • 将环境变量设置PYTHONPATH为用冒号分隔的目录列表,以搜索导入的模块。
  • 在您的程序中,用于sys.path.append('/path/to/search')添加您希望Python搜索导入的模块的目录名称。sys.path只是Python在每次被要求导入模块时搜索的目录列表,您可以根据需要进行更改(尽管我不建议删除任何标准目录!)。Python启动时,您放入环境变量中的所有目录PYTHONPATH都将插入sys.path其中。
  • 用于site.addsitedir向添加目录sys.path。此和纯附加的区别在于,当您使用时addsitedir,它还会.pth在该目录中查找文件,并使用它们sys.path根据文件的内容向其中添加其他目录。有关更多详细信息,请参见文档。

您要使用哪一种取决于您的情况。请记住,当您将项目分发给其他用户时,他们通常会以某种方式安装该项目,以便Python的导入程序会自动检测到Python代码文件(即,程序包通常安装在site-packages目录中),因此,如果您弄乱了sys.path代码,当该代码在另一台计算机上运行时,可能是不必要的,甚至可能产生不利影响。对于开发而言,我会大胆猜测设置PYTHONPATH通常是最好的选择。

但是,当您使用仅在自己的计算机上运行的内容时(或当您进行非标准设置时(例如有时在Web应用程序框架中)),执行诸如此类的操作并非完全不常见。

import sys
from os.path import dirname
sys.path.append(dirname(__file__))

There are a few possible ways to do this:

  • Set the environment variable PYTHONPATH to a colon-separated list of directories to search for imported modules.
  • In your program, use sys.path.append('/path/to/search') to add the names of directories you want Python to search for imported modules. sys.path is just the list of directories Python searches every time it gets asked to import a module, and you can alter it as needed (although I wouldn’t recommend removing any of the standard directories!). Any directories you put in the environment variable PYTHONPATH will be inserted into sys.path when Python starts up.
  • Use site.addsitedir to add a directory to sys.path. The difference between this and just plain appending is that when you use addsitedir, it also looks for .pth files within that directory and uses them to possibly add additional directories to sys.path based on the contents of the files. See the documentation for more detail.

Which one of these you want to use depends on your situation. Remember that when you distribute your project to other users, they typically install it in such a manner that the Python code files will be automatically detected by Python’s importer (i.e. packages are usually installed in the site-packages directory), so if you mess with sys.path in your code, that may be unnecessary and might even have adverse effects when that code runs on another computer. For development, I would venture a guess that setting PYTHONPATH is usually the best way to go.

However, when you’re using something that just runs on your own computer (or when you have nonstandard setups, e.g. sometimes in web app frameworks), it’s not entirely uncommon to do something like

import sys
from os.path import dirname
sys.path.append(dirname(__file__))

回答 1

您还应该在这里阅读有关python软件包的信息:http : //docs.python.org/tutorial/modules.html

从您的示例中,我想您确实在拥有一个软件包~/codez/project__init__.pypython目录中的文件将目录映射到命名空间。如果所有子目录都有一个__init__.py文件,则只需将基本目录添加到中PYTHONPATH。例如:

PYTHONPATH = $ PYTHONPATH:$ HOME / adaifotis / project

正如David解释的那样,除了测试您的PYTHONPATH环境变量外,您还可以像这样在python中对其进行测试:

$ python
>>> import project                      # should work if PYTHONPATH set
>>> import sys
>>> for line in sys.path: print line    # print current python path

You should also read about python packages here: http://docs.python.org/tutorial/modules.html.

From your example, I would guess that you really have a package at ~/codez/project. The file __init__.py in a python directory maps a directory into a namespace. If your subdirectories all have an __init__.py file, then you only need to add the base directory to your PYTHONPATH. For example:

PYTHONPATH=$PYTHONPATH:$HOME/adaifotis/project

In addition to testing your PYTHONPATH environment variable, as David explains, you can test it in python like this:

$ python
>>> import project                      # should work if PYTHONPATH set
>>> import sys
>>> for line in sys.path: print line    # print current python path


回答 2

我知道这个线程有些陈旧,但是花了我一些时间才能真正理解这一点,所以我想分享一下。

在我的项目中,我的主脚本位于父目录中,为了区分模块,我将所有支持的模块放在一个称为“模块”的子文件夹中。在我的主脚本中,我像这样导入这些模块(对于名为report.py的模块):

from modules.report import report, reportError

如果我调用主脚本,则可以正常工作。但是,我想通过main()在每个模块中包含一个并直接调用每个模块来测试每个模块,如下所示:

python modules/report.py

现在,Python抱怨它找不到“称为模块的模块”。这里的关键是,默认情况下,Python在其搜索路径中包括脚本的文件夹,但不是CWD。因此,此错误实际上表示“我找不到模块子文件夹”。这是因为report.py模块所在的目录中没有“ modules”子目录。

我发现最巧妙的解决方案是通过在顶部包括以下内容来将CWD附加到Python搜索路径中:

import sys

sys.path.append(".")

现在,Python搜索CWD(当前目录),找到“模块”子文件夹,一切顺利。

I know this thread is a bit old, but it took me some time to get to the heart of this, so I wanted to share.

In my project, I had the main script in a parent directory, and, to differentiate the modules, I put all the supporting modules in a sub-folder called “modules”. In my main script, I import these modules like this (for a module called report.py):

from modules.report import report, reportError

If I call my main script, this works. HOWEVER, I wanted to test each module by including a main() in each, and calling each directly, as:

python modules/report.py

Now Python complains that it can’t find “a module called modules”. The key here is that, by default, Python includes the folder of the script in its search path, BUT NOT THE CWD. So what this error says, really, is “I can’t find a modules subfolder”. The is because there is no “modules” subdirectory from the directory where the report.py module resides.

I find that the neatest solution to this is to append the CWD in Python search path by including this at the top:

import sys

sys.path.append(".")

Now Python searches the CWD (current directory), finds the “modules” sub-folder, and all is well.


回答 3

我阅读了此问题以寻找答案,但不喜欢其中任何一个。

所以我写了一个快速而肮脏的解决方案。只需将其放在sys.path上的某个位置,它将在folder(来自当前工作目录)或下添加任何目录abspath

#using.py

import sys, os.path

def all_from(folder='', abspath=None):
    """add all dirs under `folder` to sys.path if any .py files are found.
    Use an abspath if you'd rather do it that way.

    Uses the current working directory as the location of using.py. 
    Keep in mind that os.walk goes *all the way* down the directory tree.
    With that, try not to use this on something too close to '/'

    """
    add = set(sys.path)
    if abspath is None:
        cwd = os.path.abspath(os.path.curdir)
        abspath = os.path.join(cwd, folder)
    for root, dirs, files in os.walk(abspath):
        for f in files:
            if f[-3:] in '.py':
                add.add(root)
                break
    for i in add: sys.path.append(i)

>>> import using, sys, pprint
>>> using.all_from('py') #if in ~, /home/user/py/
>>> pprint.pprint(sys.path)
[
#that was easy
]

我之所以喜欢它,是因为我可以为一些随机工具提供一个文件夹,而不必让它们成为软件包或任何东西的一部分,并且仍然可以通过几行代码访问其中的一些(或全部)。

I read this question looking for an answer, and didn’t like any of them.

So I wrote a quick and dirty solution. Just put this somewhere on your sys.path, and it’ll add any directory under folder (from the current working directory), or under abspath:

#using.py

import sys, os.path

def all_from(folder='', abspath=None):
    """add all dirs under `folder` to sys.path if any .py files are found.
    Use an abspath if you'd rather do it that way.

    Uses the current working directory as the location of using.py. 
    Keep in mind that os.walk goes *all the way* down the directory tree.
    With that, try not to use this on something too close to '/'

    """
    add = set(sys.path)
    if abspath is None:
        cwd = os.path.abspath(os.path.curdir)
        abspath = os.path.join(cwd, folder)
    for root, dirs, files in os.walk(abspath):
        for f in files:
            if f[-3:] in '.py':
                add.add(root)
                break
    for i in add: sys.path.append(i)

>>> import using, sys, pprint
>>> using.all_from('py') #if in ~, /home/user/py/
>>> pprint.pprint(sys.path)
[
#that was easy
]

And I like it because I can have a folder for some random tools and not have them be a part of packages or anything, and still get access to some (or all) of them in a couple lines of code.


回答 4

我发现最简单的方法是创建一个文件“ any_name.pth”,并将其放在文件夹“ \ Lib \ site-packages”中。您应该在安装python的任何位置都找到该文件夹​​。

在该文件中,放入要保留要导入模块的目录列表。例如,在该文件中这样一行:

C:\ Users \ example … \ example

您可以通过在python中运行它来告诉它工作:

import sys
for line in sys: print line

您将看到已打印的目录,以及从中可以导入的目录。现在,您可以轻松地导入该目录中的“ mymodule.py”文件:

import mymodule

这将不会导入子文件夹。为此,您可以想象创建一个python脚本来创建一个.pth文件,其中包含您定义的文件夹的所有子文件夹。让它在启动时运行。

The easiest way I find is to create a file “any_name.pth” and put it in your folder “\Lib\site-packages”. You should find that folder wherever python is installed.

In that file, put a list of directories where you want to keep modules for importing. For instance, make a line in that file like this:

C:\Users\example…\example

You will be able to tell it works by running this in python:

import sys
for line in sys: print line

You will see your directory printed out, amongst others from where you can also import. Now you can import a “mymodule.py” file that sits in that directory as easily as:

import mymodule

This will not import subfolders. For that you could imagine creating a python script to create a .pth file containing all sub folders of a folder you define. Have it run at startup perhaps.


回答 5

旧问题的新选择。在Debian上
安装fail2ban软件包,看起来像是硬编码安装/usr/lib/python3/dist-packages/fail2ban在python3以外的路径上sys.path


> python3
Python 3.7.3 (v3.7.3:ef4ec6ed12, Jun 25 2019, 18:51:50)
[GCC 6.3.0 20170516] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>> sys.path
['', '/usr/lib/python37.zip', '/usr/lib/python3.7', '/usr/lib/python3.7/lib-dynload', '/usr/lib/python3.7/site-packages']
>>>

因此,我(bash)将库链接到较新的版本,而不仅仅是复制。
将来对原始应用程序的更新也将自动应用于链接版本。

 if [ -d /usr/lib/python3/dist-packages/fail2ban ]
   then
      for d in /usr/lib/python3.*
      do
         [ -d ${d}/fail2ban ] || \
            ln -vs /usr/lib/python3/dist-packages/fail2ban ${d}/
      done
   fi

New option for old question.
Installing fail2ban package on Debian, looks like it’s hardcoded to install on /usr/lib/python3/dist-packages/fail2ban a path not on python3 sys.path.


> python3
Python 3.7.3 (v3.7.3:ef4ec6ed12, Jun 25 2019, 18:51:50)
[GCC 6.3.0 20170516] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>> sys.path
['', '/usr/lib/python37.zip', '/usr/lib/python3.7', '/usr/lib/python3.7/lib-dynload', '/usr/lib/python3.7/site-packages']
>>>

so, instead of just copying, I (bash) linked the library to newer versions.
Future updates to the original app, will also be automatically applied to the linked versions.

 if [ -d /usr/lib/python3/dist-packages/fail2ban ]
   then
      for d in /usr/lib/python3.*
      do
         [ -d ${d}/fail2ban ] || \
            ln -vs /usr/lib/python3/dist-packages/fail2ban ${d}/
      done
   fi