如何在Python中创建命名空间包?

问题:如何在Python中创建命名空间包?

在Python中,命名空间包可让您在多个项目中传播Python代码。当您要将相关的库作为单独的下载发布时,这很有用。例如,目录Package-1Package-2PYTHONPATH

Package-1/namespace/__init__.py
Package-1/namespace/module1/__init__.py
Package-2/namespace/__init__.py
Package-2/namespace/module2/__init__.py

最终用户可以import namespace.module1import namespace.module2

定义命名空间包的最佳方法是什么,以便多个Python产品可以在该命名空间中定义模块?

In Python, a namespace package allows you to spread Python code among several projects. This is useful when you want to release related libraries as separate downloads. For example, with the directories Package-1 and Package-2 in PYTHONPATH,

Package-1/namespace/__init__.py
Package-1/namespace/module1/__init__.py
Package-2/namespace/__init__.py
Package-2/namespace/module2/__init__.py

the end-user can import namespace.module1 and import namespace.module2.

What’s the best way to define a namespace package so more than one Python product can define modules in that namespace?


回答 0

TL; DR:

在Python 3.3上,您无需执行任何操作,只需将任何内容都不放在__init__.py命名空间包目录中就可以了。在3.3之前的版本中,请选择一种pkgutil.extend_path()解决方案pkg_resources.declare_namespace(),因为它是面向未来的并且已经与隐式命名空间包兼容。


Python 3.3引入了隐式命名空间包,请参阅PEP 420

这意味着一个对象现在可以创建三种类型的对象import foo

  • foo.py文件代表的模块
  • 常规软件包,由foo包含__init__.py文件的目录表示
  • 一个命名空间包,由一个或多个目录表示,foo没有任何__init__.py文件

包也是模块,但是当我说“模块”时,我的意思是“非包模块”。

首先,它扫描sys.path模块或常规软件包。如果成功,它将停止搜索并创建并初始化模块或程序包。如果没有找到模块或常规包,但是找到了至少一个目录,它将创建并初始化一个命名空间包。

模块和常规软件包已__file__设置.py为创建它们的文件。常规和命名空间包已__path__设置为创建它们的目录。

完成此操作后import foo.bar,将首先针对进行上述搜索foo,然后如果找到了软件包,bar则使用foo.__path__搜索路径而不是进行搜索sys.path。如果foo.bar找到,foofoo.bar创建和初始化。

那么常规软件包和命名空间软件包如何混合使用?通常它们不会,但是旧的pkgutil显式命名空间包方法已扩展为包括隐式命名空间包。

如果您已有这样的常规软件包__init__.py

from pkgutil import extend_path
__path__ = extend_path(__path__, __name__)

…遗留行为是在搜索到的路径中将其他任何常规软件包添加到其__path__。但是在Python 3.3中,它也添加了命名空间包。

因此,您可以具有以下目录结构:

├── path1
   └── package
       ├── __init__.py
       └── foo.py
├── path2
   └── package
       └── bar.py
└── path3
    └── package
        ├── __init__.py
        └── baz.py

……只要两个__init__.pyextend_path行(和path1path2path3在你的sys.pathimport package.fooimport package.bar并且import package.baz将所有的工作。

pkg_resources.declare_namespace(__name__) 尚未更新为包括隐式命名空间包。

TL;DR:

On Python 3.3 you don’t have to do anything, just don’t put any __init__.py in your namespace package directories and it will just work. On pre-3.3, choose the pkgutil.extend_path() solution over the pkg_resources.declare_namespace() one, because it’s future-proof and already compatible with implicit namespace packages.


Python 3.3 introduces implicit namespace packages, see PEP 420.

This means there are now three types of object that can be created by an import foo:

  • A module represented by a foo.py file
  • A regular package, represented by a directory foo containing an __init__.py file
  • A namespace package, represented by one or more directories foo without any __init__.py files

Packages are modules too, but here I mean “non-package module” when I say “module”.

First it scans sys.path for a module or regular package. If it succeeds, it stops searching and creates and initalizes the module or package. If it found no module or regular package, but it found at least one directory, it creates and initializes a namespace package.

Modules and regular packages have __file__ set to the .py file they were created from. Regular and namespace packages have __path__set to the directory or directories they were created from.

When you do import foo.bar, the above search happens first for foo, then if a package was found, the search for bar is done with foo.__path__as the search path instead of sys.path. If foo.bar is found, foo and foo.bar are created and initialized.

So how do regular packages and namespace packages mix? Normally they don’t, but the old pkgutil explicit namespace package method has been extended to include implicit namespace packages.

If you have an existing regular package that has an __init__.py like this:

from pkgutil import extend_path
__path__ = extend_path(__path__, __name__)

… the legacy behavior is to add any other regular packages on the searched path to its __path__. But in Python 3.3, it also adds namespace packages.

So you can have the following directory structure:

├── path1
│   └── package
│       ├── __init__.py
│       └── foo.py
├── path2
│   └── package
│       └── bar.py
└── path3
    └── package
        ├── __init__.py
        └── baz.py

… and as long as the two __init__.py have the extend_path lines (and path1, path2 and path3 are in your sys.path) import package.foo, import package.bar and import package.baz will all work.

pkg_resources.declare_namespace(__name__) has not been updated to include implicit namespace packages.


回答 1

有一个称为pkgutil的标准模块,您可以使用该模块将模块“附加”到给定的命名空间。

使用目录结构,您可以提供:

Package-1/namespace/__init__.py
Package-1/namespace/module1/__init__.py
Package-2/namespace/__init__.py
Package-2/namespace/module2/__init__.py

你应该把在这两个两行Package-1/namespace/__init__.pyPackage-2/namespace/__init__.py(*):

from pkgutil import extend_path
__path__ = extend_path(__path__, __name__)

(*由于-除非您声明它们之间的依赖关系-您不知道将首先识别其中的哪个- 有关更多信息,请参见PEP 420

文档所述

这将添加到以包命名__path__的目录的所有目录子目录中sys.path

从现在开始,您应该能够独立分发这两个软件包。

There’s a standard module, called pkgutil, with which you can ‘append’ modules to a given namespace.

With the directory structure you’ve provided:

Package-1/namespace/__init__.py
Package-1/namespace/module1/__init__.py
Package-2/namespace/__init__.py
Package-2/namespace/module2/__init__.py

You should put those two lines in both Package-1/namespace/__init__.py and Package-2/namespace/__init__.py (*):

from pkgutil import extend_path
__path__ = extend_path(__path__, __name__)

(* since -unless you state a dependency between them- you don’t know which of them will be recognized first – see PEP 420 for more information)

As the documentation says:

This will add to the package’s __path__ all subdirectories of directories on sys.path named after the package.

From now on, you should be able to distribute those two packages independently.


回答 2

本节应该很不言自明。

简而言之,将命名空间代码放入中__init__.py,进行更新setup.py以声明一个命名空间,您可以随意使用。

This section should be pretty self-explanatory.

In short, put the namespace code in __init__.py, update setup.py to declare a namespace, and you are free to go.


回答 3

这是一个古老的问题,但是最近有人在我的博客上评论说,我有关命名空间包的帖子仍然有意义,因此我想在这里链接到它,因为它提供了如何实现它的实用示例:

https://web.archive.org/web/20150425043954/http://cdent.tumblr.com/post/216241761/python-namespace-packages-for-tiddlyweb

链接到本文以了解发生的主要事情:

http://www.siafoo.net/article/77#multiple-distributions-one-virtual-package

__import__("pkg_resources").declare_namespace(__name__)技巧是相当多的驱动器在插件管理TiddlyWeb和迄今似乎是工作了。

This is an old question, but someone recently commented on my blog that my posting about namespace packages was still relevant, so thought I would link to it here as it provides a practical example of how to make it go:

https://web.archive.org/web/20150425043954/http://cdent.tumblr.com/post/216241761/python-namespace-packages-for-tiddlyweb

That links to this article for the main guts of what’s going on:

http://www.siafoo.net/article/77#multiple-distributions-one-virtual-package

The __import__("pkg_resources").declare_namespace(__name__) trick is pretty much drives the management of plugins in TiddlyWeb and thus far seems to be working out.


回答 4

您已经将Python命名空间概念放到了最前面,在python中无法将包放入模块中。软件包中包含模块,而不是相反。

Python包只是包含__init__.py文件的文件夹。模块是包中(或直接在上PYTHONPATH)具有.py扩展名的任何其他文件。因此,在您的示例中,您有两个包,但未定义任何模块。如果您认为软件包是文件系统文件夹,而模块是文件,那么您会明白为什么软件包包含模块,而不是相反。

因此,在示例中,假设Package-1和Package-2是放在Python路径上的文件系统上的文件夹,则可以具有以下内容:

Package-1/
  namespace/
  __init__.py
  module1.py
Package-2/
  namespace/
  __init__.py
  module2.py

你现在有一个包namespace有两个模块module1module2。除非您有充分的理由,否则您应该将模块放在文件夹中,并且仅将其放在python路径中,如下所示:

Package-1/
  namespace/
  __init__.py
  module1.py
  module2.py

You have your Python namespace concepts back to front, it is not possible in python to put packages into modules. Packages contain modules not the other way around.

A Python package is simply a folder containing a __init__.py file. A module is any other file in a package (or directly on the PYTHONPATH) that has a .py extension. So in your example you have two packages but no modules defined. If you consider that a package is a file system folder and a module is file then you see why packages contain modules and not the other way around.

So in your example assuming Package-1 and Package-2 are folders on the file system that you have put on the Python path you can have the following:

Package-1/
  namespace/
  __init__.py
  module1.py
Package-2/
  namespace/
  __init__.py
  module2.py

You now have one package namespace with two modules module1 and module2. and unless you have a good reason you should probably put the modules in the folder and have only that on the python path like below:

Package-1/
  namespace/
  __init__.py
  module1.py
  module2.py