如何编写setup.py以包含Git存储库作为依赖项

问题:如何编写setup.py以包含Git存储库作为依赖项

我正在尝试setup.py为我的包裹写东西。我的软件包需要指定对另一个Git存储库的依赖。

这是我到目前为止所拥有的:

from setuptools import setup, find_packages

setup(
    name='abc',
    packages=find_packages(),
    url='https://github.abc.com/abc/myabc',
    description='This is a description for abc',
    long_description=open('README.md').read(),
    install_requires=[
        "requests==2.7.0",
        "SomePrivateLib>=0.1.0",
        ],
    dependency_links = [
     "git+git://github.abc.com/abc/SomePrivateLib.git#egg=SomePrivateLib",
    ],
    include_package_data=True,
)

当我跑步时:

pip install -e https://github.abc.com/abc/myabc.git#egg=analyse

我懂了

找不到满足SomePrivateLib> = 0.1.0要求的版本(来自分析)(来自版本:)找不到SomePrivateLib> = 0.1.0(来自分析)的匹配分布

我究竟做错了什么?

I am trying to write setup.py for my package. My package needs to specify a dependency on another git repo.

This is what I have so far:

from setuptools import setup, find_packages

setup(
    name='abc',
    packages=find_packages(),
    url='https://github.abc.com/abc/myabc',
    description='This is a description for abc',
    long_description=open('README.md').read(),
    install_requires=[
        "requests==2.7.0",
        "SomePrivateLib>=0.1.0",
        ],
    dependency_links = [
     "git+git://github.abc.com/abc/SomePrivateLib.git#egg=SomePrivateLib",
    ],
    include_package_data=True,
)

When I run:

pip install -e https://github.abc.com/abc/myabc.git#egg=analyse

I get

Could not find a version that satisfies the requirement SomePrivateLib>=0.1.0 (from analyse) (from versions: ) No matching distribution found for SomePrivateLib>=0.1.0 (from analyse)

What am I doing wrong ?


回答 0

您可以在此处找到正确的方法。

dependency_links=['http://github.com/user/repo/tarball/master#egg=package-1.0']

关键不是提供指向Git存储库的链接,而是提供指向tarball的链接。如果您/tarball/master像上面那样添加,GitHub会为您创建master分支的压缩包。

You can find the right way to do it here.

dependency_links=['http://github.com/user/repo/tarball/master#egg=package-1.0']

The key is not to give a link to a git repository, but a link to a tarball. Github creates a tarball of the master branch for you if you append /tarball/master as shown above.


回答 1

在通过上面的评论中@muon链接的pip问题3939PEP-508规范后,我发现成功地通过setup.py使用以下规范模式来安装我的私有repo依赖项install_requires(不再dependency_links):

install_requires = [
  'some-pkg @ git+ssh://git@github.com/someorgname/pkg-repo-name@v1.1#egg=some-pkg',
]

@v1.1指示在GitHub上创建的发布标志,并可以用一个分支,提交,或不同类型的标签来代替。

After digging through the pip issue 3939 linked by @muon in the comments above and then the PEP-508 specification, I found success getting my private repo dependency to install via setup.py using this specification pattern in install_requires (no more dependency_links):

install_requires = [
  'some-pkg @ git+ssh://git@github.com/someorgname/pkg-repo-name@v1.1#egg=some-pkg',
]

The @v1.1 indicates the release tag created on github and could be replaced with a branch, commit, or different type of tag.


回答 2

以下答案不适用于Pip 19+


不幸的是,其他答案不适用于私有存储库,这是最常见的用例之一。我最终的确使用了如下setup.py文件:

from setuptools import setup, find_packages

setup(
    name = 'MyProject',
    version = '0.1.0',
    url = '',
    description = '',
    packages = find_packages(),
    install_requires = [
        # Github Private Repository - needs entry in `dependency_links`
        'ExampleRepo'
    ],

    dependency_links=[
        # Make sure to include the `#egg` portion so the `install_requires` recognizes the package
        'git+ssh://git@github.com/example_organization/ExampleRepo.git#egg=ExampleRepo-0.1'
    ]
)

较新版本的pip消除了使用“ dependency_links”的需要,从而使操作更加轻松-

from setuptools import setup, find_packages

setup(
    name = 'MyProject',
    version = '0.1.0',
    url = '',
    description = '',
    packages = find_packages(),
    install_requires = [
        # Github Private Repository
        'ExampleRepo @ git+ssh://git@github.com/example_organization/ExampleRepo.git#egg=ExampleRepo-0.1'
    ]
)

The following answer is deprecated for Pip 19+


Unfortunately the other answer does not work with private repositories, which is one of the most common use cases for this. I eventually did get it working with a setup.py file that looks like this:

from setuptools import setup, find_packages

setup(
    name = 'MyProject',
    version = '0.1.0',
    url = '',
    description = '',
    packages = find_packages(),
    install_requires = [
        # Github Private Repository - needs entry in `dependency_links`
        'ExampleRepo'
    ],

    dependency_links=[
        # Make sure to include the `#egg` portion so the `install_requires` recognizes the package
        'git+ssh://git@github.com/example_organization/ExampleRepo.git#egg=ExampleRepo-0.1'
    ]
)

Newer versions of pip make this even easier by removing the need to use “dependency_links”-

from setuptools import setup, find_packages

setup(
    name = 'MyProject',
    version = '0.1.0',
    url = '',
    description = '',
    packages = find_packages(),
    install_requires = [
        # Github Private Repository
        'ExampleRepo @ git+ssh://git@github.com/example_organization/ExampleRepo.git#egg=ExampleRepo-0.1'
    ]
)

回答 3

一个更一般的答案:要从requirements.txt文件中获取信息,请执行以下操作:

from setuptools import setup, find_packages
from os import path

loc = path.abspath(path.dirname(__file__))

with open(loc + '/requirements.txt') as f:
    requirements = f.read().splitlines()

required = []
dependency_links = []

# Do not add to required lines pointing to Git repositories
EGG_MARK = '#egg='
for line in requirements:
    if line.startswith('-e git:') or line.startswith('-e git+') or \
            line.startswith('git:') or line.startswith('git+'):
        if EGG_MARK in line:
            package_name = line[line.find(EGG_MARK) + len(EGG_MARK):]
            required.append(package_name)
            dependency_links.append(line)
        else:
            print('Dependency to a git repository should have the format:')
            print('git+ssh://git@github.com/xxxxx/xxxxxx#egg=package_name')
    else:
        required.append(line)

setup(
    name='myproject',  # Required
    version='0.0.1',  # Required
    description='Description here....',  # Required
    packages=find_packages(),  # Required
    install_requires=required,
    dependency_links=dependency_links,
)

A more general answer, to get the information from the requeriments.txt file I do:

from setuptools import setup, find_packages
from os import path

loc = path.abspath(path.dirname(__file__))

with open(loc + '/requirements.txt') as f:
    requirements = f.read().splitlines()

required = []
dependency_links = []
# do not add to required lines pointing to git repositories
EGG_MARK = '#egg='
for line in requirements:
    if line.startswith('-e git:') or line.startswith('-e git+') or \
            line.startswith('git:') or line.startswith('git+'):
        if EGG_MARK in line:
            package_name = line[line.find(EGG_MARK) + len(EGG_MARK):]
            required.append(package_name)
            dependency_links.append(line)
        else:
            print('Dependency to a git repository should have the format:')
            print('git+ssh://git@github.com/xxxxx/xxxxxx#egg=package_name')
    else:
        required.append(line)

setup(
    name='myproject',  # Required
    version='0.0.1',  # Required
    description='Description here....',  # Required
    packages=find_packages(),  # Required
    install_requires=required,
    dependency_links=dependency_links,
) 

回答 4

实际上,如果您希望递归安装软件包(YourCurrentPackage包含SomePrivateLib),例如,当您希望将YourCurrentPackage包含到另一个软件包中(例如OuterPackage→YourCurrentPackage→SomePrivateLib)时,您将同时需要:

install_requires=[
    ...,
    "SomePrivateLib @ git+ssh://github.abc.com/abc/SomePrivateLib.git@0.1.0#egg=SomePrivateLib"
],
dependency_links = [
    "git+ssh://github.abc.com/abc/SomePrivateLib.git@0.1.0#egg=SomePrivateLib"
]

并确保您使用版本号创建了一个标签。

同样,如果您的Git项目是私有的,并且您想将其安装在容器中(例如DockerGitLab运行器),那么您将需要对存储库的授权访问。请考虑将Git + HTTPS与访问令牌一起使用(例如在GitLab上:https : //docs.gitlab.com/ee/user/profile/personal_access_tokens.html ):

import os
from setuptools import setup

TOKEN_VALUE = os.getenv('EXPORTED_VAR_WITH_TOKEN')

setup(
    ....

    install_requires=[
            ...,
            f"SomePrivateLib @ git+https://gitlab-ci-token:{TOKEN_VALUE}@gitlab.server.com/abc/SomePrivateLib.git@0.1.0#egg=SomePrivateLib"
    ],
    dependency_links = [
            f"git+https://gitlab-ci-token:{TOKEN_VALUE}@gitlab.server.com/abc/SomePrivateLib.git@0.1.0#egg=SomePrivateLib"
    ]
)

Actually if you like to make your packages installable recursivelly (YourCurrentPackage includes your SomePrivateLib), e.g. when you want to include YourCurrentPackage into another one (like OuterPackage -> YourCurrentPackage -> SomePrivateLib) you’ll need both:

install_requires=[
    ...,
    "SomePrivateLib @ git+ssh://github.abc.com/abc/SomePrivateLib.git@0.1.0#egg=SomePrivateLib"
],
dependency_links = [
    "git+ssh://github.abc.com/abc/SomePrivateLib.git@0.1.0#egg=SomePrivateLib"
]

And make sure you have a tag created with your version number.

Also if your git project is private and you like to install it inside the container e.g. Docker or GitLab runner, you will need authorized access to your repo. Please consider to use git+https with access tokens (like on GitLab: https://docs.gitlab.com/ee/user/profile/personal_access_tokens.html):

import os
from setuptools import setup

TOKEN_VALUE = os.getenv('EXPORTED_VAR_WITH_TOKEN')

setup(
    ....

    install_requires=[
            ...,
            f"SomePrivateLib @ git+https://gitlab-ci-token:{TOKEN_VALUE}@gitlab.server.com/abc/SomePrivateLib.git@0.1.0#egg=SomePrivateLib"
    ],
    dependency_links = [
            f"git+https://gitlab-ci-token:{TOKEN_VALUE}@gitlab.server.com/abc/SomePrivateLib.git@0.1.0#egg=SomePrivateLib"
    ]
)

回答 5

我在GitLab中成功使用了这三个选项。我正在使用GitLab版本11。

选项1-未指定令牌。Shell将提示输入用户名/密码。

from setuptools import setup

TOKEN_VALUE = os.getenv('EXPORTED_VAR_WITH_TOKEN')

setup(
    install_requires=[
        "SomePrivateLib @ git+https://gitlab.server.com/abc/SomePrivateLib.git@0.1.0#egg=SomePrivateLib"
    ]
)

选项2-指定了用户访问令牌。通过转到GitLab→帐户右上方→设置→访问令牌生成的令牌。创建具有read_repository权限的令牌。

例:

import os
from setuptools import setup

TOKEN_VALUE = os.getenv('EXPORTED_VAR_WITH_TOKEN')

setup(
    install_requires=[
        f"SomePrivateLib @ git+https://gitlab-ci-token:{TOKEN_VALUE}@gitlab.server.com/abc/SomePrivateLib.git@0.1.0#egg=SomePrivateLib"
    ]
)

选项3-指定存储库级别的令牌。通过转到存储库→设置→存储库→部署令牌生成的令牌。在这里,创建一个具有read_repository权限的令牌。

例:

import os
from setuptools import setup

TOKEN_USER = os.getenv('EXPORTED_TOKEN_USER')
TOKEN_VALUE = os.getenv('EXPORTED_VAR_WITH_TOKEN')

setup(
    install_requires=[
        f"SomePrivateLib @ git+https://{TOKEN_USER}:{TOKEN_VALUE}@gitlab.server.com/abc/SomePrivateLib.git@0.1.0#egg=SomePrivateLib"
    ]
)

在这三者中,我都可以简单地做:“ SomePrivateLib @ git + https://gitlab.server.com/abc/SomePrivateLib.git”,最后没有#egg标记。

I was successful with these 3 options in gitlab. I am using version 11 of gitlab.

option 1 – no token specified. shell will prompt for username/password.

from setuptools import setup

TOKEN_VALUE = os.getenv('EXPORTED_VAR_WITH_TOKEN')

setup(
    install_requires=[
        "SomePrivateLib @ git+https://gitlab.server.com/abc/SomePrivateLib.git@0.1.0#egg=SomePrivateLib"
    ]
)

option 2 – user access token specified. token generated by going to gitlab > account top right > settings > access tokens. create token with read_repository rights.

example:

import os
from setuptools import setup

TOKEN_VALUE = os.getenv('EXPORTED_VAR_WITH_TOKEN')

setup(
    install_requires=[
        f"SomePrivateLib @ git+https://gitlab-ci-token:{TOKEN_VALUE}@gitlab.server.com/abc/SomePrivateLib.git@0.1.0#egg=SomePrivateLib"
    ]
)

option 3 – repository-level token specified. token generated by going to the repository > settings > repository > deploy tokens. from here create a token with read_repository rights.

example:

import os
from setuptools import setup

TOKEN_USER = os.getenv('EXPORTED_TOKEN_USER')
TOKEN_VALUE = os.getenv('EXPORTED_VAR_WITH_TOKEN')

setup(
    install_requires=[
        f"SomePrivateLib @ git+https://{TOKEN_USER}:{TOKEN_VALUE}@gitlab.server.com/abc/SomePrivateLib.git@0.1.0#egg=SomePrivateLib"
    ]
)

In all 3, I was able to do simply: “SomePrivateLib @ git+https://gitlab.server.com/abc/SomePrivateLib.git” without the #egg marking at the end.