如何在Anaconda(Conda)环境中跟踪点子安装的软件包?

问题:如何在Anaconda(Conda)环境中跟踪点子安装的软件包?

我已经安装并一直在使用Anaconda Python发行版,并且已经开始使用Anaconda(Conda)环境。我可以使用标准conda install...命令将分发包中的软件包放入我的环境中,但要使用其他pip install在活动环境中需要使用的东西(例如Flask-WTF,flask-sqlalchemy和alembic)。但是,当我查看环境的内容时,无论是在目录中还是使用conda list这些pip installed包,都不会显示。

使用pip freezepip list仅列出我曾经安装的每个软件包。

有没有办法跟踪我的每个Anaconda env(安装pipconda安装的)中的物品?

I’ve installed and have been using the Anaconda Python distribution, and I have started using the Anaconda (Conda) environment. I can use the standard conda install... command to put packages from the distribution into my environments, but to use anything outside (i.e. Flask-WTF, flask-sqlalchemy, and alembic) I need to use pip install in the active environment. However, when I look at the contents of the environment, either in the directory, or using conda list these pip installed packages don’t show up.

Using pip freeze and pip list just lists every package I’ve ever installed.

Is there a way to keep track of what is in each of my Anaconda envs (both pip and conda installed)?


回答 0

conda-env现在可以自动执行此操作(如果pip与conda一起安装)。

您可以使用用于迁移环境的导出工具来查看其工作原理:

conda env export -n <env-name> > environment.yml

该文件将列出conda软件包和pip软件包:

name: stats
channels:
  - javascript
dependencies:
  - python=3.4
  - bokeh=0.9.2
  - numpy=1.9.*
  - nodejs=0.10.*
  - flask
  - pip:
    - Flask-Testing

如果您希望继续导出环境,请移至environment.yml新的主机并运行:

conda env create -f path/to/environment.yml

conda-env now does this automatically (if pip was installed with conda).

You can see how this works by using the export tool used for migrating an environment:

conda env export -n <env-name> > environment.yml

The file will list both conda packages and pip packages:

name: stats
channels:
  - javascript
dependencies:
  - python=3.4
  - bokeh=0.9.2
  - numpy=1.9.*
  - nodejs=0.10.*
  - flask
  - pip:
    - Flask-Testing

If you’re looking to follow through with exporting the environment, move environment.yml to the new host machine and run:

conda env create -f path/to/environment.yml

回答 1

conda将仅跟踪其安装的软件包。并pip会为您提供使用pip安装程序本身安装的软件包,或setuptoolssetup.pyconda构建中使用的软件包生成的egg信息。因此,您基本上有三个选择。

  1. 你可以采取的联合conda listpip freeze和管理使用安装的软件包conda(在这个节目conda list用)conda包管理器,并确保安装与个位pip(即秀pip freeze,但不是在conda list)与pip

  2. 仅在和环境中安装pythonpip然后使用distribute打包和管理所有内容pip。(如果您使用的是Windows,那么这并不是一件容易的事…)

  3. 构建自己的conda软件包,并使用进行管理conda

我个人推荐第三个选项,因为它很容易构建conda软件包。在连续体的github帐户上有一个示例食谱的git存储库。但通常可以归结为:

 conda skeleton pypi PACKAGE
 conda build PACKAGE

要不就:

conda pipbuild PACKAGE

同样,一旦构建它们,您就可以将它们上传到https://binstar.org/并从那里安装。

然后,您将使用进行所有管理conda

conda will only keep track of the packages it installed. And pip will give you the packages that were either installed using the pip installer itself or they used setuptools in their setup.py so conda build generated the egg information. So you have basically three options.

  1. You can take the union of the conda list and pip freeze and manage packages that were installed using conda (that show in the conda list) with the conda package manager and the ones that are installed with pip (that show in pip freeze but not in conda list) with pip.

  2. Install in your environment only the python, pip and distribute packages and manage everything with pip. (This is not that trivial if you’re on Windows…)

  3. Build your own conda packages, and manage everything with conda.

I would personally recommend the third option since it’s very easy to build conda packages. There is a git repository of example recipes on the continuum’s github account. But it usually boils down to:

 conda skeleton pypi PACKAGE
 conda build PACKAGE

or just:

conda pipbuild PACKAGE

Also when you have built them once, you can upload them to https://binstar.org/ and just install from there.

Then you’ll have everything managed using conda.


回答 2

有一个conda(new-pypi-install)分支,它可以更好地与pip和PyPI集成。特别是conda列表还将显示pip安装的软件包,conda安装将首先尝试查找conda软件包,如果失败,将使用pip安装软件包。

该分支计划于本周晚些时候合并,以便conda 2.1版将更好地与conda进行点集成。

There is a branch of conda (new-pypi-install) that adds better integration with pip and PyPI. In particular conda list will also show pip installed packages and conda install will first try to find a conda package and failing that will use pip to install the package.

This branch is scheduled to be merged later this week so that version 2.1 of conda will have better pip-integration with conda.


回答 3

我遵循@Viktor Kerkez的回答,并取得了不同的成功。我发现有时候这个食谱

康达骨架pypi包装

康达构建包

看起来一切正常,但我无法成功导入PACKAGE。最近,我在Anaconda用户组上询问了此问题,并从@Travis Oliphant自己那里听说了使用conda构建和管理Anaconda附带的软件包的最佳方法。您可以在此处阅读此主题,但我将在下面描述该方法,以期使OP的问题的答案更加完整…

示例:我将使用conda 2.2.5在Windows上安装出色的prettyplotlib软件包

1a) conda build --build-recipe prettyplotlib

在构建的最后“测试”部分之前,您将看到构建消息看起来都不错。我看到了这个错误

文件“ C:\ Anaconda \ conda-bld \ test-tmp_dir \ run_test.py”,第23行导入None语法错误:无法分配为None测试失败:prettyplotlib-0.1.3-py27_0

1b)进入/ conda-recipes / prettyplotlib并编辑meta.yaml文件。当前,像在步骤1a中一样设置的软件包会导致yaml文件在该test部分中出错。例如,这是我的寻找方式prettyplotlib

test:   # Python imports   imports:
    - 
    - prettyplotlib
    - prettyplotlib

编辑此部分以删除-开头的空白行,并删除多余的prettyplotlib行。在撰写本文时,我发现我需要为使用conda安装的外部软件包编辑这样的大多数meta.yaml文件,这意味着有空白的导入行会导致错误以及给定软件包的冗余导入。

1c)从1a重新运行命令,这一次应该正确完成。在构建结束时,系统会询问您是否要将构建上传到binstar。我输入“否”,然后看到以下消息:

如果您想稍后将此包上传到binstar.org,请键入:

$ binstar上传C:\ Anaconda \ conda-bld \ win-64 \ prettyplotlib-0.1.3-py27_0.tar.bz2

该tar.bz2文件是您现在实际需要安装的构建。

2) conda install C:\Anaconda\conda-bld\win-64\prettyplotlib-0.1.3-py27_0.tar.bz2

按照这些步骤,我已经成功地使用conda安装了Anaconda随附的许多软件包。以前,我已经使用pip安装了其中的一些工具,因此pip uninstall PACKAGE在使用conda安装PACKAGE之前,我已经完成了安装。使用conda,我现在可以通过一种方法来管理(几乎)所有软件包,而无需将conda,pip,easy_install和python setup.py安装混合在一起。

对于上下文,我认为@Travis Oliphant的最新博客文章将对像我这样的人有所帮助,他们不喜欢使用健壮的Python包装的所有内容,但可以肯定当东西“起作用时”。康达似乎是前进的好方法…

I followed @Viktor Kerkez’s answer and have had mixed success. I found that sometimes this recipe of

conda skeleton pypi PACKAGE

conda build PACKAGE

would look like everything worked but I could not successfully import PACKAGE. Recently I asked about this on the Anaconda user group and heard from @Travis Oliphant himself on the best way to use conda to build and manage packages that do not ship with Anaconda. You can read this thread here, but I’ll describe the approach below to hopefully make the answers to the OP’s question more complete…

Example: I am going to install the excellent prettyplotlib package on Windows using conda 2.2.5.

1a) conda build --build-recipe prettyplotlib

You’ll see the build messages all look good until the final TEST section of the build. I saw this error

File “C:\Anaconda\conda-bld\test-tmp_dir\run_test.py”, line 23 import None SyntaxError: cannot assign to None TESTS FAILED: prettyplotlib-0.1.3-py27_0

1b) Go into /conda-recipes/prettyplotlib and edit the meta.yaml file. Presently, the packages being set up like in step 1a result in yaml files that have an error in the test section. For example, here is how mine looked for prettyplotlib

test:   # Python imports   imports:
    - 
    - prettyplotlib
    - prettyplotlib

Edit this section to remove the blank line preceded by the – and also remove the redundant prettyplotlib line. At the time of this writing I have found that I need to edit most meta.yaml files like this for external packages I am installing with conda, meaning that there is a blank import line causing the error along with a redundant import of the given package.

1c) Rerun the command from 1a, which should complete with out error this time. At the end of the build you’ll be asked if you want to upload the build to binstar. I entered No and then saw this message:

If you want to upload this package to binstar.org later, type:

$ binstar upload C:\Anaconda\conda-bld\win-64\prettyplotlib-0.1.3-py27_0.tar.bz2

That tar.bz2 file is the build that you now need to actually install.

2) conda install C:\Anaconda\conda-bld\win-64\prettyplotlib-0.1.3-py27_0.tar.bz2

Following these steps I have successfully used conda to install a number of packages that do not come with Anaconda. Previously, I had installed some of these using pip, so I did pip uninstall PACKAGE prior to installing PACKAGE with conda. Using conda, I can now manage (almost) all of my packages with a single approach rather than having a mix of stuff installed with conda, pip, easy_install, and python setup.py install.

For context, I think this recent blog post by @Travis Oliphant will be helpful for people like me who do not appreciate everything that goes into robust Python packaging but certainly appreciate when stuff “just works”. conda seems like a great way forward…


回答 4

这就是为什么我写Picky的原因:http : //picky.readthedocs.io/

这是一个python软件包,可跟踪在virtualenvs和conda envs中随pip或conda安装的软件包。

This is why I wrote Picky: http://picky.readthedocs.io/

It’s a python package that tracks packages installed with either pip or conda in either virtualenvs and conda envs.


回答 5

我认为这里缺少的是:

>pip install .

要使用setup.py安装本地软件包,它将安装一个对使用相同版本 python的所有 conda env 可见的软件包。注意我正在使用conda版本的pip!

例如,如果我使用的是python2.7,它将本地包放在此处:

/usr/local/anaconda/lib/python2.7/site-packages

如果以后再用python = 2.7(=默认值)创建一个新的conda env:

>conda create --name new

>source activate new

然后执行:

(new)>conda list    // empty - conda is not aware of any packages yet

但是,如果我这样做:

(new)>pip list      // the local package installed above is present

因此,在这种情况下,conda不了解pip包,但是python可以使用该包。

但是,如果我不是安装在本地包(再次使用PIP)后,我已经创建(激活)新康达ENV,现在畅达看到它:

(new)>conda list   // sees that the package is there and was installed by pip

因此,我认为conda和pip之间的交互存在一些问题-即,使用pip从一个conda env中安装本地软件包会使该软件包可用于同一python版本的所有其他conda env(但不能通过conda列表看到)。

I think what’s missing here is that when you do:

>pip install .

to install a local package with a setup.py, it installs a package that is visible to all the conda envs that use the same version of python. Note I am using the conda version of pip!

e.g., if I’m using python2.7 it puts the local package here:

/usr/local/anaconda/lib/python2.7/site-packages

If I then later create a new conda env with python=2.7 (= the default):

>conda create --name new

>source activate new

And then do:

(new)>conda list    // empty - conda is not aware of any packages yet

However, if I do:

(new)>pip list      // the local package installed above is present

So in this case, conda does not know about the pip package, but the package is available to python.

However, If I instead install the local package (again using pip) after I’ve created (and activated) the new conda env, now conda sees it:

(new)>conda list   // sees that the package is there and was installed by pip

So I think the interaction between conda and pip has some issues – ie, using pip to install a local package from within one conda env makes that package available (but not seen via conda list) to all other conda envs of the same python version.


回答 6

conda env export列出环境中的所有conda和pip软件包。conda-env必须安装在conda根目录(conda install -c conda conda-env)中。

编写environment.yml描述当前环境的文件:

conda env export > environment.yml

参考文献:

conda env export lists all conda and pip packages in an environment. conda-env must be installed in the conda root (conda install -c conda conda-env).

To write an environment.yml file describing the current environment:

conda env export > environment.yml

References:


回答 7

我通常会在“ pip”命令之前为要安装软件包的特定环境添加“ bin / pip”文件夹前缀。例如,如果要在环境py34中安装pymc3,则应使用以下命令:

~/anaconda/envs/py34/bin/pip install git+https://github.com/pymc-devs/pymc3 

基本上,您只需要找到环境“ bin / pip”文件夹的正确路径,并将其放在install命令之前即可。

I usually prefix the ‘bin/pip’ folder for the specific environment you want to install the package before the ‘pip’ command. For instance, if you would like to install pymc3 in the environment py34, you should use this command:

~/anaconda/envs/py34/bin/pip install git+https://github.com/pymc-devs/pymc3 

You basically just need to find the right path to your environment ‘bin/pip’ folder and put it before the install command.


回答 8

which pip显示了以下路径:

$ which pip
/home/kmario23/anaconda3/bin/pip

因此,pip install <package-name>使用以下命令导出列表时,无论我安装使用的任何软件包,都必须反映在软件包列表中:

$ conda list --export > conda_list.txt

但是,我没有。因此,我改用了其他一些人建议的以下命令:

# get environment name by
$ conda-env list

# get list of all installed packages by (conda, pip, etc.,)
$ conda-env export -n <my-environment-name> > all_packages.yml
# if you haven't created any specific env, then just use 'root'

现在,我可以看到all-packages.yml文件中的所有软件包。

My which pip shows the following path:

$ which pip
/home/kmario23/anaconda3/bin/pip

So, whatever package I install using pip install <package-name> will have to be reflected in the list of packages when the list is exported using:

$ conda list --export > conda_list.txt

But, I don’t. So, instead I used the following command as suggested by several others:

# get environment name by
$ conda-env list

# get list of all installed packages by (conda, pip, etc.,)
$ conda-env export -n <my-environment-name> > all_packages.yml
# if you haven't created any specific env, then just use 'root'

Now, I can see all the packages in my all-packages.yml file.


回答 9

您可以先在conda环境中安装以下给定命令开始:

conda install pip

然后安装环境中所需的所有pip软件包。

安装所有conda和pip软件包以导出环境后,请使用:

conda env export -n <env-name> > environment.yml

这将在文件夹中创建所需的文件

You can start by installing the below given command in the conda environment:

conda install pip

Followed by installing all pip packages you need in the environment.

After installing all the conda and pip packages to export the environment use:

conda env export -n <env-name> > environment.yml

This will create the required file in the folder