标签归档：python-library

github

Celery-分布式任务队列(开发分支)

版本： 5.1.2(太阳谐波)
网站： https://docs.celeryproject.org/en/stable/index.html
下载： https://pypi.org/project/celery/
来源： https://github.com/celery/celery/
关键词： 任务、队列、作业、异步、rabbitmq、amqp、redis、python、分布式、参与者

捐款

这项工程有赖于你的慷慨捐助。

如果您正在使用celery创建商业产品，请考虑成为我们的backer或者我们的sponsor确保celery的未来

对于企业而言

作为Tidelift订阅的一部分提供

的维护人员celery数以千计的其他软件包正在与Tidelift合作，为您用来构建应用程序的开源依赖项提供商业支持和维护。节省时间、降低风险并提高代码的健全性，同时付钱给您使用的确切依赖项的维护人员。Learn more.

什么是任务队列？

任务队列用作跨线程或计算机分配工作的机制

任务队列的输入是一个工作单元，称为任务，专用的工作进程然后不断地监视队列以查找要执行的新工作

celery通过消息进行通信，通常使用经纪人在客户和工人之间进行调解。为了启动任务，客户端将消息放到队列中，然后代理将消息传递给工作器

celery系统可以由多个工作进程和代理组成，为高可用性和水平扩展让路。

celery是用Python编写的，但是协议可以用任何语言实现。除了Python，还有node-celery对于Node.js，一个PHP client，gocelery对于戈朗，还有rusty-celery对于生锈

语言互操作性也可以通过使用WebHook来实现，其方式是客户端将工作人员请求的URL入队

我需要什么？

celery5.1.2版运行在

  • Python(3.6、3.7、3.8、3.9)
  • Pypy3.6(7.6)

这是下一版本的celery，将支持Python 3.6或更高版本

如果您运行的是较旧版本的Python，则需要运行较旧版本的celery：

  • Python 2.6：celery系列3.1或更早版本
  • Python 2.5：celery系列3.0或更早版本
  • Python 2.4：celery系列2.2或更早版本
  • Python 2.7：celery4.x系列

celery是一个资金最少的项目，所以我们不支持Microsoft Windows。请不要打开任何与该平台相关的问题

celery通常与message broker一起使用来发送和接收消息。RabbitMQ、Redis传输功能齐全，但也有对无数其他解决方案的实验支持，包括使用SQLite进行本地开发

celery可以在单台计算机、多台计算机甚至跨数据中心运行

开始使用

如果这是您第一次尝试使用celery，或者您是以前版本的celery5.0.5或5.1.2的新手，那么您应该阅读我们的入门教程：

您还可以通过使用托管代理传输CloudAMQP开始使用celery。RabbitMQ最大的主机提供商是celery的骄傲赞助商

celery是

  • 简单

    celery很容易使用和维护，而且确实不需要配置文件

    它有一个活跃、友好的社区，您可以与之交谈以寻求支持，例如在我们的mailing-list，或IRC通道

    下面是您可以制作的最简单的应用程序之一：

    from celery import Celery

app = Celery('hello', broker='amqp://guest@localhost//')

@app.task
def hello():
    return 'hello world'
  • 高可用性

    在连接丢失或故障的情况下，工作人员和客户端将自动重试，一些代理通过以下方式支持HA主要/主要或主/复制副本复制

  • 快地

    单个celery进程一分钟可以处理数百万个任务，往返延迟为亚毫秒级(使用RabbitMQ、py-librabbitmq和优化设置)

  • 灵活性

    几乎每一个地方celery可以自己扩展或使用，定制池实现、序列化程序、压缩方案、日志记录、调度程序、使用者、生产者、代理传输等等

它支持

  • 消息传输
  • 并发性
  • 结果商店
    • AMQP，Redis
    • memcached
    • SQL炼金术，Django ORM
    • Apache Cassandra、IronCache、Elasticsearch
  • 序列化
    • 泡菜，JSON，YAML，消息包
    • zlib，bzip2压缩
    • 加密消息签名

框架集成

celery很容易与web框架集成，其中一些框架甚至有集成包：

Django 不需要
Pyramid pyramid_celery
Pylons celery-pylons
Flask 不需要
web2py web2py-celery
Tornado tornado-celery

集成包并不是严格必需的，但是它们可以简化开发，有时它们还会添加重要的挂钩，比如在以下位置关闭数据库连接fork

文档

这个latest documentation托管在Read the Docs上，包含用户指南、教程和API参考

最新的中文文档托管在https://www.celerycn.io/中，包含用户指南、教程、接口接口等。

安装

您可以通过Python包索引(PyPI)或从源代码安装celery

要使用安装，请执行以下操作pip：

$ pip install -U Celery

捆绑包

celery还定义了一组可用于安装celery的捆绑包以及给定功能的依赖项

您可以在需求中或在pip命令行中使用方括号。可以通过逗号分隔多个捆绑包来指定它们

$ pip install "celery[librabbitmq]"

$ pip install "celery[librabbitmq,redis,auth,msgpack]"

以下捆绑包可用：

序列化程序

celery[auth]： 用于使用auth安全序列化程序
celery[msgpack]： 用于使用msgpack序列化程序
celery[yaml]： 用于使用YAML序列化程序

并发性

celery[eventlet]： 用于使用eventlet游泳池
celery[gevent]： 用于使用gevent游泳池

传输和后端

celery[librabbitmq]： 用于使用librabbitmq C库
celery[redis]： 将Redis用作消息传输或作为结果的后端
celery[sqs]： 使用Amazon SQS作为消息传输
celery[tblib]： 用于使用task_remote_tracebacks功能
celery[memcache]： 用于将memcached用作结果后端(使用pylibmc)
celery[pymemcache]： 用于将memcached用作结果后端(纯Python实现)
celery[cassandra]： 用于将Apache Cassandra用作DataStAX驱动程序的结果后端
celery[azureblockblob]： 使用Azure Storage作为结果后端(使用azure-storage)
celery[s3]： 使用S3存储作为结果后端
celery[couchbase]： 将Couchbase用作结果后端
celery[arangodb]： 使用ArangoDB作为结果后端
celery[elasticsearch]： 用于将Elasticsearch用作结果后端
celery[riak]： 使用Riak作为结果后端
celery[cosmosdbsql]： 使用Azure Cosmos DB作为结果后端(使用pydocumentdb)
celery[zookeeper]： 用于将ZooKeeper用作消息传输
celery[sqlalchemy]： 使用SQLAlChemy作为结果后端(支持)
celery[pyro]： 为了使用Pyro4消息传输(实验性的)
celery[slmq]： 为了使用SoftLayer消息队列传输(实验性的)
celery[consul]： 用于将Consul.io密钥/值存储用作消息传输或结果后端(实验性的)
celery[django]： 指定Django支持的可能最低版本

您可能不应该在您的需求中使用它，它在这里仅供参考

从源下载并安装

从PyPI下载最新版本的celery：

https://pypi.org/project/celery/

您可以通过执行以下操作来安装它：

$ tar xvfz celery-0.0.0.tar.gz
$ cd celery-0.0.0
$ python setup.py build
# python setup.py install

如果您当前未使用虚拟环境，则必须以特权用户身份执行最后一个命令

使用开发版本

带管道

celery开发版本还需要kombu，amqp，billiard，以及vine

您可以使用以下pip命令安装这些快照的最新版本：

$ pip install https://github.com/celery/celery/zipball/master#egg=celery
$ pip install https://github.com/celery/billiard/zipball/master#egg=billiard
$ pip install https://github.com/celery/py-amqp/zipball/master#egg=amqp
$ pip install https://github.com/celery/kombu/zipball/master#egg=kombu
$ pip install https://github.com/celery/vine/zipball/master#egg=vine

使用GIT

请参阅投稿部分

获取帮助

邮件列表

有关celery的用途、发展和未来的讨论，请加入celery-users邮件列表

IRC

来和我们在IRC上聊天吧。这个#celery频道位于Freenode网络

漏洞跟踪器

如果您有任何建议、错误报告或烦恼，请将它们报告给我们的问题跟踪器，地址为https://github.com/celery/celery/issues/

维基

https://github.com/celery/celery/wiki

学分

贡献者

这个项目的存在要归功于所有做出贡献的人。celery的开发发生在GitHub：https://github.com/celery/celery

我们非常鼓励您参与celery的开发。如果您不喜欢GitHub(出于某种原因)，欢迎您发送常规补丁

请务必同时阅读Contributing to Celery部分，请参阅文档中的

支持者

感谢我们所有的支持者！🙏[Become a backer]

赞助商

通过成为赞助商来支持这个项目。您的徽标将在此处显示，并带有指向您的网站的链接。[Become a sponsor]


许可证

此软件在新的BSD许可证下获得许可。请参阅LICENSE完整许可证文本的顶级分发目录中的文件

github

Rich-Rich是一个Python库，用于终端中的富文本和美观的格式设置

中文 readme·Lengua española readme·Deutsche readme·Läs på svenska·日本語 readme·한국어 readme

Rich是一个Python库，用于富有终端中的文本和美观的格式

这个Rich API使您可以轻松地向终端输出添加颜色和样式。Rich还可以呈现漂亮的表格、进度条、标记、语法突出显示的源代码、回溯等等–开箱即用

有关Rich的视频介绍，请参阅calmcode.io通过@fishnets88

看看什么people are saying about Rich

兼容性

Rich适用于Linux、OSX和Windows。真彩色/表情符号适用于新的Windows终端，经典终端仅限16色。Rich需要Python 3.6.1或更高版本

Rich与Jupyter notebooks无需额外配置

正在安装

随一起安装pip或您最喜欢的PyPI包管理器

pip install rich

运行以下命令在您的终端上测试Rich Output：

python -m rich

丰富多彩的印刷品

若要毫不费力地向应用程序添加丰富的输出，可以将rich print方法，该方法与内置Python函数具有相同的签名。试试这个：

from rich import print

print("Hello, [bold magenta]World[/bold magenta]!", ":vampire:", locals())

丰富的REPL

Rich可以安装在Python REPL中，这样任何数据结构都会非常漂亮地打印和突出显示

>>> from rich import pretty
>>> pretty.install()

使用控制台

要更好地控制富终端内容，请导入并构造Console对象

from rich.console import Console

console = Console()

Console对象有一个print方法，该方法的接口有意与构建的print功能。下面是一个使用示例：

console.print("Hello", "World!")

如您所料，这将打印出来"Hello World!"去航站楼。请注意，与建筑不同的是print函数时，Rich将对文本进行自动换行以适应终端宽度

有几种方法可以将颜色和样式添加到输出中。您可以为整个输出设置样式，方法是将style关键字参数。下面是一个示例：

console.print("Hello", "World!", style="bold red")

输出将如下所示：

对于一次设置一行文本的样式来说，这是很好的。对于更细粒度的样式，Rich呈现了一个特殊的标记，该标记在语法上类似于bbcode下面是一个示例：

console.print("Where there is a [bold cyan]Will[/bold cyan] there [u]is[/u] a [i]way[/i].")

您可以使用Console对象以最小的工作量生成复杂的输出。请参阅Console API详细信息请参阅文档

丰富考察

里奇有一个inspect可以生成有关任何Python对象(如类、实例或构建)的报告的函数

>>> my_list = ["foo", "bar"]
>>> from rich import inspect
>>> inspect(my_list, methods=True)

请参阅inspect docs有关详细信息，请参阅

丰富的图书馆

RICH包含多个建筑可渲染对象您可以使用在CLI中创建优雅的输出，并帮助您调试代码

有关详细信息，请单击以下标题：

日志

Console对象有一个log()方法，该方法具有类似于print()，而且还呈现当前时间的列以及进行调用的文件和行。默认情况下，Rich将对Python结构和REPR字符串进行语法高亮显示。如果您记录一个集合(例如，词典或列表)，Rich会漂亮地打印它，以便它可以放在可用的空间中。以下是其中一些功能的示例

from rich.console import Console
console = Console()

test_data = [
    {"jsonrpc": "2.0", "method": "sum", "params": [None, 1, 2, 4, False, True], "id": "1",},
    {"jsonrpc": "2.0", "method": "notify_hello", "params": [7]},
    {"jsonrpc": "2.0", "method": "subtract", "params": [42, 23], "id": "2"},
]

def test_log():
    enabled = False
    context = {
        "foo": "bar",
    }
    movies = ["Deadpool", "Rise of the Skywalker"]
    console.log("Hello from", console, "!")
    console.log(test_data, log_locals=True)


test_log()

以上将产生以下输出：

请注意log_locals参数，该参数输出一个包含调用log方法的局部变量的表

LOG方法可用于登录到终端，用于长时间运行的应用程序(如服务器)，但也是非常好的调试辅助工具

日志记录处理程序

您还可以使用内置的Handler class对来自Python日志记录模块的输出进行格式化和着色。以下是输出的示例：

表情符号

要在控制台输出中插入表情符号，请将名称放在两个冒号之间。下面是一个示例：

>>> console.print(":smiley: :vampire: :pile_of_poo: :thumbs_up: :raccoon:")
😃 🧛 💩 👍 🦝

请明智地使用此功能

表格

丰富的可以灵活地呈现tables使用Unicode方框字符。边框、样式、单元格对齐等有多种格式选项

上面的动画是用table_movie.py在Examples目录中

下面是一个更简单的表格示例：

from rich.console import Console
from rich.table import Table

console = Console()

table = Table(show_header=True, header_style="bold magenta")
table.add_column("Date", style="dim", width=12)
table.add_column("Title")
table.add_column("Production Budget", justify="right")
table.add_column("Box Office", justify="right")
table.add_row(
    "Dev 20, 2019", "Star Wars: The Rise of Skywalker", "$275,000,000", "$375,126,118"
)
table.add_row(
    "May 25, 2018",
    "[red]Solo[/red]: A Star Wars Story",
    "$275,000,000",
    "$393,151,347",
)
table.add_row(
    "Dec 15, 2017",
    "Star Wars Ep. VIII: The Last Jedi",
    "$262,000,000",
    "[bold]$1,332,539,889[/bold]",
)

console.print(table)

这将产生以下输出：

请注意，控制台标记的呈现方式与print()和log()事实上，Rich可以呈现的任何内容都可能包含在标题/行中(甚至其他表)

这个Table类足够智能，可以调整列的大小以适应终端的可用宽度，并根据需要对文本进行换行。下面是相同的示例，端子比上表小：

进度条

丰富的可以呈现多个无闪烁progress用于跟踪长期运行任务的条形图

对于基本用法，将任何序列包装在track函数并迭代结果。下面是一个示例：

from rich.progress import track

for step in track(range(100)):
    do_step(step)

添加多个进度条并不难。以下是文档中的一个示例：

这些列可以配置为显示您想要的任何详细信息。内置列包括完成百分比、文件大小、文件速度和剩余时间。下面是另一个示例，显示正在进行的下载：

要亲自尝试此功能，请参见examples/downloader.py它可以在显示进度的同时同时下载多个URL

状态

对于很难计算进度的情况，可以使用status方法，该方法将显示“微调器”动画和消息。动画不会阻止您正常使用控制台。下面是一个示例：

from time import sleep
from rich.console import Console

console = Console()
tasks = [f"task {n}" for n in range(1, 11)]

with console.status("[bold green]Working on tasks...") as status:
    while tasks:
        task = tasks.pop(0)
        sleep(1)
        console.log(f"{task} complete")

这将在终端中生成以下输出

微调器动画借用自cli-spinners您可以通过指定spinner参数。运行以下命令以查看可用值：

python -m rich.spinner

上面的命令在终端中生成以下输出：

树

Rich可以呈现一个tree带着指引线。树是显示文件结构或任何其他分层数据的理想选择

树的标签可以是简单的文本，也可以是Rich可以呈现的任何其他内容。运行以下命令进行演示：

python -m rich.tree

这将生成以下输出：

请参阅tree.py显示任何目录的树视图的脚本示例，类似于Linuxtree命令

列

Rich可以整齐地呈现内容columns具有相等或最佳宽度的。下面是(MacOS/Linux)的一个非常基本的克隆ls按列显示目录列表的命令：

import os
import sys

from rich import print
from rich.columns import Columns

directory = os.listdir(sys.argv[1])
print(Columns(directory))

下面的屏幕截图是columns example它以列的形式显示从API拉取的数据：

降价

Rich可以渲染markdown并且合理地将格式转换到终端

若要呈现标记，请将Markdown类，并使用包含标记代码的字符串构造它。然后将其打印到控制台。下面是一个示例：

from rich.console import Console
from rich.markdown import Markdown

console = Console()
with open("README.md") as readme:
    markdown = Markdown(readme.read())
console.print(markdown)

这将产生类似以下内容的输出：

语法突出显示

Rich使用pygments要实施的库syntax highlighting用法类似于呈现标记；构造Syntax对象，并将其打印到控制台。下面是一个示例：

from rich.console import Console
from rich.syntax import Syntax

my_code = '''
def iter_first_last(values: Iterable[T]) -> Iterable[Tuple[bool, bool, T]]:
    """Iterate and generate a tuple with a flag for first and last value."""
    iter_values = iter(values)
    try:
        previous_value = next(iter_values)
    except StopIteration:
        return
    first = True
    for value in iter_values:
        yield first, False, previous_value
        first = False
        previous_value = value
    yield first, True, previous_value
'''
syntax = Syntax(my_code, "python", theme="monokai", line_numbers=True)
console = Console()
console.print(syntax)

这将产生以下输出：

跟踪回溯

Rich可以渲染beautiful tracebacks它们比标准Python回溯更容易阅读和显示更多代码。您可以将Rich设置为默认的回溯处理程序，这样所有未捕获的异常都将由Rich呈现

下面是它在OSX上的样子(与Linux类似)：

所有丰富渲染器都使用Console Protocol，您还可以使用它来实现您自己的富内容

为企业发财致富

作为Tidelift订阅的一部分提供

Rich和数千个其他软件包的维护者正在与Tidelift合作，为您用来构建应用程序的开源软件包提供商业支持和维护。节省时间、降低风险并提高代码的健全性，同时付钱给您所使用的包的维护者。Learn more.

使用Rich的项目

以下是一些使用Rich的项目：

github

Awesome-python-一个不错的Python框架、库、软件和资源的精选列表

Awesome Python

一个不错的Python框架、库、软件和资源的精选列表

灵感来自awawed-php

管理面板

管理接口库

  • ajenti – The admin panel your servers deserve.
  • django-grappelli – A jazzy skin for the Django Admin-Interface.
  • django-jet – Modern responsive template for the Django admin interface with improved functionality.
  • django-suit – Alternative Django Admin-Interface (free only for Non-commercial use).
  • django-xadmin – Drop-in replacement of Django admin comes with lots of goodies.
  • flask-admin – Simple and extensible administrative interface framework for Flask.
  • flower – Real-time monitor and web admin for Celery.
  • jet-bridge – Admin panel framework for any application with nice UI (ex Jet Django)
  • wooey – A Django app which creates automatic web UIs for Python scripts.

算法和设计模式

数据结构、算法和设计模式的Python实现。另见令人敬畏的算法

  • Algorithms
    • algorithms – Minimal examples of data structures and algorithms.
    • python-ds – A collection of data structure and algorithms for coding interviews.
    • sortedcontainers – Fast and pure-Python implementation of sorted collections.
    • TheAlgorithms – All Algorithms implemented in Python.
  • Design Patterns
    • PyPattyrn – A simple yet effective library for implementing common design patterns.
    • python-patterns – A collection of design patterns in Python.
    • transitions – A lightweight, object-oriented finite state machine implementation.

ASGI服务器

与ASGI兼容的Web服务器

  • daphne – A HTTP, HTTP2 and WebSocket protocol server for ASGI and ASGI-HTTP.
  • uvicorn – A lightning-fast ASGI server implementation, using uvloop and httptools.

异步编程

  • asyncio – (Python standard library) Asynchronous I/O, event loop, coroutines and tasks.
  • trio – A friendly library for async concurrency and I/O.
  • Twisted – An event-driven networking engine.
  • uvloop – Ultra fast asyncio event loop.

音频

用于操作音频及其元数据的库

  • Audio
    • audioread – Cross-library (GStreamer + Core Audio + MAD + FFmpeg) audio decoding.
    • dejavu – Audio fingerprinting and recognition.
    • kapre – Keras Audio Preprocessors
    • librosa – Python library for audio and music analysis
    • matchering – A library for automated reference audio mastering.
    • mingus – An advanced music theory and notation package with MIDI file and playback support.
    • pyAudioAnalysis – Audio feature extraction, classification, segmentation and applications.
    • pydub – Manipulate audio with a simple and easy high level interface.
    • TimeSide – Open web audio processing framework.
  • Metadata
    • beets – A music library manager and MusicBrainz tagger.
    • eyeD3 – A tool for working with audio files, specifically MP3 files containing ID3 metadata.
    • mutagen – A Python module to handle audio metadata.
    • tinytag – A library for reading music meta data of MP3, OGG, FLAC and Wave files.

身份验证

用于实现身份验证方案的库

  • OAuth
    • authlib – JavaScript Object Signing and Encryption draft implementation.
    • django-allauth – Authentication app for Django that “just works.”
    • django-oauth-toolkit – OAuth 2 goodies for Django.
    • oauthlib – A generic and thorough implementation of the OAuth request-signing logic.
    • python-oauth2 – A fully tested, abstract interface to creating OAuth clients and servers.
    • python-social-auth – An easy-to-setup social authentication mechanism.
  • JWT
    • pyjwt – JSON Web Token implementation in Python.
    • python-jose – A JOSE implementation in Python.
    • python-jwt – A module for generating and verifying JSON Web Tokens.

构建工具

从源代码编译软件

  • BitBake – A make-like build tool for embedded Linux.
  • buildout – A build system for creating, assembling and deploying applications from multiple parts.
  • PlatformIO – A console tool to build code with different development platforms.
  • pybuilder – A continuous build tool written in pure Python.
  • SCons – A software construction tool.

内置类增强功能

用于增强Python内置类的库

  • attrs – Replacement for __init__, __eq__, __repr__, etc. boilerplate in class definitions.
  • bidict – Efficient, Pythonic bidirectional map data structures and related functionality..
  • Box – Python dictionaries with advanced dot notation access.
  • dataclasses – (Python standard library) Data classes.
  • DottedDict – A library that provides a method of accessing lists and dicts with a dotted path notation.

CMS

内容管理系统

  • django-cms – An Open source enterprise CMS based on the Django.
  • feincms – One of the most advanced Content Management Systems built on Django.
  • indico – A feature-rich event management system, made @ CERN.
  • Kotti – A high-level, Pythonic web application framework built on Pyramid.
  • mezzanine – A powerful, consistent, and flexible content management platform.
  • plone – A CMS built on top of the open source application server Zope.
  • quokka – Flexible, extensible, small CMS powered by Flask and MongoDB.
  • wagtail – A Django content management system.

缓存

用于缓存数据的库

  • beaker – A WSGI middleware for sessions and caching.
  • django-cache-machine – Automatic caching and invalidation for Django models.
  • django-cacheops – A slick ORM cache with automatic granular event-driven invalidation.
  • dogpile.cache – dogpile.cache is next generation replacement for Beaker made by same authors.
  • HermesCache – Python caching library with tag-based invalidation and dogpile effect prevention.
  • pylibmc – A Python wrapper around the libmemcached interface.
  • python-diskcache – SQLite and file backed cache backend with faster lookups than memcached and redis.

ChatOps工具

Chatbot开发库

  • errbot – The easiest and most popular chatbot to implement ChatOps.

代码分析

静电分析工具，链接器和代码质量检查器。另见棒极了-静电-分析

  • Code Analysis
    • coala – Language independent and easily extendable code analysis application.
    • code2flow – Turn your Python and JavaScript code into DOT flowcharts.
    • prospector – A tool to analyse Python code.
    • pycallgraph – A library that visualises the flow (call graph) of your Python application.
    • vulture – A tool for finding and analysing dead Python code.
  • Code Linters
  • Code Formatters
    • black – The uncompromising Python code formatter.
    • isort – A Python utility / library to sort imports.
    • yapf – Yet another Python code formatter from Google.
  • Static Type Checkers, also see awesome-python-typing
    • mypy – Check variable types during compile time.
    • pyre-check – Performant type checking.
    • typeshed – Collection of library stubs for Python, with static types.
  • Static Type Annotations Generators
    • MonkeyType – A system for Python that generates static type annotations by collecting runtime types.
    • pyannotate – Auto-generate PEP-484 annotations.
    • pytype – Pytype checks and infers types for Python code – without requiring type annotations.

命令行界面开发

用于构建命令行应用程序的库

  • Command-line Application Development
    • cement – CLI Application Framework for Python.
    • click – A package for creating beautiful command line interfaces in a composable way.
    • cliff – A framework for creating command-line programs with multi-level commands.
    • docopt – Pythonic command line arguments parser.
    • python-fire – A library for creating command line interfaces from absolutely any Python object.
    • python-prompt-toolkit – A library for building powerful interactive command lines.
  • Terminal Rendering
    • alive-progress – A new kind of Progress Bar, with real-time throughput, eta and very cool animations.
    • asciimatics – A package to create full-screen text UIs (from interactive forms to ASCII animations).
    • bashplotlib – Making basic plots in the terminal.
    • colorama – Cross-platform colored terminal text.
    • rich – Python library for rich text and beautiful formatting in the terminal. Also provides a great RichHandler log handler.
    • tqdm – Fast, extensible progress bar for loops and CLI.

命令行工具

基于CLI的实用工具，可提高工作效率

  • Productivity Tools
    • copier – A library and command-line utility for rendering projects templates.
    • cookiecutter – A command-line utility that creates projects from cookiecutters (project templates).
    • doitlive – A tool for live presentations in the terminal.
    • howdoi – Instant coding answers via the command line.
    • Invoke – A tool for managing shell-oriented subprocesses and organizing executable Python code into CLI-invokable tasks.
    • PathPicker – Select files out of bash output.
    • percol – Adds flavor of interactive selection to the traditional pipe concept on UNIX.
    • thefuck – Correcting your previous console command.
    • tmuxp – A tmux session manager.
    • try – A dead simple CLI to try out python packages – it’s never been easier.
  • CLI Enhancements
    • httpie – A command line HTTP client, a user-friendly cURL replacement.
    • iredis – Redis CLI with autocompletion and syntax highlighting.
    • kube-shell – An integrated shell for working with the Kubernetes CLI.
    • litecli – SQLite CLI with autocompletion and syntax highlighting.
    • mycli – MySQL CLI with autocompletion and syntax highlighting.
    • pgcli – PostgreSQL CLI with autocompletion and syntax highlighting.
    • saws – A Supercharged aws-cli.

兼容性

用于从Python 2迁移到3的库

  • python-future – The missing compatibility layer between Python 2 and Python 3.
  • modernize – Modernizes Python code for eventual Python 3 migration.
  • six – Python 2 and 3 compatibility utilities.

计算机视觉

计算机视觉图书馆

  • EasyOCR – Ready-to-use OCR with 40+ languages supported.
  • Face Recognition – Simple facial recognition library.
  • Kornia – Open Source Differentiable Computer Vision Library for PyTorch.
  • OpenCV – Open Source Computer Vision Library.
  • pytesseract – A wrapper for Google Tesseract OCR.
  • SimpleCV – An open source framework for building computer vision applications.
  • tesserocr – Another simple, Pillow-friendly, wrapper around the tesseract-ocr API for OCR.

并发性和并行性

用于并发和并行执行的库。也请参见AWOWE-Asyncio

  • concurrent.futures – (Python standard library) A high-level interface for asynchronously executing callables.
  • eventlet – Asynchronous framework with WSGI support.
  • gevent – A coroutine-based Python networking library that uses greenlet.
  • multiprocessing – (Python standard library) Process-based parallelism.
  • scoop – Scalable Concurrent Operations in Python.
  • uvloop – Ultra fast implementation of asyncio event loop on top of libuv.

配置

用于存储和解析配置选项的库

  • configobj – INI file parser with validation.
  • configparser – (Python standard library) INI file parser.
  • hydra – Hydra is a framework for elegantly configuring complex applications.
  • profig – Config from multiple formats with value conversion.
  • python-decouple – Strict separation of settings from code.

密码学

  • cryptography – A package designed to expose cryptographic primitives and recipes to Python developers.
  • paramiko – The leading native Python SSHv2 protocol library.
  • passlib – Secure password storage/hashing library, very high level.
  • pynacl – Python binding to the Networking and Cryptography (NaCl) library.

数据分析

用于数据分析的库

  • AWS Data Wrangler – Pandas on AWS.
  • Blaze – NumPy and Pandas interface to Big Data.
  • Open Mining – Business Intelligence (BI) in Pandas interface.
  • Optimus – Agile Data Science Workflows made easy with PySpark.
  • Orange – Data mining, data visualization, analysis and machine learning through visual programming or scripts.
  • Pandas – A library providing high-performance, easy-to-use data structures and data analysis tools.

数据验证

用于验证数据的库。在许多情况下用于表单

  • Cerberus – A lightweight and extensible data validation library.
  • colander – Validating and deserializing data obtained via XML, JSON, an HTML form post.
  • jsonschema – An implementation of JSON Schema for Python.
  • schema – A library for validating Python data structures.
  • Schematics – Data Structure Validation.
  • valideer – Lightweight extensible data validation and adaptation library.
  • voluptuous – A Python data validation library.

数据可视化

用于可视化数据的库。另请参阅awawed-javascript

  • Altair – Declarative statistical visualization library for Python.
  • Bokeh – Interactive Web Plotting for Python.
  • bqplot – Interactive Plotting Library for the Jupyter Notebook
  • Cartopy – A cartographic python library with matplotlib support
  • Dash – Built on top of Flask, React and Plotly aimed at analytical web applications.
  • diagrams – Diagram as Code.
  • Matplotlib – A Python 2D plotting library.
  • plotnine – A grammar of graphics for Python based on ggplot2.
  • Pygal – A Python SVG Charts Creator.
  • PyGraphviz – Python interface to Graphviz.
  • PyQtGraph – Interactive and realtime 2D/3D/Image plotting and science/engineering widgets.
  • Seaborn – Statistical data visualization using Matplotlib.
  • VisPy – High-performance scientific visualization based on OpenGL.

数据库

在Python中实现的数据库

  • pickleDB – A simple and lightweight key-value store for Python.
  • tinydb – A tiny, document-oriented database.
  • ZODB – A native object database for Python. A key-value and object graph database.

数据库驱动程序

用于连接和操作数据库的库

  • MySQL – awesome-mysql
  • PostgreSQL – awesome-postgres
    • psycopg2 – The most popular PostgreSQL adapter for Python.
    • queries – A wrapper of the psycopg2 library for interacting with PostgreSQL.
  • SQlite – awesome-sqlite
    • sqlite3 – (Python standard library) SQlite interface compliant with DB-API 2.0
    • SuperSQLite – A supercharged SQLite library built on top of apsw.
  • Other Relational Databases
    • pymssql – A simple database interface to Microsoft SQL Server.
    • clickhouse-driver – Python driver with native interface for ClickHouse.
  • NoSQL Databases
    • cassandra-driver – The Python Driver for Apache Cassandra.
    • happybase – A developer-friendly library for Apache HBase.
    • kafka-python – The Python client for Apache Kafka.
    • py2neo – A client library and toolkit for working with Neo4j.
    • pymongo – The official Python client for MongoDB.
    • redis-py – The Python client for Redis.
  • Asynchronous Clients
    • motor – The async Python driver for MongoDB.

日期和时间

用于处理日期和时间的库

  • Arrow – A Python library that offers a sensible and human-friendly approach to creating, manipulating, formatting and converting dates, times and timestamps.
  • Chronyk – A Python 3 library for parsing human-written times and dates.
  • dateutil – Extensions to the standard Python datetime module.
  • delorean – A library for clearing up the inconvenient truths that arise dealing with datetimes.
  • maya – Datetimes for Humans.
  • moment – A Python library for dealing with dates/times. Inspired by Moment.js.
  • Pendulum – Python datetimes made easy.
  • PyTime – An easy-to-use Python module which aims to operate date/time/datetime by string.
  • pytz – World timezone definitions, modern and historical. Brings the tz database into Python.
  • when.py – Providing user-friendly functions to help perform common date and time actions.

调试工具

用于调试代码的库

  • pdb-like Debugger
    • ipdb – IPython-enabled pdb.
    • pdb++ – Another drop-in replacement for pdb.
    • pudb – A full-screen, console-based Python debugger.
    • wdb – An improbable web debugger through WebSockets.
  • Tracing
    • lptrace – strace for Python programs.
    • manhole – Debugging UNIX socket connections and present the stacktraces for all threads and an interactive prompt.
    • pyringe – Debugger capable of attaching to and injecting code into Python processes.
    • python-hunter – A flexible code tracing toolkit.
  • Profiler
    • line_profiler – Line-by-line profiling.
    • memory_profiler – Monitor Memory usage of Python code.
    • py-spy – A sampling profiler for Python programs. Written in Rust.
    • pyflame – A ptracing profiler For Python.
    • vprof – Visual Python profiler.
  • Others
    • django-debug-toolbar – Display various debug information for Django.
    • django-devserver – A drop-in replacement for Django’s runserver.
    • flask-debugtoolbar – A port of the django-debug-toolbar to flask.
    • icecream – Inspect variables, expressions, and program execution with a single, simple function call.
    • pyelftools – Parsing and analyzing ELF files and DWARF debugging information.

深度学习

神经网络和深度学习的框架。另请参阅令人敬畏的深度学习

  • caffe – A fast open framework for deep learning..
  • keras – A high-level neural networks library and capable of running on top of either TensorFlow or Theano.
  • mxnet – A deep learning framework designed for both efficiency and flexibility.
  • pytorch – Tensors and Dynamic neural networks in Python with strong GPU acceleration.
  • SerpentAI – Game agent framework. Use any video game as a deep learning sandbox.
  • tensorflow – The most popular Deep Learning framework created by Google.
  • Theano – A library for fast numerical computation.

DevOps工具

适用于DevOps的软件和库

  • Configuration Management
    • ansible – A radically simple IT automation platform.
    • cloudinit – A multi-distribution package that handles early initialization of a cloud instance.
    • OpenStack – Open source software for building private and public clouds.
    • pyinfra – A versatile CLI tools and python libraries to automate infrastructure.
    • saltstack – Infrastructure automation and management system.
  • SSH-style Deployment
    • cuisine – Chef-like functionality for Fabric.
    • fabric – A simple, Pythonic tool for remote execution and deployment.
    • fabtools – Tools for writing awesome Fabric files.
  • Process Management
    • honcho – A Python clone of Foreman, for managing Procfile-based applications.
    • supervisor – Supervisor process control system for UNIX.
  • Monitoring
    • psutil – A cross-platform process and system utilities module.
  • Backup
    • BorgBackup – A deduplicating archiver with compression and encryption.
  • Others

分布式计算

分布式计算的框架和库

  • Batch Processing
    • dask – A flexible parallel computing library for analytic computing.
    • luigi – A module that helps you build complex pipelines of batch jobs.
    • mrjob – Run MapReduce jobs on Hadoop or Amazon Web Services.
    • PySpark – Apache Spark Python API.
    • Ray – A system for parallel and distributed Python that unifies the machine learning ecosystem.
  • Stream Processing

分布

为发行版创建打包的可执行文件的库

  • dh-virtualenv – Build and distribute a virtualenv as a Debian package.
  • Nuitka – Compile scripts, modules, packages to an executable or extension module.
  • py2app – Freezes Python scripts (Mac OS X).
  • py2exe – Freezes Python scripts (Windows).
  • pyarmor – A tool used to obfuscate python scripts, bind obfuscated scripts to fixed machine or expire obfuscated scripts.
  • PyInstaller – Converts Python programs into stand-alone executables (cross-platform).
  • pynsist – A tool to build Windows installers, installers bundle Python itself.
  • shiv – A command line utility for building fully self-contained zipapps (PEP 441), but with all their dependencies included.

文档

用于生成项目文档的库

  • sphinx – Python Documentation generator.
  • pdoc – Epydoc replacement to auto generate API documentation for Python libraries.
  • pycco – The literate-programming-style documentation generator.

下载器

用于下载的库

  • akshare – A financial data interface library, built for human beings!
  • s3cmd – A command line tool for managing Amazon S3 and CloudFront.
  • s4cmd – Super S3 command line tool, good for higher performance.
  • you-get – A YouTube/Youku/Niconico video downloader written in Python 3.
  • youtube-dl – A small command-line program to download videos from YouTube.

电子商务

电子商务和支付的框架和库

  • alipay – Unofficial Alipay API for Python.
  • Cartridge – A shopping cart app built using the Mezzanine.
  • django-oscar – An open-source e-commerce framework for Django.
  • django-shop – A Django based shop system.
  • forex-python – Foreign exchange rates, Bitcoin price index and currency conversion.
  • merchant – A Django app to accept payments from various payment processors.
  • money – Money class with optional CLDR-backed locale-aware formatting and an extensible currency exchange.
  • python-currencies – Display money format and its filthy currencies.
  • saleor – An e-commerce storefront for Django.
  • shoop – An open source E-Commerce platform based on Django.

编辑器插件和IDE

  • Emacs
    • elpy – Emacs Python Development Environment.
  • Sublime Text
    • anaconda – Anaconda turns your Sublime Text 3 in a full featured Python development IDE.
    • SublimeJEDI – A Sublime Text plugin to the awesome auto-complete library Jedi.
  • Vim
    • jedi-vim – Vim bindings for the Jedi auto-completion library for Python.
    • python-mode – An all in one plugin for turning Vim into a Python IDE.
    • YouCompleteMe – Includes Jedi-based completion engine for Python.
  • Visual Studio
    • PTVS – Python Tools for Visual Studio.
  • Visual Studio Code
    • Python – The official VSCode extension with rich support for Python.
  • IDE
    • PyCharm – Commercial Python IDE by JetBrains. Has free community edition available.
    • spyder – Open Source Python IDE.

电子邮件

用于发送和解析电子邮件的库

  • Mail Servers
    • modoboa – A mail hosting and management platform including a modern Web UI.
    • salmon – A Python Mail Server.
  • Clients
    • imbox – Python IMAP for Humans.
    • yagmail – Yet another Gmail/SMTP client.
  • Others
    • flanker – An email address and Mime parsing library.
    • mailer – High-performance extensible mail delivery framework.

企业应用程序集成

用于在企业环境中进行系统集成的平台和工具

  • Zato – ESB, SOA, REST, APIs and Cloud Integrations in Python.

环境管理

用于Python版本和虚拟环境管理的库

  • pyenv – Simple Python version management.
  • virtualenv – A tool to create isolated Python environments.

文件

用于文件操作和MIME类型检测的库

  • mimetypes – (Python standard library) Map filenames to MIME types.
  • path.py – A module wrapper for os.path.
  • pathlib – (Python standard library) An cross-platform, object-oriented path library.
  • PyFilesystem2 – Python’s filesystem abstraction layer.
  • python-magic – A Python interface to the libmagic file type identification library.
  • Unipath – An object-oriented approach to file/directory operations.
  • watchdog – API and shell utilities to monitor file system events.

外来函数接口

用于提供外来函数接口的库

  • cffi – Foreign Function Interface for Python calling C code.
  • ctypes – (Python standard library) Foreign Function Interface for Python calling C code.
  • PyCUDA – A Python wrapper for Nvidia’s CUDA API.
  • SWIG – Simplified Wrapper and Interface Generator.

表格

用于处理表单的库

  • Deform – Python HTML form generation library influenced by the formish form generation library.
  • django-bootstrap3 – Bootstrap 3 integration with Django.
  • django-bootstrap4 – Bootstrap 4 integration with Django.
  • django-crispy-forms – A Django app which lets you create beautiful forms in a very elegant and DRY way.
  • django-remote-forms – A platform independent Django form serializer.
  • WTForms – A flexible forms validation and rendering library.

函数式程序设计

使用Python进行函数式编程

  • Coconut – A variant of Python built for simple, elegant, Pythonic functional programming.
  • CyToolz – Cython implementation of Toolz: High performance functional utilities.
  • fn.py – Functional programming in Python: implementation of missing features to enjoy FP.
  • funcy – A fancy and practical functional tools.
  • more-itertools – More routines for operating on iterables, beyond itertools.
  • returns – A set of type-safe monads, transformers, and composition utilities.
  • Toolz – A collection of functional utilities for iterators, functions, and dictionaries.

GUI开发

用于使用图形用户界面应用程序的库

  • curses – Built-in wrapper for ncurses used to create terminal GUI applications.
  • Eel – A library for making simple Electron-like offline HTML/JS GUI apps.
  • enaml – Creating beautiful user-interfaces with Declarative Syntax like QML.
  • Flexx – Flexx is a pure Python toolkit for creating GUI’s, that uses web technology for its rendering.
  • Gooey – Turn command line programs into a full GUI application with one line.
  • kivy – A library for creating NUI applications, running on Windows, Linux, Mac OS X, Android and iOS.
  • pyglet – A cross-platform windowing and multimedia library for Python.
  • PyGObject – Python Bindings for GLib/GObject/GIO/GTK+ (GTK+3).
  • PyQt – Python bindings for the Qt cross-platform application and UI framework.
  • PySimpleGUI – Wrapper for tkinter, Qt, WxPython and Remi.
  • pywebview – A lightweight cross-platform native wrapper around a webview component.
  • Tkinter – Tkinter is Python’s de-facto standard GUI package.
  • Toga – A Python native, OS native GUI toolkit.
  • urwid – A library for creating terminal GUI applications with strong support for widgets, events, rich colors, etc.
  • wxPython – A blending of the wxWidgets C++ class library with the Python.
  • DearPyGui – A Simple GPU accelerated Python GUI framework

图形QL

用于使用GraphQL的库

  • graphene – GraphQL framework for Python.
  • tartiflette-aiohttp – An aiohttp-based wrapper for Tartiflette to expose GraphQL APIs over HTTP.
  • tartiflette-asgi – ASGI support for the Tartiflette GraphQL engine.
  • tartiflette – SDL-first GraphQL engine implementation for Python 3.6+ and asyncio.

游戏开发

超棒的游戏开发库

  • Arcade – Arcade is a modern Python framework for crafting games with compelling graphics and sound.
  • Cocos2d – cocos2d is a framework for building 2D games, demos, and other graphical/interactive applications.
  • Harfang3D – Python framework for 3D, VR and game development.
  • Panda3D – 3D game engine developed by Disney.
  • Pygame – Pygame is a set of Python modules designed for writing games.
  • PyOgre – Python bindings for the Ogre 3D render engine, can be used for games, simulations, anything 3D.
  • PyOpenGL – Python ctypes bindings for OpenGL and it’s related APIs.
  • PySDL2 – A ctypes based wrapper for the SDL2 library.
  • RenPy – A Visual Novel engine.

地理位置

用于对地址进行地理编码以及处理纬度和经度的库

  • django-countries – A Django app that provides a country field for models and forms.
  • GeoDjango – A world-class geographic web framework.
  • GeoIP – Python API for MaxMind GeoIP Legacy Database.
  • geojson – Python bindings and utilities for GeoJSON.
  • geopy – Python Geocoding Toolbox.

HTML操作

用于处理HTML和XML的库

  • BeautifulSoup – Providing Pythonic idioms for iterating, searching, and modifying HTML or XML.
  • bleach – A whitelist-based HTML sanitization and text linkification library.
  • cssutils – A CSS library for Python.
  • html5lib – A standards-compliant library for parsing and serializing HTML documents and fragments.
  • lxml – A very fast, easy-to-use and versatile library for handling HTML and XML.
  • MarkupSafe – Implements a XML/HTML/XHTML Markup safe string for Python.
  • pyquery – A jQuery-like library for parsing HTML.
  • untangle – Converts XML documents to Python objects for easy access.
  • WeasyPrint – A visual rendering engine for HTML and CSS that can export to PDF.
  • xmldataset – Simple XML Parsing.
  • xmltodict – Working with XML feel like you are working with JSON.

HTTP客户端

用于使用HTTP的库

  • grequests – requests + gevent for asynchronous HTTP requests.
  • httplib2 – Comprehensive HTTP client library.
  • httpx – A next generation HTTP client for Python.
  • requests – HTTP Requests for Humans.
  • treq – Python requests like API built on top of Twisted’s HTTP client.
  • urllib3 – A HTTP library with thread-safe connection pooling, file post support, sanity friendly.

硬件

用于硬件编程的库

  • ino – Command line toolkit for working with Arduino.
  • keyboard – Hook and simulate global keyboard events on Windows and Linux.
  • mouse – Hook and simulate global mouse events on Windows and Linux.
  • Pingo – Pingo provides a uniform API to program devices like the Raspberry Pi, pcDuino, Intel Galileo, etc.
  • PyUserInput – A module for cross-platform control of the mouse and keyboard.
  • scapy – A brilliant packet manipulation library.

图像处理

用于操作图像的库

  • hmap – Image histogram remapping.
  • imgSeek – A project for searching a collection of images using visual similarity.
  • nude.py – Nudity detection.
  • pagan – Retro identicon (Avatar) generation based on input string and hash.
  • pillow – Pillow is the friendly PIL fork.
  • python-barcode – Create barcodes in Python with no extra dependencies.
  • pygram – Instagram-like image filters.
  • PyMatting – A library for alpha matting.
  • python-qrcode – A pure Python QR Code generator.
  • pywal – A tool that generates color schemes from images.
  • pyvips – A fast image processing library with low memory needs.
  • Quads – Computer art based on quadtrees.
  • scikit-image – A Python library for (scientific) image processing.
  • thumbor – A smart imaging service. It enables on-demand crop, re-sizing and flipping of images.
  • wand – Python bindings for MagickWand, C API for ImageMagick.

实施方案

Python的实现

  • CLPython – Implementation of the Python programming language written in Common Lisp.
  • CPython – Default, most widely used implementation of the Python programming language written in C.
  • Cython – Optimizing Static Compiler for Python.
  • Grumpy – More compiler than interpreter as more powerful CPython2.7 replacement (alpha).
  • IronPython – Implementation of the Python programming language written in C#.
  • Jython – Implementation of Python programming language written in Java for the JVM.
  • MicroPython – A lean and efficient Python programming language implementation.
  • Numba – Python JIT compiler to LLVM aimed at scientific Python.
  • PeachPy – x86-64 assembler embedded in Python.
  • Pyjion – A JIT for Python based upon CoreCLR.
  • PyPy – A very fast and compliant implementation of the Python language.
  • Pyston – A Python implementation using JIT techniques.
  • Stackless Python – An enhanced version of the Python programming language.

交互式口译器

交互式Python解释器(REPL)

国际化

用于使用I18n的库

  • Babel – An internationalization library for Python.
  • PyICU – A wrapper of International Components for Unicode C++ library (ICU).

作业调度器

用于调度作业的库

  • Airflow – Airflow is a platform to programmatically author, schedule and monitor workflows.
  • APScheduler – A light but powerful in-process task scheduler that lets you schedule functions.
  • django-schedule – A calendaring app for Django.
  • doit – A task runner and build tool.
  • gunnery – Multipurpose task execution tool for distributed systems with web-based interface.
  • Joblib – A set of tools to provide lightweight pipelining in Python.
  • Plan – Writing crontab file in Python like a charm.
  • Prefect – A modern workflow orchestration framework that makes it easy to build, schedule and monitor robust data pipelines.
  • schedule – Python job scheduling for humans.
  • Spiff – A powerful workflow engine implemented in pure Python.
  • TaskFlow – A Python library that helps to make task execution easy, consistent and reliable.

日志记录

用于生成和使用日志的库

  • logbook – Logging replacement for Python.
  • logging – (Python standard library) Logging facility for Python.
  • loguru – Library which aims to bring enjoyable logging in Python.
  • sentry-python – Sentry SDK for Python.
  • structlog – Structured logging made easy.

机器学习

机器学习库。另见令人敬畏的机器学习

  • gym – A toolkit for developing and comparing reinforcement learning algorithms.
  • H2O – Open Source Fast Scalable Machine Learning Platform.
  • Metrics – Machine learning evaluation metrics.
  • NuPIC – Numenta Platform for Intelligent Computing.
  • scikit-learn – The most popular Python library for Machine Learning.
  • Spark ML – Apache Spark‘s scalable Machine Learning library.
  • vowpal_porpoise – A lightweight Python wrapper for Vowpal Wabbit.
  • xgboost – A scalable, portable, and distributed gradient boosting library.
  • MindsDB – MindsDB is an open source AI layer for existing databases that allows you to effortlessly develop, train and deploy state-of-the-art machine learning models using standard queries.

Microsoft Windows

Microsoft Windows上的Python编程

  • Python(x,y) – Scientific-applications-oriented Python Distribution based on Qt and Spyder.
  • pythonlibs – Unofficial Windows binaries for Python extension packages.
  • PythonNet – Python Integration with the .NET Common Language Runtime (CLR).
  • PyWin32 – Python Extensions for Windows.
  • WinPython – Portable development environment for Windows 7/8.

杂项

不属于上述类别的有用的库或工具

  • blinker – A fast Python in-process signal/event dispatching system.
  • boltons – A set of pure-Python utilities.
  • itsdangerous – Various helpers to pass trusted data to untrusted environments.
  • magenta – A tool to generate music and art using artificial intelligence.
  • pluginbase – A simple but flexible plugin system for Python.
  • tryton – A general purpose business framework.

自然语言处理

用于处理人类语言的库

  • General
    • gensim – Topic Modeling for Humans.
    • langid.py – Stand-alone language identification system.
    • nltk – A leading platform for building Python programs to work with human language data.
    • pattern – A web mining module.
    • polyglot – Natural language pipeline supporting hundreds of languages.
    • pytext – A natural language modeling framework based on PyTorch.
    • PyTorch-NLP – A toolkit enabling rapid deep learning NLP prototyping for research.
    • spacy – A library for industrial-strength natural language processing in Python and Cython.
    • Stanza – The Stanford NLP Group’s official Python library, supporting 60+ languages.
  • Chinese
    • funNLP – A collection of tools and datasets for Chinese NLP.
    • jieba – The most popular Chinese text segmentation library.
    • pkuseg-python – A toolkit for Chinese word segmentation in various domains.
    • snownlp – A library for processing Chinese text.

网络虚拟化

虚拟网络和SDN(软件定义网络)的工具和库

  • mininet – A popular network emulator and API written in Python.
  • napalm – Cross-vendor API to manipulate network devices.
  • pox – A Python-based SDN control applications, such as OpenFlow SDN controllers.

新闻提要

用于构建用户活动的库

ORM

实现对象关系映射或数据映射技术的库

  • Relational Databases
    • Django Models – The Django ORM.
    • SQLAlchemy – The Python SQL Toolkit and Object Relational Mapper.
    • dataset – Store Python dicts in a database – works with SQLite, MySQL, and PostgreSQL.
    • orator – The Orator ORM provides a simple yet beautiful ActiveRecord implementation.
    • orm – An async ORM.
    • peewee – A small, expressive ORM.
    • pony – ORM that provides a generator-oriented interface to SQL.
    • pydal – A pure Python Database Abstraction Layer.
  • NoSQL Databases
    • hot-redis – Rich Python data types for Redis.
    • mongoengine – A Python Object-Document-Mapper for working with MongoDB.
    • PynamoDB – A Pythonic interface for Amazon DynamoDB.
    • redisco – A Python Library for Simple Models and Containers Persisted in Redis.

套餐管理

用于包和依赖项管理的库

  • pip – The package installer for Python.
    • pip-tools – A set of tools to keep your pinned Python dependencies fresh.
    • PyPI
  • conda – Cross-platform, Python-agnostic binary package manager.
  • poetry – Python dependency management and packaging made easy.

包资料档案库

本地PyPI存储库服务器和代理

  • bandersnatch – PyPI mirroring tool provided by Python Packaging Authority (PyPA).
  • devpi – PyPI server and packaging/testing/release tool.
  • localshop – Local PyPI server (custom packages and auto-mirroring of pypi).
  • warehouse – Next generation Python Package Repository (PyPI).

渗透试验

渗透测试的框架和工具

  • fsociety – A Penetration testing framework.
  • setoolkit – A toolkit for social engineering.
  • sqlmap – Automatic SQL injection and database takeover tool.

权限

允许或拒绝用户访问数据或功能的库

  • django-guardian – Implementation of per object permissions for Django 1.2+
  • django-rules – A tiny but powerful app providing object-level permissions to Django, without requiring a database.

流程

用于启动操作系统进程并与其通信的库

推荐系统

用于构建推荐系统的库

  • annoy – Approximate Nearest Neighbors in C++/Python optimized for memory usage.
  • fastFM – A library for Factorization Machines.
  • implicit – A fast Python implementation of collaborative filtering for implicit datasets.
  • libffm – A library for Field-aware Factorization Machine (FFM).
  • lightfm – A Python implementation of a number of popular recommendation algorithms.
  • spotlight – Deep recommender models using PyTorch.
  • Surprise – A scikit for building and analyzing recommender systems.
  • tensorrec – A Recommendation Engine Framework in TensorFlow.

重构

Python的重构工具和库

  • Bicycle Repair Man – Bicycle Repair Man, a refactoring tool for Python.
  • Bowler – Safe code refactoring for modern Python.
  • Rope – Rope is a python refactoring library.

RESTful API

用于构建REST风格API的库

  • Django
  • Flask
    • eve – REST API framework powered by Flask, MongoDB and good intentions.
    • flask-api – Browsable Web APIs for Flask.
    • flask-restful – Quickly building REST APIs for Flask.
  • Pyramid
    • cornice – A RESTful framework for Pyramid.
  • Framework agnostic
    • apistar – A smart Web API framework, designed for Python 3.
    • falcon – A high-performance framework for building cloud APIs and web app backends.
    • fastapi – A modern, fast, web framework for building APIs with Python 3.6+ based on standard Python type hints.
    • hug – A Python 3 framework for cleanly exposing APIs.
    • sandman2 – Automated REST APIs for existing database-driven systems.
    • sanic – A Python 3.6+ web server and web framework that’s written to go fast.
    • vibora – Fast, efficient and asynchronous Web framework inspired by Flask.

机器人学

机器人图书馆

  • PythonRobotics – This is a compilation of various robotics algorithms with visualizations.
  • rospy – This is a library for ROS (Robot Operating System).

RPC服务器

与RPC兼容的服务器

  • RPyC (Remote Python Call) – A transparent and symmetric RPC library for Python
  • zeroRPC – zerorpc is a flexible RPC implementation based on ZeroMQ and MessagePack.

科学

用于科学计算的库。另请参阅面向科学家的Python

  • astropy – A community Python library for Astronomy.
  • bcbio-nextgen – Providing best-practice pipelines for fully automated high throughput sequencing analysis.
  • bccb – Collection of useful code related to biological analysis.
  • Biopython – Biopython is a set of freely available tools for biological computation.
  • cclib – A library for parsing and interpreting the results of computational chemistry packages.
  • Colour – Implementing a comprehensive number of colour theory transformations and algorithms.
  • Karate Club – Unsupervised machine learning toolbox for graph structured data.
  • NetworkX – A high-productivity software for complex networks.
  • NIPY – A collection of neuroimaging toolkits.
  • NumPy – A fundamental package for scientific computing with Python.
  • ObsPy – A Python toolbox for seismology.
  • Open Babel – A chemical toolbox designed to speak the many languages of chemical data.
  • PyDy – Short for Python Dynamics, used to assist with workflow in the modeling of dynamic motion.
  • PyMC – Markov Chain Monte Carlo sampling toolkit.
  • QuTiP – Quantum Toolbox in Python.
  • RDKit – Cheminformatics and Machine Learning Software.
  • SciPy – A Python-based ecosystem of open-source software for mathematics, science, and engineering.
  • SimPy – A process-based discrete-event simulation framework.
  • statsmodels – Statistical modeling and econometrics in Python.
  • SymPy – A Python library for symbolic mathematics.
  • Zipline – A Pythonic algorithmic trading library.

搜索

用于对数据进行索引和执行搜索查询的库和软件

序列化

用于序列化复杂数据类型的库

无服务器框架

用于开发无服务器Python代码的框架

  • python-lambda – A toolkit for developing and deploying Python code in AWS Lambda.
  • Zappa – A tool for deploying WSGI applications on AWS Lambda and API Gateway.

壳

基于Python的Shell

  • xonsh – A Python-powered, cross-platform, Unix-gazing shell language and command prompt.

特定格式处理

用于解析和操作特定文本格式的库

  • General
    • tablib – A module for Tabular Datasets in XLS, CSV, JSON, YAML.
  • Office
    • docxtpl – Editing a docx document by jinja2 template
    • openpyxl – A library for reading and writing Excel 2010 xlsx/xlsm/xltx/xltm files.
    • pyexcel – Providing one API for reading, manipulating and writing csv, ods, xls, xlsx and xlsm files.
    • python-docx – Reads, queries and modifies Microsoft Word 2007/2008 docx files.
    • python-pptx – Python library for creating and updating PowerPoint (.pptx) files.
    • unoconv – Convert between any document format supported by LibreOffice/OpenOffice.
    • XlsxWriter – A Python module for creating Excel .xlsx files.
    • xlwings – A BSD-licensed library that makes it easy to call Python from Excel and vice versa.
    • xlwt / xlrd – Writing and reading data and formatting information from Excel files.
  • PDF
    • PDFMiner – A tool for extracting information from PDF documents.
    • PyPDF2 – A library capable of splitting, merging and transforming PDF pages.
    • ReportLab – Allowing Rapid creation of rich PDF documents.
  • Markdown
    • Mistune – Fastest and full featured pure Python parsers of Markdown.
    • Python-Markdown – A Python implementation of John Gruber’s Markdown.
  • YAML
    • PyYAML – YAML implementations for Python.
  • CSV
    • csvkit – Utilities for converting to and working with CSV.
  • Archive
    • unp – A command line tool that can unpack archives easily.

静电网站生成器

静电网站生成器是一个软件，它接受一些文本+模板作为输入，并在输出上生成Html文件

  • lektor – An easy to use static CMS and blog engine.
  • mkdocs – Markdown friendly documentation generator.
  • makesite – Simple, lightweight, and magic-free static site/blog generator (< 130 lines).
  • nikola – A static website and blog generator.
  • pelican – Static site generator that supports Markdown and reST syntax.

加标签

用于标记项目的库

任务队列

用于处理任务队列的库

  • celery – An asynchronous task queue/job queue based on distributed message passing.
  • dramatiq – A fast and reliable background task processing library for Python 3.
  • huey – Little multi-threaded task queue.
  • mrq – A distributed worker task queue in Python using Redis & gevent.
  • rq – Simple job queues for Python.

模板引擎

用于模板化和词法分析的库和工具

  • Genshi – Python templating toolkit for generation of web-aware output.
  • Jinja2 – A modern and designer friendly templating language.
  • Mako – Hyperfast and lightweight templating for the Python platform.

测试

用于测试代码库和生成测试数据的库

  • Testing Frameworks
    • hypothesis – Hypothesis is an advanced Quickcheck style property based testing library.
    • nose2 – The successor to nose, based on `unittest2.
    • pytest – A mature full-featured Python testing tool.
    • Robot Framework – A generic test automation framework.
    • unittest – (Python standard library) Unit testing framework.
  • Test Runners
    • green – A clean, colorful test runner.
    • mamba – The definitive testing tool for Python. Born under the banner of BDD.
    • tox – Auto builds and tests distributions in multiple Python versions
  • GUI / Web Testing
    • locust – Scalable user load testing tool written in Python.
    • PyAutoGUI – PyAutoGUI is a cross-platform GUI automation Python module for human beings.
    • Schemathesis – A tool for automatic property-based testing of web applications built with Open API / Swagger specifications.
    • Selenium – Python bindings for Selenium WebDriver.
    • sixpack – A language-agnostic A/B Testing framework.
    • splinter – Open source tool for testing web applications.
  • Mock
    • doublex – Powerful test doubles framework for Python.
    • freezegun – Travel through time by mocking the datetime module.
    • httmock – A mocking library for requests for Python 2.6+ and 3.2+.
    • httpretty – HTTP request mock tool for Python.
    • mock – (Python standard library) A mocking and patching library.
    • mocket – A socket mock framework with gevent/asyncio/SSL support.
    • responses – A utility library for mocking out the requests Python library.
    • VCR.py – Record and replay HTTP interactions on your tests.
  • Object Factories
    • factory_boy – A test fixtures replacement for Python.
    • mixer – Another fixtures replacement. Supports Django, Flask, SQLAlchemy, Peewee and etc.
    • model_mommy – Creating random fixtures for testing in Django.
  • Code Coverage
    • coverage – Code coverage measurement.
  • Fake Data
    • fake2db – Fake database generator.
    • faker – A Python package that generates fake data.
    • mimesis – is a Python library that help you generate fake data.
    • radar – Generate random datetime / time.

文本处理

用于解析和操作纯文本的库

  • General
    • chardet – Python 2/3 compatible character encoding detector.
    • difflib – (Python standard library) Helpers for computing deltas.
    • ftfy – Makes Unicode text less broken and more consistent automagically.
    • fuzzywuzzy – Fuzzy String Matching.
    • Levenshtein – Fast computation of Levenshtein distance and string similarity.
    • pangu.py – Paranoid text spacing.
    • pyfiglet – An implementation of figlet written in Python.
    • pypinyin – Convert Chinese hanzi (漢字) to pinyin (拼音).
    • textdistance – Compute distance between sequences with 30+ algorithms.
    • unidecode – ASCII transliterations of Unicode text.
  • Slugify
    • awesome-slugify – A Python slugify library that can preserve unicode.
    • python-slugify – A Python slugify library that translates unicode to ASCII.
    • unicode-slugify – A slugifier that generates unicode slugs with Django as a dependency.
  • Unique identifiers
    • hashids – Implementation of hashids in Python.
    • shortuuid – A generator library for concise, unambiguous and URL-safe UUIDs.
  • Parser
    • ply – Implementation of lex and yacc parsing tools for Python.
    • pygments – A generic syntax highlighter.
    • pyparsing – A general purpose framework for generating parsers.
    • python-nameparser – Parsing human names into their individual components.
    • python-phonenumbers – Parsing, formatting, storing and validating international phone numbers.
    • python-user-agents – Browser user agent parser.
    • sqlparse – A non-validating SQL parser.

第三方接口

用于访问第三方服务API的库。另请参阅Python API包装器和库列表

URL操作

用于解析URL的库

  • furl – A small Python library that makes parsing and manipulating URLs easy.
  • purl – A simple, immutable URL class with a clean API for interrogation and manipulation.
  • pyshorteners – A pure Python URL shortening lib.
  • webargs – A friendly library for parsing HTTP request arguments with built-in support for popular web frameworks.

视频

用于操作视频和GIF的库

  • moviepy – A module for script-based movie editing with many formats, including animated GIFs.
  • scikit-video – Video processing routines for SciPy.
  • vidgear – Most Powerful multi-threaded Video Processing framework.

Web资产管理

用于管理、压缩和缩小网站资产的工具

  • django-compressor – Compresses linked and inline JavaScript or CSS into a single cached file.
  • django-pipeline – An asset packaging library for Django.
  • django-storages – A collection of custom storage back ends for Django.
  • fanstatic – Packages, optimizes, and serves static file dependencies as Python packages.
  • fileconveyor – A daemon to detect and sync files to CDNs, S3 and FTP.
  • flask-assets – Helps you integrate webassets into your Flask app.
  • webassets – Bundles, optimizes, and manages unique cache-busting URLs for static resources.

Web内容提取

用于提取Web内容的库

  • html2text – Convert HTML to Markdown-formatted text.
  • lassie – Web Content Retrieval for Humans.
  • micawber – A small library for extracting rich content from URLs.
  • newspaper – News extraction, article extraction and content curation in Python.
  • python-readability – Fast Python port of arc90’s readability tool.
  • requests-html – Pythonic HTML Parsing for Humans.
  • sumy – A module for automatic summarization of text documents and HTML pages.
  • textract – Extract text from any document, Word, PowerPoint, PDFs, etc.
  • toapi – Every web site provides APIs.

网络爬行

用于自动执行Web抓取的库

  • cola – A distributed crawling framework.
  • feedparser – Universal feed parser.
  • grab – Site scraping framework.
  • MechanicalSoup – A Python library for automating interaction with websites.
  • portia – Visual scraping for Scrapy.
  • pyspider – A powerful spider system.
  • robobrowser – A simple, Pythonic library for browsing the web without a standalone web browser.
  • scrapy – A fast high-level screen scraping and web crawling framework.

Web框架

传统的全栈Web框架。另请参阅REST风格的API

WebSocket

用于使用WebSocket的库

  • autobahn-python – WebSocket & WAMP for Python on Twisted and asyncio.
  • channels – Developer-friendly asynchrony for Django.
  • websockets – A library for building WebSocket servers and clients with a focus on correctness and simplicity.

WSGI服务器

与WSGI兼容的Web服务器

  • bjoern – Asynchronous, very fast and written in C.
  • gunicorn – Pre-forked, ported from Ruby’s Unicorn project.
  • uWSGI – A project aims at developing a full stack for building hosting services, written in C.
  • waitress – Multi-threaded, powers Pyramid.
  • werkzeug – A WSGI utility library for Python that powers Flask and can easily be embedded into your own projects.

Resources

在何处查找学习资源或新的Python库

书籍

网站

时事通讯

播客

Contributing

我们永远欢迎您的贡献！请先看一下投稿指南。

如果我不确定这些库是否很棒，我会保留一些拉取请求，您可以通过向它们添加：+1：来投票支持它们。当票数达到20时，拉取请求将被合并

如果您对这份固执己见的列表有任何疑问，请不要犹豫，请在Twitter上联系我@VintaChen，或者在GitHub上打开一个问题