标签归档:yaml

Dynaconf 轻松实现 Python 动态配置管理

Dynaconf 是一个库,旨在成为在 Python 中管理配置的最佳选择。

它可以从各种来源读取设置,包括环境变量、文件、服务器配置等。

它适用于任何类型的 Python 程序,包括 Flask 和 Django 扩展。

1.准备

开始之前,你要确保Python和pip已经成功安装在电脑上,如果没有,请访问这篇文章:超详细Python安装指南 进行安装。

(可选1) 如果你用Python的目的是数据分析,可以直接安装Anaconda:Python数据分析与挖掘好帮手—Anaconda,它内置了Python和pip.

(可选2) 此外,推荐大家用VSCode编辑器来编写小型Python项目:Python 编程的最好搭档—VSCode 详细指南

Windows环境下打开Cmd(开始—运行—CMD),苹果系统环境下请打开Terminal(command+空格输入Terminal),输入命令安装依赖:

pip install dynaconf

2.初步使用DynaConf

在你的项目的根目录中运行 dynaconf init 命令。

cd path/to/your/project/
dynaconf init -f toml

会有类似如下的输出,说明初始化完成:

⚙️  Configuring your Dynaconf environment
------------------------------------------
🐍 The file `config.py` was generated.

🎛️  settings.toml created to hold your settings.

🔑 .secrets.toml created to hold your secrets.

🙈 the .secrets.* is also included in `.gitignore`
beware to not push your secrets to a public repo.

🎉 Dynaconf is configured! read more on https://dynaconf.com

刚刚初始化的时候我们选择了 toml 格式。实际上你还可以选择 toml|yaml|json|ini|py,不过 toml 是默认的,也是最推荐的配置格式。

初始化完成后会创建以下文件:

.
├── config.py       # 需要被导入的配置脚本
├── .secrets.toml   # 像密码等敏感信息配置
└── settings.toml   # 应用配置

初始化完成后你就可以编写你的配置,编辑settings.toml:

key = "value"
a_boolean = false
number = 1234
a_float = 56.8
a_list = [1, 2, 3, 4]
a_dict = {hello="world"}

[a_dict.nested]
other_level = "nested value"

然后就可以在你的代码中导入并使用这些配置:

from config import settings

assert settings.key == "value"
assert settings.number == 789
assert settings.a_dict.nested.other_level == "nested value"
assert settings['a_boolean'] is False
assert settings.get("DONTEXIST", default=1) == 1

如果是密码等敏感信息,你可以配置在 .secrets.toml 中:

password = "s3cr3t"
token = "dfgrfg5d4g56ds4gsdf5g74984we5345-"
message = "This file doesn't go to your pub repo"

.secrets.toml 文件会被自动加入到 .gitignore 文件中,这些信息不会被上传到Git仓库上。

同时,DYNACONF还支持带前缀的环境变量:

export DYNACONF_NUMBER=789
export DYNACONF_FOO=false
export DYNACONF_DATA__CAN__BE__NESTED=value
export DYNACONF_FORMATTED_KEY="@format {this.FOO}/BAR"
export DYNACONF_TEMPLATED_KEY="@jinja {{ env['HOME'] | abspath }}"

3.高级使用

你还可以在Flask或Django中使用DynaConf,以Django为例,第一步要先确保已经设置 DJANGO_SETTINGS_MODULE 环境变量:

export DJANGO_SETTINGS_MODULE=yourproject.settings

然后在 manage.py 相同文件夹下运行初始化命令:

dynaconf init -f yaml

然后按照终端上的说明进行操作:

Django app detected
⚙️  Configuring your Dynaconf environment
------------------------------------------
🎛️  settings.yaml created to hold your settings.

🔑 .secrets.yaml created to hold your secrets.

🙈 the .secrets.yaml is also included in `.gitignore`
beware to not push your secrets to a public repo
or use dynaconf builtin support for Vault Servers.

⁉  path/to/yourproject/settings.py is found do you want to add dynaconf? [y/N]:

回答 y:

🎠  Now your Django settings are managed by Dynaconf
🎉  Dynaconf is configured! read more on https://dynaconf.com

在 Django 上,推荐的文件格式是yaml,因为它可以更轻松地保存复杂的数据结构,但是你依然可以选择使用 toml、json、ini 甚至将你的配置保存为 .py 格式。

初始化 dynaconf 后,在现有的settings.py底部包含以下内容:

# HERE STARTS DYNACONF EXTENSION LOAD
import dynaconf  # noqa
settings = dynaconf.DjangoDynaconf(__name__)  # noqa
# HERE ENDS DYNACONF EXTENSION LOAD (No more code below this line)

现在,在你的 Django 视图、模型和所有其他地方,你现在可以正常使用 django.conf.settings,因为它已被 Dynaconf 设置对象替换。

from django.conf import settings


def index(request):
    assert settings.DEBUG is True
    assert settings.NAME == "Bruno"
    assert settings.DATABASES.default.name == "db"
    assert settings.get("NONEXISTENT", 2) == 2

现在,通过修改 manage.py 相同文件夹下的配置文件,就能让配置全局生效了。

我们的文章到此就结束啦,如果你喜欢今天的 Python 教程,请持续关注Python实用宝典。

有任何问题,可以在公众号后台回复:加群,回答相应验证信息,进入互助群询问。

原创不易,希望你能在下面点个赞和在看支持我继续创作,谢谢!


​Python实用宝典 ( pythondict.com )
不只是一个宝典
欢迎关注公众号:Python实用宝典

如何将YAML格式的数据写入文件?

问题:如何将YAML格式的数据写入文件?

我需要使用Python将以下数据写入yaml文件:

{A:a, B:{C:c, D:d, E:e}} 

即字典中的字典。我该如何实现?

I need to write the below data to yaml file using Python:

{A:a, B:{C:c, D:d, E:e}} 

i.e., dictionary in a dictionary. How can I achieve this?


回答 0

import yaml

data = dict(
    A = 'a',
    B = dict(
        C = 'c',
        D = 'd',
        E = 'e',
    )
)

with open('data.yml', 'w') as outfile:
    yaml.dump(data, outfile, default_flow_style=False)

default_flow_style=False参数对于产生所需的格式(流样式)是必需的,否则对于嵌套集合,它将产生块样式:

A: a
B: {C: c, D: d, E: e}
import yaml

data = dict(
    A = 'a',
    B = dict(
        C = 'c',
        D = 'd',
        E = 'e',
    )
)

with open('data.yml', 'w') as outfile:
    yaml.dump(data, outfile, default_flow_style=False)

The default_flow_style=False parameter is necessary to produce the format you want (flow style), otherwise for nested collections it produces block style:

A: a
B: {C: c, D: d, E: e}

回答 1

链接到PyYAML文档,其中显示了default_flow_style参数的差异。要将其以块模式写入文件(通常更具可读性):

d = {'A':'a', 'B':{'C':'c', 'D':'d', 'E':'e'}}
with open('result.yml', 'w') as yaml_file:
    yaml.dump(d, yaml_file, default_flow_style=False)

生成:

A: a
B:
  C: c
  D: d
  E: e

Link to the PyYAML documentation showing the difference for the default_flow_style parameter. To write it to a file in block mode (often more readable):

d = {'A':'a', 'B':{'C':'c', 'D':'d', 'E':'e'}}
with open('result.yml', 'w') as yaml_file:
    yaml.dump(d, yaml_file, default_flow_style=False)

produces:

A: a
B:
  C: c
  D: d
  E: e

在Python中,如何将YAML映射加载为OrderedDicts?

问题:在Python中,如何将YAML映射加载为OrderedDicts?

我想让PyYAML的加载器将映射(和有序映射)加载到Python 2.7+ OrderedDict类型中,而不是dict它当前使用的普通和​​对列表。

最好的方法是什么?

I’d like to get PyYAML‘s loader to load mappings (and ordered mappings) into the Python 2.7+ OrderedDict type, instead of the vanilla dict and the list of pairs it currently uses.

What’s the best way to do that?


回答 0

更新:在python 3.6+中OrderedDict,由于新的dict实现已经在pypy中使用了一段时间(尽管现在考虑了CPython实现的细节),您可能根本不需要。

更新:在python 3.7+中,dict对象的插入顺序保留性质已声明为Python语言规范的正式组成部分,请参阅Python 3.7的新增功能

我喜欢@James的简单解决方案。但是,它更改了默认的全局yaml.Loader类,这可能导致麻烦的副作用。特别是在编写库代码时,这是一个坏主意。另外,它不能直接与使用yaml.safe_load()

幸运的是,无需付出太多努力即可改进该解决方案:

import yaml
from collections import OrderedDict

def ordered_load(stream, Loader=yaml.Loader, object_pairs_hook=OrderedDict):
    class OrderedLoader(Loader):
        pass
    def construct_mapping(loader, node):
        loader.flatten_mapping(node)
        return object_pairs_hook(loader.construct_pairs(node))
    OrderedLoader.add_constructor(
        yaml.resolver.BaseResolver.DEFAULT_MAPPING_TAG,
        construct_mapping)
    return yaml.load(stream, OrderedLoader)

# usage example:
ordered_load(stream, yaml.SafeLoader)

对于序列化,我不知道明显的概括,但是至少这应该没有任何副作用:

def ordered_dump(data, stream=None, Dumper=yaml.Dumper, **kwds):
    class OrderedDumper(Dumper):
        pass
    def _dict_representer(dumper, data):
        return dumper.represent_mapping(
            yaml.resolver.BaseResolver.DEFAULT_MAPPING_TAG,
            data.items())
    OrderedDumper.add_representer(OrderedDict, _dict_representer)
    return yaml.dump(data, stream, OrderedDumper, **kwds)

# usage:
ordered_dump(data, Dumper=yaml.SafeDumper)

Update: In python 3.6+ you probably don’t need OrderedDict at all due to the new dict implementation that has been in use in pypy for some time (although considered CPython implementation detail for now).

Update: In python 3.7+, the insertion-order preservation nature of dict objects has been declared to be an official part of the Python language spec, see What’s New In Python 3.7.

I like @James’ solution for its simplicity. However, it changes the default global yaml.Loader class, which can lead to troublesome side effects. Especially, when writing library code this is a bad idea. Also, it doesn’t directly work with yaml.safe_load().

Fortunately, the solution can be improved without much effort:

import yaml
from collections import OrderedDict

def ordered_load(stream, Loader=yaml.Loader, object_pairs_hook=OrderedDict):
    class OrderedLoader(Loader):
        pass
    def construct_mapping(loader, node):
        loader.flatten_mapping(node)
        return object_pairs_hook(loader.construct_pairs(node))
    OrderedLoader.add_constructor(
        yaml.resolver.BaseResolver.DEFAULT_MAPPING_TAG,
        construct_mapping)
    return yaml.load(stream, OrderedLoader)

# usage example:
ordered_load(stream, yaml.SafeLoader)

For serialization, I don’t know an obvious generalization, but at least this shouldn’t have any side effects:

def ordered_dump(data, stream=None, Dumper=yaml.Dumper, **kwds):
    class OrderedDumper(Dumper):
        pass
    def _dict_representer(dumper, data):
        return dumper.represent_mapping(
            yaml.resolver.BaseResolver.DEFAULT_MAPPING_TAG,
            data.items())
    OrderedDumper.add_representer(OrderedDict, _dict_representer)
    return yaml.dump(data, stream, OrderedDumper, **kwds)

# usage:
ordered_dump(data, Dumper=yaml.SafeDumper)

回答 1

yaml模块允许您指定自定义“表示器”以将Python对象转换为文本,并指定“构造器”以逆转该过程。

_mapping_tag = yaml.resolver.BaseResolver.DEFAULT_MAPPING_TAG

def dict_representer(dumper, data):
    return dumper.represent_dict(data.iteritems())

def dict_constructor(loader, node):
    return collections.OrderedDict(loader.construct_pairs(node))

yaml.add_representer(collections.OrderedDict, dict_representer)
yaml.add_constructor(_mapping_tag, dict_constructor)

The yaml module allow you to specify custom ‘representers’ to convert Python objects to text and ‘constructors’ to reverse the process.

_mapping_tag = yaml.resolver.BaseResolver.DEFAULT_MAPPING_TAG

def dict_representer(dumper, data):
    return dumper.represent_dict(data.iteritems())

def dict_constructor(loader, node):
    return collections.OrderedDict(loader.construct_pairs(node))

yaml.add_representer(collections.OrderedDict, dict_representer)
yaml.add_constructor(_mapping_tag, dict_constructor)

回答 2

2018年选项:

oyamlPyYAML的直接替代品,保留了字典排序。同时支持Python 2和Python 3。只需pip install oyaml导入,如下所示:

import oyaml as yaml

在转储/加载时,您将不再为搞砸的映射而烦恼。

注意:我是oyaml的作者。

2018 option:

oyaml is a drop-in replacement for PyYAML which preserves dict ordering. Both Python 2 and Python 3 are supported. Just pip install oyaml, and import as shown below:

import oyaml as yaml

You’ll no longer be annoyed by screwed-up mappings when dumping/loading.

Note: I’m the author of oyaml.


回答 3

2015(及更高版本)选项:

ruamel.yaml是PyYAML的替代品(免责声明:我是该软件包的作者)。保留映射的顺序是2015年在第一版(0.1)中添加的内容之一。它不仅保留字典的顺序,还保留注释,锚点名称,标签并支持YAML 1.2规范(2009年发布)

规范说,不能保证排序,但是YAML文件中当然有排序,并且适当的解析器可以仅保留该排序器,并透明地生成一个保持排序的对象。您只需要选择正确的解析器,加载器和转储器¹:

import sys
from ruamel.yaml import YAML

yaml_str = """\
3: abc
conf:
    10: def
    3: gij     # h is missing
more:
- what
- else
"""

yaml = YAML()
data = yaml.load(yaml_str)
data['conf'][10] = 'klm'
data['conf'][3] = 'jig'
yaml.dump(data, sys.stdout)

会给你:

3: abc
conf:
  10: klm
  3: jig       # h is missing
more:
- what
- else

dataCommentedMap类似dict 的类型,但具有额外的信息,这些信息会一直保留直到被转储(包括保留的注释!)

2015 (and later) option:

ruamel.yaml is a drop in replacement for PyYAML (disclaimer: I am the author of that package). Preserving the order of the mappings was one of the things added in the first version (0.1) back in 2015. Not only does it preserve the order of your dictionaries, it will also preserve comments, anchor names, tags and does support the YAML 1.2 specification (released 2009)

The specification says that the ordering is not guaranteed, but of course there is ordering in the YAML file and the appropriate parser can just hold on to that and transparently generate an object that keeps the ordering. You just need to choose the right parser, loader and dumper¹:

import sys
from ruamel.yaml import YAML

yaml_str = """\
3: abc
conf:
    10: def
    3: gij     # h is missing
more:
- what
- else
"""

yaml = YAML()
data = yaml.load(yaml_str)
data['conf'][10] = 'klm'
data['conf'][3] = 'jig'
yaml.dump(data, sys.stdout)

will give you:

3: abc
conf:
  10: klm
  3: jig       # h is missing
more:
- what
- else

data is of type CommentedMap which functions like a dict, but has extra information that is kept around until being dumped (including the preserved comment!)


回答 4

注意:有一个基于以下答案的库,该库还实现了CLoader和CDumpers:Phynix / yamlloader

我非常怀疑这是最好的方法,但这是我想出的方法,并且确实有效。也可作为要点

import yaml
import yaml.constructor

try:
    # included in standard lib from Python 2.7
    from collections import OrderedDict
except ImportError:
    # try importing the backported drop-in replacement
    # it's available on PyPI
    from ordereddict import OrderedDict

class OrderedDictYAMLLoader(yaml.Loader):
    """
    A YAML loader that loads mappings into ordered dictionaries.
    """

    def __init__(self, *args, **kwargs):
        yaml.Loader.__init__(self, *args, **kwargs)

        self.add_constructor(u'tag:yaml.org,2002:map', type(self).construct_yaml_map)
        self.add_constructor(u'tag:yaml.org,2002:omap', type(self).construct_yaml_map)

    def construct_yaml_map(self, node):
        data = OrderedDict()
        yield data
        value = self.construct_mapping(node)
        data.update(value)

    def construct_mapping(self, node, deep=False):
        if isinstance(node, yaml.MappingNode):
            self.flatten_mapping(node)
        else:
            raise yaml.constructor.ConstructorError(None, None,
                'expected a mapping node, but found %s' % node.id, node.start_mark)

        mapping = OrderedDict()
        for key_node, value_node in node.value:
            key = self.construct_object(key_node, deep=deep)
            try:
                hash(key)
            except TypeError, exc:
                raise yaml.constructor.ConstructorError('while constructing a mapping',
                    node.start_mark, 'found unacceptable key (%s)' % exc, key_node.start_mark)
            value = self.construct_object(value_node, deep=deep)
            mapping[key] = value
        return mapping

Note: there is a library, based on the following answer, which implements also the CLoader and CDumpers: Phynix/yamlloader

I doubt very much that this is the best way to do it, but this is the way I came up with, and it does work. Also available as a gist.

import yaml
import yaml.constructor

try:
    # included in standard lib from Python 2.7
    from collections import OrderedDict
except ImportError:
    # try importing the backported drop-in replacement
    # it's available on PyPI
    from ordereddict import OrderedDict

class OrderedDictYAMLLoader(yaml.Loader):
    """
    A YAML loader that loads mappings into ordered dictionaries.
    """

    def __init__(self, *args, **kwargs):
        yaml.Loader.__init__(self, *args, **kwargs)

        self.add_constructor(u'tag:yaml.org,2002:map', type(self).construct_yaml_map)
        self.add_constructor(u'tag:yaml.org,2002:omap', type(self).construct_yaml_map)

    def construct_yaml_map(self, node):
        data = OrderedDict()
        yield data
        value = self.construct_mapping(node)
        data.update(value)

    def construct_mapping(self, node, deep=False):
        if isinstance(node, yaml.MappingNode):
            self.flatten_mapping(node)
        else:
            raise yaml.constructor.ConstructorError(None, None,
                'expected a mapping node, but found %s' % node.id, node.start_mark)

        mapping = OrderedDict()
        for key_node, value_node in node.value:
            key = self.construct_object(key_node, deep=deep)
            try:
                hash(key)
            except TypeError, exc:
                raise yaml.constructor.ConstructorError('while constructing a mapping',
                    node.start_mark, 'found unacceptable key (%s)' % exc, key_node.start_mark)
            value = self.construct_object(value_node, deep=deep)
            mapping[key] = value
        return mapping

回答 5

更新:不赞成使用该库,而推荐使用yamlloader(它基于yamlordereddictloader)

我刚刚找到了一个Python库(https://pypi.python.org/pypi/yamlordereddictloader/0.1.1),该库是基于此问题的答案而创建的,使用起来非常简单:

import yaml
import yamlordereddictloader

datas = yaml.load(open('myfile.yml'), Loader=yamlordereddictloader.Loader)

Update: the library was deprecated in favor of the yamlloader (which is based on the yamlordereddictloader)

I’ve just found a Python library (https://pypi.python.org/pypi/yamlordereddictloader/0.1.1) which was created based on answers to this question and is quite simple to use:

import yaml
import yamlordereddictloader

datas = yaml.load(open('myfile.yml'), Loader=yamlordereddictloader.Loader)

回答 6

在针对Python 2.7的For PyYaml安装中,我更新了__init __。py,constructor.py和loader.py。现在支持用于加载命令的object_pairs_hook选项。我所做的更改差异如下。

__init__.py

$ diff __init__.py Original
64c64
< def load(stream, Loader=Loader, **kwds):
---
> def load(stream, Loader=Loader):
69c69
<     loader = Loader(stream, **kwds)
---
>     loader = Loader(stream)
75c75
< def load_all(stream, Loader=Loader, **kwds):
---
> def load_all(stream, Loader=Loader):
80c80
<     loader = Loader(stream, **kwds)
---
>     loader = Loader(stream)

constructor.py

$ diff constructor.py Original
20,21c20
<     def __init__(self, object_pairs_hook=dict):
<         self.object_pairs_hook = object_pairs_hook
---
>     def __init__(self):
27,29d25
<     def create_object_hook(self):
<         return self.object_pairs_hook()
<
54,55c50,51
<         self.constructed_objects = self.create_object_hook()
<         self.recursive_objects = self.create_object_hook()
---
>         self.constructed_objects = {}
>         self.recursive_objects = {}
129c125
<         mapping = self.create_object_hook()
---
>         mapping = {}
400c396
<         data = self.create_object_hook()
---
>         data = {}
595c591
<             dictitems = self.create_object_hook()
---
>             dictitems = {}
602c598
<             dictitems = value.get('dictitems', self.create_object_hook())
---
>             dictitems = value.get('dictitems', {})

loader.py

$ diff loader.py Original
13c13
<     def __init__(self, stream, **constructKwds):
---
>     def __init__(self, stream):
18c18
<         BaseConstructor.__init__(self, **constructKwds)
---
>         BaseConstructor.__init__(self)
23c23
<     def __init__(self, stream, **constructKwds):
---
>     def __init__(self, stream):
28c28
<         SafeConstructor.__init__(self, **constructKwds)
---
>         SafeConstructor.__init__(self)
33c33
<     def __init__(self, stream, **constructKwds):
---
>     def __init__(self, stream):
38c38
<         Constructor.__init__(self, **constructKwds)
---
>         Constructor.__init__(self)

On my For PyYaml installation for Python 2.7 I updated __init__.py, constructor.py, and loader.py. Now supports object_pairs_hook option for load commands. Diff of changes I made is below.

__init__.py

$ diff __init__.py Original
64c64
< def load(stream, Loader=Loader, **kwds):
---
> def load(stream, Loader=Loader):
69c69
<     loader = Loader(stream, **kwds)
---
>     loader = Loader(stream)
75c75
< def load_all(stream, Loader=Loader, **kwds):
---
> def load_all(stream, Loader=Loader):
80c80
<     loader = Loader(stream, **kwds)
---
>     loader = Loader(stream)

constructor.py

$ diff constructor.py Original
20,21c20
<     def __init__(self, object_pairs_hook=dict):
<         self.object_pairs_hook = object_pairs_hook
---
>     def __init__(self):
27,29d25
<     def create_object_hook(self):
<         return self.object_pairs_hook()
<
54,55c50,51
<         self.constructed_objects = self.create_object_hook()
<         self.recursive_objects = self.create_object_hook()
---
>         self.constructed_objects = {}
>         self.recursive_objects = {}
129c125
<         mapping = self.create_object_hook()
---
>         mapping = {}
400c396
<         data = self.create_object_hook()
---
>         data = {}
595c591
<             dictitems = self.create_object_hook()
---
>             dictitems = {}
602c598
<             dictitems = value.get('dictitems', self.create_object_hook())
---
>             dictitems = value.get('dictitems', {})

loader.py

$ diff loader.py Original
13c13
<     def __init__(self, stream, **constructKwds):
---
>     def __init__(self, stream):
18c18
<         BaseConstructor.__init__(self, **constructKwds)
---
>         BaseConstructor.__init__(self)
23c23
<     def __init__(self, stream, **constructKwds):
---
>     def __init__(self, stream):
28c28
<         SafeConstructor.__init__(self, **constructKwds)
---
>         SafeConstructor.__init__(self)
33c33
<     def __init__(self, stream, **constructKwds):
---
>     def __init__(self, stream):
38c38
<         Constructor.__init__(self, **constructKwds)
---
>         Constructor.__init__(self)

回答 7

这是一个简单的解决方案,还可以检查地图中是否有重复的顶级键。

import yaml
import re
from collections import OrderedDict

def yaml_load_od(fname):
    "load a yaml file as an OrderedDict"
    # detects any duped keys (fail on this) and preserves order of top level keys
    with open(fname, 'r') as f:
        lines = open(fname, "r").read().splitlines()
        top_keys = []
        duped_keys = []
        for line in lines:
            m = re.search(r'^([A-Za-z0-9_]+) *:', line)
            if m:
                if m.group(1) in top_keys:
                    duped_keys.append(m.group(1))
                else:
                    top_keys.append(m.group(1))
        if duped_keys:
            raise Exception('ERROR: duplicate keys: {}'.format(duped_keys))
    # 2nd pass to set up the OrderedDict
    with open(fname, 'r') as f:
        d_tmp = yaml.load(f)
    return OrderedDict([(key, d_tmp[key]) for key in top_keys])

here’s a simple solution that also checks for duplicated top level keys in your map.

import yaml
import re
from collections import OrderedDict

def yaml_load_od(fname):
    "load a yaml file as an OrderedDict"
    # detects any duped keys (fail on this) and preserves order of top level keys
    with open(fname, 'r') as f:
        lines = open(fname, "r").read().splitlines()
        top_keys = []
        duped_keys = []
        for line in lines:
            m = re.search(r'^([A-Za-z0-9_]+) *:', line)
            if m:
                if m.group(1) in top_keys:
                    duped_keys.append(m.group(1))
                else:
                    top_keys.append(m.group(1))
        if duped_keys:
            raise Exception('ERROR: duplicate keys: {}'.format(duped_keys))
    # 2nd pass to set up the OrderedDict
    with open(fname, 'r') as f:
        d_tmp = yaml.load(f)
    return OrderedDict([(key, d_tmp[key]) for key in top_keys])

如何安装适用于Python的yaml软件包?

问题:如何安装适用于Python的yaml软件包?

我有一个使用YAML的Python程序。我尝试使用将其安装在新服务器上pip install yaml,并且返回以下内容:

$ sudo pip install yaml
Downloading/unpacking yaml
  Could not find any downloads that satisfy the requirement yaml
No distributions at all found for yaml
Storing complete log in /home/pa/.pip/pip.log

如何安装适用于Python的yaml软件包?我正在运行Python 2.7。(作业系统:Debian Wheezy)

I have a Python program that uses YAML. I attempted to install it on a new server using pip install yaml and it returns the following:

$ sudo pip install yaml
Downloading/unpacking yaml
  Could not find any downloads that satisfy the requirement yaml
No distributions at all found for yaml
Storing complete log in /home/pa/.pip/pip.log

How do I install the yaml package for Python? I’m running Python 2.7. (OS: Debian Wheezy)


回答 0

您可以尝试点子搜索功能,

$ pip search yaml

它在简短说明中使用yaml在PyPI中查找软件包。这揭示了各种软件包,包括PyYaml,yamltools和PySyck等(请注意,PySyck文档建议使用PyYaml,因为syck已过时)。现在您知道了特定的软件包名称,可以安装它:

$ pip install pyyaml

如果要在linux系统范围内安装python yaml,也可以使用软件包管理器,例如aptitudeyum

$ sudo apt-get install python-yaml
$ sudo yum install python-yaml

You could try the search feature in pip,

$ pip search yaml

which looks for packages in PyPI with yaml in the short description. That reveals various packages, including PyYaml, yamltools, and PySyck, among others (Note that PySyck docs recommend using PyYaml, since syck is out of date). Now you know a specific package name, you can install it:

$ pip install pyyaml

If you want to install python yaml system-wide in linux, you can also use a package manager, like aptitude or yum:

$ sudo apt-get install python-yaml
$ sudo yum install python-yaml

回答 1

pip install pyyaml

如果没有pip,请运行easy_install pip安装pip,这是必备的软件包安装程序- 为什么在easy_install上使用pip?。如果您喜欢坚持使用easy_install,则easy_install pyyaml

pip install pyyaml

If you don’t have pip, run easy_install pip to install pip, which is the go-to package installer – Why use pip over easy_install?. If you prefer to stick with easy_install, then easy_install pyyaml


回答 2

更新:如今,安装已通过pip完成,但仍需要libyaml来构建C扩展(在Mac上):

brew install libyaml
python -m pip install pyyaml

过时的方法

对于MacOSX(小牛),以下方法似乎有效:

brew install libyaml
sudo python -m easy_install pyyaml

Update: Nowadays installing is done with pip, but libyaml is still required to build the C extension (on mac):

brew install libyaml
python -m pip install pyyaml

Outdated method:

For MacOSX (mavericks), the following seems to work:

brew install libyaml
sudo python -m easy_install pyyaml

回答 3

pip install PyYAML

如果找不到libyaml或编译的PyYAML可以在Mavericks上不使用它。

pip install PyYAML

If libyaml is not found or compiled PyYAML can do without it on Mavericks.


回答 4

有三个支持YAML的软件包。Syck(pip install syck),从2002年开始实施YAML 1.0规范;pip install pyyaml遵循2004年的YAML 1.1规范的PyYAML();和ruamel.yaml下面的最新(YAML 1.2,从2009年)规范。

您可以使用以下命令安装YAML 1.2兼容软件包,pip install ruamel.yaml或者如果您正在运行Debian / Ubuntu(或衍生版本)的现代版本,则可以使用:

sudo apt-get install python-ruamel.yaml

There are three YAML capable packages. Syck (pip install syck) which implements the YAML 1.0 specification from 2002; PyYAML (pip install pyyaml) which follows the YAML 1.1 specification from 2004; and ruamel.yaml which follows the latest (YAML 1.2, from 2009) specification.

You can install the YAML 1.2 compatible package with pip install ruamel.yaml or if you are running a modern version of Debian/Ubuntu (or derivative) with:

sudo apt-get install python-ruamel.yaml

回答 5

基于Debian的系统:

$ sudo aptitude install python-yaml

或更高版本的python3

$ sudo aptitude install python3-yaml

Debian-based systems:

$ sudo aptitude install python-yaml

or newer for python3

$ sudo aptitude install python3-yaml


回答 6

以下命令将下载pyyaml,其中还包括yaml

pip install pyYaml

following command will download pyyaml, which also includes yaml

pip install pyYaml

回答 7

“应该有一种-最好只有一种-显而易见的方法。” 因此,我再添加一个。这更像是Debian / Ubuntu的“从源代码安装”,来自https://github.com/yaml/pyyaml

安装libYAML及其标头:

sudo apt-get install libyaml-dev

下载 pyyaml来源:

wget http://pyyaml.org/download/pyyaml/PyYAML-3.13.tar.gz

从源代码安装(不要忘记激活您的venv):

. your/env/bin/activate
tar xzf PyYAML-3.13.tar.gz
cd PyYAML-3.13.tar.gz
(env)$ python setup.py install
(env)$ python setup.py test 

“There should be one — and preferably only one — obvious way to do it.” So let me add another one. This one is more like “install from sources” for Debian/Ubuntu, from https://github.com/yaml/pyyaml

Install the libYAML and it’s headers:

sudo apt-get install libyaml-dev

Download the pyyaml sources:

wget http://pyyaml.org/download/pyyaml/PyYAML-3.13.tar.gz

Install from sources, (don’t forget to activate your venv):

. your/env/bin/activate
tar xzf PyYAML-3.13.tar.gz
cd PyYAML-3.13.tar.gz
(env)$ python setup.py install
(env)$ python setup.py test 

回答 8

使用strictyaml代替

如果您可以自行创建yaml文件,或者不需要常规yaml的任何这些功能,则建议使用strictyaml而不是标准pyyaml软件包。

简而言之,默认yaml在安全性,接口和可预测性方面存在一些严重缺陷。strictyaml是yaml规范的一个子集,没有这些问题(并且有更好的记录)。

您可以在这里阅读更多有关常规Yaml问题的信息

意见: strictyaml应为yaml的默认实现,而旧的yaml规范应作废。

Use strictyaml instead

If you have the luxury of creating the yaml file yourself, or if you don’t require any of these features of regular yaml, I recommend using strictyaml instead of the standard pyyaml package.

In short, default yaml has some serious flaws in terms of security, interface, and predictability. strictyaml is a subset of the yaml spec that does not have those issues (and is better documented).

You can read more about the problems with regular yaml here

OPINION: strictyaml should be the default implementation of yaml and the old yaml spec should be obsoleted.


回答 9

对我来说,安装libyaml的开发版本即可。

yum install libyaml-devel         #centos
apt-get install libyaml-dev       # ubuntu

For me, installing development version of libyaml did it.

yum install libyaml-devel         #centos
apt-get install libyaml-dev       # ubuntu

在Python中使用设置文件的最佳做法是什么?[关闭]

问题:在Python中使用设置文件的最佳做法是什么?[关闭]

我有一个运行有很多参数的命令行脚本。现在到了我有太多参数的地步,我也想以字典形式有一些参数。

因此,为了简化操作,我想使用设置文件来运行脚本。我真的不知道该使用什么库来解析文件。最佳做法是什么?我当然可以自己动手做一些事情,但是如果有图书馆可以帮助我,我会不胜枚举。

一些“需求”:

  • 与其使用pickle我,我不希望它是一个易于阅读和编辑的简单文本文件。
  • 我希望能够在其中添加类似字典的数据,即应支持某种形式的嵌套。

简化的伪示例文件:

truck:
    color: blue
    brand: ford
city: new york
cabriolet:
    color: black
    engine:
        cylinders: 8
        placement: mid
    doors: 2

I have a command line script that I run with a lot of arguments. I have now come to a point where I have too many arguments, and I want to have some arguments in dictionary form too.

So in order to simplify things I would like to run the script with a settings file instead. I don’t really know what libraries to use for the parsing of the file. What’s the best practice for doing this? I could of course hammer something out myself, but if there is some library for this, I’m all ears.

A few ‘demands’:

  • Rather than using pickle I would like it to be a straight forward text file that can easily be read and edited.
  • I want to be able to add dictionary-like data in it, i.e., some form of nesting should be supported.

A simplified pseudo example file:

truck:
    color: blue
    brand: ford
city: new york
cabriolet:
    color: black
    engine:
        cylinders: 8
        placement: mid
    doors: 2

回答 0

您可以有一个常规的Python模块,例如config.py,如下所示:

truck = dict(
    color = 'blue',
    brand = 'ford',
)
city = 'new york'
cabriolet = dict(
    color = 'black',
    engine = dict(
        cylinders = 8,
        placement = 'mid',
    ),
    doors = 2,
)

并像这样使用它:

import config
print config.truck['color']  

You can have a regular Python module, say config.py, like this:

truck = dict(
    color = 'blue',
    brand = 'ford',
)
city = 'new york'
cabriolet = dict(
    color = 'black',
    engine = dict(
        cylinders = 8,
        placement = 'mid',
    ),
    doors = 2,
)

and use it like this:

import config
print config.truck['color']  

回答 1

您提供的样本配置实际上是有效的YAML。实际上,YAML可以满足您的所有需求,并以多种语言实现,并且非常人性化。我强烈建议您使用它。该PyYAML项目提供了一个很好的Python模块,实现YAML。

使用yaml模块非常简单:

import yaml
config = yaml.safe_load(open("path/to/config.yml"))

The sample config you provided is actually valid YAML. In fact, YAML meets all of your demands, is implemented in a large number of languages, and is extremely human friendly. I would highly recommend you use it. The PyYAML project provides a nice python module, that implements YAML.

To use the yaml module is extremely simple:

import yaml
config = yaml.safe_load(open("path/to/config.yml"))

回答 2

我发现这是最有用和易于使用的 https://wiki.python.org/moin/ConfigParserExamples

您只需创建一个“ myfile.ini”,例如:

[SectionOne]
Status: Single
Name: Derek
Value: Yes
Age: 30
Single: True

[SectionTwo]
FavoriteColor=Green
[SectionThree]
FamilyName: Johnson

[Others]
Route: 66

并像这样检索数据:

>>> import ConfigParser
>>> Config = ConfigParser.ConfigParser()
>>> Config
<ConfigParser.ConfigParser instance at 0x00BA9B20>
>>> Config.read("myfile.ini")
['c:\\tomorrow.ini']
>>> Config.sections()
['Others', 'SectionThree', 'SectionOne', 'SectionTwo']
>>> Config.options('SectionOne')
['Status', 'Name', 'Value', 'Age', 'Single']
>>> Config.get('SectionOne', 'Status')
'Single'

I Found this the most useful and easy to use https://wiki.python.org/moin/ConfigParserExamples

You just create a “myfile.ini” like:

[SectionOne]
Status: Single
Name: Derek
Value: Yes
Age: 30
Single: True

[SectionTwo]
FavoriteColor=Green
[SectionThree]
FamilyName: Johnson

[Others]
Route: 66

And retrieve the data like:

>>> import ConfigParser
>>> Config = ConfigParser.ConfigParser()
>>> Config
<ConfigParser.ConfigParser instance at 0x00BA9B20>
>>> Config.read("myfile.ini")
['c:\\tomorrow.ini']
>>> Config.sections()
['Others', 'SectionThree', 'SectionOne', 'SectionTwo']
>>> Config.options('SectionOne')
['Status', 'Name', 'Value', 'Age', 'Single']
>>> Config.get('SectionOne', 'Status')
'Single'

回答 3

Yaml和Json是存储设置/配置的最简单,最常用的文件格式。PyYaml可用于解析yaml。Json已经从2.5开始成为python的一部分。Yaml是Json的超集。Json将解决大多数使用情况,但需要转义的多行字符串除外。Yaml也会处理这些情况。

>>> import json
>>> config = {'handler' : 'adminhandler.py', 'timeoutsec' : 5 }
>>> json.dump(config, open('/tmp/config.json', 'w'))
>>> json.load(open('/tmp/config.json'))   
{u'handler': u'adminhandler.py', u'timeoutsec': 5}

Yaml and Json are the simplest and most commonly used file formats to store settings/config. PyYaml can be used to parse yaml. Json is already part of python from 2.5. Yaml is a superset of Json. Json will solve most uses cases except multi line strings where escaping is required. Yaml takes care of these cases too.

>>> import json
>>> config = {'handler' : 'adminhandler.py', 'timeoutsec' : 5 }
>>> json.dump(config, open('/tmp/config.json', 'w'))
>>> json.load(open('/tmp/config.json'))   
{u'handler': u'adminhandler.py', u'timeoutsec': 5}

如何在Python中解析YAML文件

问题:如何在Python中解析YAML文件

如何在Python中解析YAML文件?

How can I parse a YAML file in Python?


回答 0

不依赖C标头的最简单,最纯净的方法是PyYaml(文档),可以通过pip install pyyaml以下方式安装:

#!/usr/bin/env python

import yaml

with open("example.yaml", 'r') as stream:
    try:
        print(yaml.safe_load(stream))
    except yaml.YAMLError as exc:
        print(exc)

就是这样。一个普通的yaml.load()函数也存在,但是yaml.safe_load()除非您明确需要提供的任意对象序列化/反序列化,以避免引入执行任意代码的可能性,否则应始终首选该函数。

请注意,PyYaml项目支持YAML 1.1规范之前的版本。如果需要YAML 1.2规范支持,请参见ruamel.yaml,本答案中所述

The easiest and purest method without relying on C headers is PyYaml (documentation), which can be installed via pip install pyyaml:

#!/usr/bin/env python

import yaml

with open("example.yaml", 'r') as stream:
    try:
        print(yaml.safe_load(stream))
    except yaml.YAMLError as exc:
        print(exc)

And that’s it. A plain yaml.load() function also exists, but yaml.safe_load() should always be preferred unless you explicitly need the arbitrary object serialization/deserialization provided in order to avoid introducing the possibility for arbitrary code execution.

Note the PyYaml project supports versions up through the YAML 1.1 specification. If YAML 1.2 specification support is needed, see ruamel.yaml as noted in this answer.


回答 1

使用Python 2 + 3(和Unicode)读写YAML文件

# -*- coding: utf-8 -*-
import yaml
import io

# Define data
data = {
    'a list': [
        1, 
        42, 
        3.141, 
        1337, 
        'help', 
        u'€'
    ],
    'a string': 'bla',
    'another dict': {
        'foo': 'bar',
        'key': 'value',
        'the answer': 42
    }
}

# Write YAML file
with io.open('data.yaml', 'w', encoding='utf8') as outfile:
    yaml.dump(data, outfile, default_flow_style=False, allow_unicode=True)

# Read YAML file
with open("data.yaml", 'r') as stream:
    data_loaded = yaml.safe_load(stream)

print(data == data_loaded)

创建的YAML文件

a list:
- 1
- 42
- 3.141
- 1337
- help
- 
a string: bla
another dict:
  foo: bar
  key: value
  the answer: 42

通用文件结尾

.yml.yaml

备择方案

对于您的应用程序,以下内容可能很重要:

  • 其他编程语言的支持
  • 阅读/写作表现
  • 紧凑度(文件大小)

另请参阅:数据序列化格式的比较

如果您想寻找一种制作配置文件的方法,则可能需要阅读我的短文《Python中的配置文件》。

Read & Write YAML files with Python 2+3 (and unicode)

# -*- coding: utf-8 -*-
import yaml
import io

# Define data
data = {
    'a list': [
        1, 
        42, 
        3.141, 
        1337, 
        'help', 
        u'€'
    ],
    'a string': 'bla',
    'another dict': {
        'foo': 'bar',
        'key': 'value',
        'the answer': 42
    }
}

# Write YAML file
with io.open('data.yaml', 'w', encoding='utf8') as outfile:
    yaml.dump(data, outfile, default_flow_style=False, allow_unicode=True)

# Read YAML file
with open("data.yaml", 'r') as stream:
    data_loaded = yaml.safe_load(stream)

print(data == data_loaded)

Created YAML file

a list:
- 1
- 42
- 3.141
- 1337
- help
- €
a string: bla
another dict:
  foo: bar
  key: value
  the answer: 42

Common file endings

.yml and .yaml

Alternatives

For your application, the following might be important:

  • Support by other programming languages
  • Reading / writing performance
  • Compactness (file size)

See also: Comparison of data serialization formats

In case you are rather looking for a way to make configuration files, you might want to read my short article Configuration files in Python


回答 2

如果您具有符合YAML 1.2规范(2009年发布)的YAML,则应使用ruamel.yaml(免责声明:我是该软件包的作者)。它本质上是PyYAML的超集,它支持大多数YAML 1.1(自2005年起)。

如果希望在往返时保留您的注释,则当然应该使用ruamel.yaml。

升级@Jon的示例很容易:

import ruamel.yaml as yaml

with open("example.yaml") as stream:
    try:
        print(yaml.safe_load(stream))
    except yaml.YAMLError as exc:
        print(exc)

safe_load()除非您真的完全控制了输入,否则就使用它(很少),并且知道您在做什么。

如果您使用pathlib Path来处理文件,则最好使用新的ruamel.yaml API:

from ruamel.yaml import YAML
from pathlib import Path

path = Path('example.yaml')
yaml = YAML(typ='safe')
data = yaml.load(path)

If you have YAML that conforms to the YAML 1.2 specification (released 2009) then you should use ruamel.yaml (disclaimer: I am the author of that package). It is essentially a superset of PyYAML, which supports most of YAML 1.1 (from 2005).

If you want to be able to preserve your comments when round-tripping, you certainly should use ruamel.yaml.

Upgrading @Jon’s example is easy:

import ruamel.yaml as yaml

with open("example.yaml") as stream:
    try:
        print(yaml.safe_load(stream))
    except yaml.YAMLError as exc:
        print(exc)

Use safe_load() unless you really have full control over the input, need it (seldom the case) and know what you are doing.

If you are using pathlib Path for manipulating files, you are better of using the new API ruamel.yaml provides:

from ruamel.yaml import YAML
from pathlib import Path

path = Path('example.yaml')
yaml = YAML(typ='safe')
data = yaml.load(path)

回答 3

首先使用pip3安装pyyaml。

然后导入yaml模块并将文件加载到名为“ my_dict”的字典中:

import yaml
with open('filename.yaml') as f:
    my_dict = yaml.safe_load(f)

这就是您所需要的。现在,整个yaml文件都在“ my_dict”字典中。

First install pyyaml using pip3.

Then import yaml module and load the file into a dictionary called ‘my_dict’:

import yaml
with open('filename.yaml') as f:
    my_dict = yaml.safe_load(f)

That’s all you need. Now the entire yaml file is in ‘my_dict’ dictionary.


回答 4

例:


defaults.yaml

url: https://www.google.com

环境

from ruamel import yaml

data = yaml.safe_load(open('defaults.yaml'))
data['url']

Example:


defaults.yaml

url: https://www.google.com

environment.py

from ruamel import yaml

data = yaml.safe_load(open('defaults.yaml'))
data['url']

回答 5

我使用ruamel.yaml详情和辩论在这里

from ruamel import yaml

with open(filename, 'r') as fp:
    read_data = yaml.load(fp)

用法ruamel.yaml是PyYAML的旧惯例兼容(有一些简单的可解决的问题),并因为它是在链接说明我公司提供,使用

from ruamel import yaml

代替

import yaml

它将解决您的大多数问题。

编辑:事实证明PyYAML并没有死,它只是保存在另一个地方。

I use ruamel.yaml. Details & debate here.

from ruamel import yaml

with open(filename, 'r') as fp:
    read_data = yaml.load(fp)

Usage of ruamel.yaml is compatible (with some simple solvable problems) with old usages of PyYAML and as it is stated in link I provided, use

from ruamel import yaml

instead of

import yaml

and it will fix most of your problems.

EDIT: PyYAML is not dead as it turns out, it’s just maintained in a different place.


回答 6

#!/usr/bin/env python

import sys
import yaml

def main(argv):

    with open(argv[0]) as stream:
        try:
            #print(yaml.load(stream))
            return 0
        except yaml.YAMLError as exc:
            print(exc)
            return 1

if __name__ == "__main__":
    sys.exit(main(sys.argv[1:]))
#!/usr/bin/env python

import sys
import yaml

def main(argv):

    with open(argv[0]) as stream:
        try:
            #print(yaml.load(stream))
            return 0
        except yaml.YAMLError as exc:
            print(exc)
            return 1

if __name__ == "__main__":
    sys.exit(main(sys.argv[1:]))