在config.py中提供全局配置变量的最Pythonic方法?[关闭]

问题:在config.py中提供全局配置变量的最Pythonic方法?[关闭]

在我对过度复杂的简单事物的无尽追求中,我正在研究最“ Pythonic”的方法来在Python egg包中的典型“ config.py ”中提供全局配置变量。

传统方式(啊,好吧,# define!)如下:

MYSQL_PORT = 3306
MYSQL_DATABASE = 'mydb'
MYSQL_DATABASE_TABLES = ['tb_users', 'tb_groups']

因此,以下列方式之一导入全局变量:

from config import *
dbname = MYSQL_DATABASE
for table in MYSQL_DATABASE_TABLES:
    print table

要么:

import config
dbname = config.MYSQL_DATABASE
assert(isinstance(config.MYSQL_PORT, int))

这是有道理的,但有时可能会有些混乱,尤其是在您要记住某些变量的名称时。此外,提供一个以变量为属性“配置”对象可能更灵活。因此,从bpython config.py文件开始,我想到了:

class Struct(object):

    def __init__(self, *args):
        self.__header__ = str(args[0]) if args else None

    def __repr__(self):
        if self.__header__ is None:
             return super(Struct, self).__repr__()
        return self.__header__

    def next(self):
        """ Fake iteration functionality.
        """
        raise StopIteration

    def __iter__(self):
        """ Fake iteration functionality.
        We skip magic attribues and Structs, and return the rest.
        """
        ks = self.__dict__.keys()
        for k in ks:
            if not k.startswith('__') and not isinstance(k, Struct):
                yield getattr(self, k)

    def __len__(self):
        """ Don't count magic attributes or Structs.
        """
        ks = self.__dict__.keys()
        return len([k for k in ks if not k.startswith('__')\
                    and not isinstance(k, Struct)])

和一个“ config.py”,该类导入该类,内容如下:

from _config import Struct as Section

mysql = Section("MySQL specific configuration")
mysql.user = 'root'
mysql.pass = 'secret'
mysql.host = 'localhost'
mysql.port = 3306
mysql.database = 'mydb'

mysql.tables = Section("Tables for 'mydb'")
mysql.tables.users = 'tb_users'
mysql.tables.groups =  'tb_groups'

并以这种方式使用:

from sqlalchemy import MetaData, Table
import config as CONFIG

assert(isinstance(CONFIG.mysql.port, int))

mdata = MetaData(
    "mysql://%s:%s@%s:%d/%s" % (
         CONFIG.mysql.user,
         CONFIG.mysql.pass,
         CONFIG.mysql.host,
         CONFIG.mysql.port,
         CONFIG.mysql.database,
     )
)

tables = []
for name in CONFIG.mysql.tables:
    tables.append(Table(name, mdata, autoload=True))

这似乎是在包内存储和获取全局变量的一种更具可读性,表现力和灵活性的方式。

有史以来最大的想法?应对这些情况的最佳实践是什么?什么是您的存储和获取全局名称和变量您的包内的方法吗?

In my endless quest in over-complicating simple stuff, I am researching the most ‘Pythonic’ way to provide global configuration variables inside the typical ‘config.py‘ found in Python egg packages.

The traditional way (aah, good ol’ #define!) is as follows:

MYSQL_PORT = 3306
MYSQL_DATABASE = 'mydb'
MYSQL_DATABASE_TABLES = ['tb_users', 'tb_groups']

Therefore global variables are imported in one of the following ways:

from config import *
dbname = MYSQL_DATABASE
for table in MYSQL_DATABASE_TABLES:
    print table

or:

import config
dbname = config.MYSQL_DATABASE
assert(isinstance(config.MYSQL_PORT, int))

It makes sense, but sometimes can be a little messy, especially when you’re trying to remember the names of certain variables. Besides, providing a ‘configuration’ object, with variables as attributes, might be more flexible. So, taking a lead from bpython config.py file, I came up with:

class Struct(object):

    def __init__(self, *args):
        self.__header__ = str(args[0]) if args else None

    def __repr__(self):
        if self.__header__ is None:
             return super(Struct, self).__repr__()
        return self.__header__

    def next(self):
        """ Fake iteration functionality.
        """
        raise StopIteration

    def __iter__(self):
        """ Fake iteration functionality.
        We skip magic attribues and Structs, and return the rest.
        """
        ks = self.__dict__.keys()
        for k in ks:
            if not k.startswith('__') and not isinstance(k, Struct):
                yield getattr(self, k)

    def __len__(self):
        """ Don't count magic attributes or Structs.
        """
        ks = self.__dict__.keys()
        return len([k for k in ks if not k.startswith('__')\
                    and not isinstance(k, Struct)])

and a ‘config.py’ that imports the class and reads as follows:

from _config import Struct as Section

mysql = Section("MySQL specific configuration")
mysql.user = 'root'
mysql.pass = 'secret'
mysql.host = 'localhost'
mysql.port = 3306
mysql.database = 'mydb'

mysql.tables = Section("Tables for 'mydb'")
mysql.tables.users = 'tb_users'
mysql.tables.groups =  'tb_groups'

and is used in this way:

from sqlalchemy import MetaData, Table
import config as CONFIG

assert(isinstance(CONFIG.mysql.port, int))

mdata = MetaData(
    "mysql://%s:%s@%s:%d/%s" % (
         CONFIG.mysql.user,
         CONFIG.mysql.pass,
         CONFIG.mysql.host,
         CONFIG.mysql.port,
         CONFIG.mysql.database,
     )
)

tables = []
for name in CONFIG.mysql.tables:
    tables.append(Table(name, mdata, autoload=True))

Which seems a more readable, expressive and flexible way of storing and fetching global variables inside a package.

Lamest idea ever? What is the best practice for coping with these situations? What is your way of storing and fetching global names and variables inside your package?


回答 0

我做了一次。最终,我发现简化的basicconfig.py可以满足我的需求。如果需要,您可以将命名空间与其他对象一起传递以供其引用。您还可以从代码中传递其他默认值。它还将属性和映射样式语法映射到同一配置对象。

I did that once. Ultimately I found my simplified basicconfig.py adequate for my needs. You can pass in a namespace with other objects for it to reference if you need to. You can also pass in additional defaults from your code. It also maps attribute and mapping style syntax to the same configuration object.


回答 1

只使用这样的内置类型怎么样:

config = {
    "mysql": {
        "user": "root",
        "pass": "secret",
        "tables": {
            "users": "tb_users"
        }
        # etc
    }
}

您可以按以下方式访问这些值:

config["mysql"]["tables"]["users"]

如果您愿意牺牲潜力在配置树中计算表达式,则可以使用YAML并得到一个更具可读性的配置文件,如下所示:

mysql:
  - user: root
  - pass: secret
  - tables:
    - users: tb_users

并使用PyYAML之类的库方便地解析和访问配置文件

How about just using the built-in types like this:

config = {
    "mysql": {
        "user": "root",
        "pass": "secret",
        "tables": {
            "users": "tb_users"
        }
        # etc
    }
}

You’d access the values as follows:

config["mysql"]["tables"]["users"]

If you are willing to sacrifice the potential to compute expressions inside your config tree, you could use YAML and end up with a more readable config file like this:

mysql:
  - user: root
  - pass: secret
  - tables:
    - users: tb_users

and use a library like PyYAML to conventiently parse and access the config file


回答 2

我喜欢用于小型应用程序的解决方案:

class App:
  __conf = {
    "username": "",
    "password": "",
    "MYSQL_PORT": 3306,
    "MYSQL_DATABASE": 'mydb',
    "MYSQL_DATABASE_TABLES": ['tb_users', 'tb_groups']
  }
  __setters = ["username", "password"]

  @staticmethod
  def config(name):
    return App.__conf[name]

  @staticmethod
  def set(name, value):
    if name in App.__setters:
      App.__conf[name] = value
    else:
      raise NameError("Name not accepted in set() method")

然后用法是:

if __name__ == "__main__":
   # from config import App
   App.config("MYSQL_PORT")     # return 3306
   App.set("username", "hi")    # set new username value
   App.config("username")       # return "hi"
   App.set("MYSQL_PORT", "abc") # this raises NameError

..您应该喜欢它,因为:

  • 使用类变量(无需传递对象/无需单例),
  • 使用封装的内置类型,看起来像是在上的方法调用App
  • 可以控制个人配置的不变性可变全局变量是最差的全局变量
  • 在您的源代码中提高常规名称的访问/可读性
  • 是一个简单的类,但是强制进行结构化访问,一种替代方法是使用@property,但是每个项目需要更多的变量处理代码,并且是基于对象的。
  • 只需进行最小的更改即可添加新的配置项并设置其可变性。

-编辑-:对于大型应用程序,将值存储在YAML(即属性)文件中并将其作为不可变数据读取是一种更好的方法(即blubb / ohaal的答案)。对于小型应用程序,上面的解决方案更简单。

I like this solution for small applications:

class App:
  __conf = {
    "username": "",
    "password": "",
    "MYSQL_PORT": 3306,
    "MYSQL_DATABASE": 'mydb',
    "MYSQL_DATABASE_TABLES": ['tb_users', 'tb_groups']
  }
  __setters = ["username", "password"]

  @staticmethod
  def config(name):
    return App.__conf[name]

  @staticmethod
  def set(name, value):
    if name in App.__setters:
      App.__conf[name] = value
    else:
      raise NameError("Name not accepted in set() method")

And then usage is:

if __name__ == "__main__":
   # from config import App
   App.config("MYSQL_PORT")     # return 3306
   App.set("username", "hi")    # set new username value
   App.config("username")       # return "hi"
   App.set("MYSQL_PORT", "abc") # this raises NameError

.. you should like it because:

  • uses class variables (no object to pass around/ no singleton required),
  • uses encapsulated built-in types and looks like (is) a method call on App,
  • has control over individual config immutability, mutable globals are the worst kind of globals.
  • promotes conventional and well named access / readability in your source code
  • is a simple class but enforces structured access, an alternative is to use @property, but that requires more variable handling code per item and is object-based.
  • requires minimal changes to add new config items and set its mutability.

–Edit–: For large applications, storing values in a YAML (i.e. properties) file and reading that in as immutable data is a better approach (i.e. blubb/ohaal’s answer). For small applications, this solution above is simpler.


回答 3

使用类怎么样?

# config.py
class MYSQL:
    PORT = 3306
    DATABASE = 'mydb'
    DATABASE_TABLES = ['tb_users', 'tb_groups']

# main.py
from config import MYSQL

print(MYSQL.PORT) # 3306

How about using classes?

# config.py
class MYSQL:
    PORT = 3306
    DATABASE = 'mydb'
    DATABASE_TABLES = ['tb_users', 'tb_groups']

# main.py
from config import MYSQL

print(MYSQL.PORT) # 3306

回答 4

类似于blubb的答案。我建议使用lambda函数构建它们以减少代码。像这样:

User = lambda passwd, hair, name: {'password':passwd, 'hair':hair, 'name':name}

#Col      Username       Password      Hair Color  Real Name
config = {'st3v3' : User('password',   'blonde',   'Steve Booker'),
          'blubb' : User('12345678',   'black',    'Bubb Ohaal'),
          'suprM' : User('kryptonite', 'black',    'Clark Kent'),
          #...
         }
#...

config['st3v3']['password']  #> password
config['blubb']['hair']      #> black

不过,这确实闻起来像您可能想上一堂课。

或者,如MarkM所述,您可以使用 namedtuple

from collections import namedtuple
#...

User = namedtuple('User', ['password', 'hair', 'name']}

#Col      Username       Password      Hair Color  Real Name
config = {'st3v3' : User('password',   'blonde',   'Steve Booker'),
          'blubb' : User('12345678',   'black',    'Bubb Ohaal'),
          'suprM' : User('kryptonite', 'black',    'Clark Kent'),
          #...
         }
#...

config['st3v3'].password   #> passwd
config['blubb'].hair       #> black

Similar to blubb’s answer. I suggest building them with lambda functions to reduce code. Like this:

User = lambda passwd, hair, name: {'password':passwd, 'hair':hair, 'name':name}

#Col      Username       Password      Hair Color  Real Name
config = {'st3v3' : User('password',   'blonde',   'Steve Booker'),
          'blubb' : User('12345678',   'black',    'Bubb Ohaal'),
          'suprM' : User('kryptonite', 'black',    'Clark Kent'),
          #...
         }
#...

config['st3v3']['password']  #> password
config['blubb']['hair']      #> black

This does smell like you may want to make a class, though.

Or, as MarkM noted, you could use namedtuple

from collections import namedtuple
#...

User = namedtuple('User', ['password', 'hair', 'name']}

#Col      Username       Password      Hair Color  Real Name
config = {'st3v3' : User('password',   'blonde',   'Steve Booker'),
          'blubb' : User('12345678',   'black',    'Bubb Ohaal'),
          'suprM' : User('kryptonite', 'black',    'Clark Kent'),
          #...
         }
#...

config['st3v3'].password   #> passwd
config['blubb'].hair       #> black

回答 5

我使用的赫斯基想法略有不同。创建一个名为“ globals”(或您喜欢的文件)的文件,然后在其中定义多个类,如下所示:

#globals.py

class dbinfo :      # for database globals
    username = 'abcd'
    password = 'xyz'

class runtime :
    debug = False
    output = 'stdio'

然后,如果您有两个代码文件c1.py和c2.py,则两者都可以位于顶部

import globals as gl

现在,所有代码都可以访问和设置值,如下所示:

gl.runtime.debug = False
print(gl.dbinfo.username)

人们会忘记存在类,即使没有实例化属于该类成员的对象也是如此。并且类中没有“自我”的变量。在类的所有实例之间共享,即使没有实例也是如此。一旦任何代码更改了“调试”,所有其他代码都将看到更改。

通过将其导入为gl,您可以拥有多个这样的文件和变量,使您可以跨代码文件,函数等访问和设置值,但不会发生命名空间冲突的危险。

这缺少其他方法的一些聪明的错误检查,但是简单易行。

A small variation on Husky’s idea that I use. Make a file called ‘globals’ (or whatever you like) and then define multiple classes in it, as such:

#globals.py

class dbinfo :      # for database globals
    username = 'abcd'
    password = 'xyz'

class runtime :
    debug = False
    output = 'stdio'

Then, if you have two code files c1.py and c2.py, both can have at the top

import globals as gl

Now all code can access and set values, as such:

gl.runtime.debug = False
print(gl.dbinfo.username)

People forget classes exist, even if no object is ever instantiated that is a member of that class. And variables in a class that aren’t preceded by ‘self.’ are shared across all instances of the class, even if there are none. Once ‘debug’ is changed by any code, all other code sees the change.

By importing it as gl, you can have multiple such files and variables that lets you access and set values across code files, functions, etc., but with no danger of namespace collision.

This lacks some of the clever error checking of other approaches, but is simple and easy to follow.


回答 6

坦白地说,我们可能应该考虑使用Python Software Foundation维护的库:

https://docs.python.org/3/library/configparser.html

配置示例:(ini格式,但可用JSON)

[DEFAULT]
ServerAliveInterval = 45
Compression = yes
CompressionLevel = 9
ForwardX11 = yes

[bitbucket.org]
User = hg

[topsecret.server.com]
Port = 50022
ForwardX11 = no

代码示例:

>>> import configparser
>>> config = configparser.ConfigParser()
>>> config.read('example.ini')
>>> config['DEFAULT']['Compression']
'yes'
>>> config['DEFAULT'].getboolean('MyCompression', fallback=True) # get_or_else

使其可全局访问:

import configpaser
class App:
 __conf = None

 @staticmethod
 def config():
  if App.__conf is None:  # Read only once, lazy.
   App.__conf = configparser.ConfigParser()
   App.__conf.read('example.ini')
  return App.__conf

if __name__ == '__main__':
 App.config()['DEFAULT']['MYSQL_PORT']
 # or, better:
 App.config().get(section='DEFAULT', option='MYSQL_PORT', fallback=3306)
 ....

缺点:

  • 不受控制的全局可变状态。

Let’s be honest, we should probably consider using a Python Software Foundation maintained library:

https://docs.python.org/3/library/configparser.html

Config example: (ini format, but JSON available)

[DEFAULT]
ServerAliveInterval = 45
Compression = yes
CompressionLevel = 9
ForwardX11 = yes

[bitbucket.org]
User = hg

[topsecret.server.com]
Port = 50022
ForwardX11 = no

Code example:

>>> import configparser
>>> config = configparser.ConfigParser()
>>> config.read('example.ini')
>>> config['DEFAULT']['Compression']
'yes'
>>> config['DEFAULT'].getboolean('MyCompression', fallback=True) # get_or_else

Making it globally-accessible:

import configpaser
class App:
 __conf = None

 @staticmethod
 def config():
  if App.__conf is None:  # Read only once, lazy.
   App.__conf = configparser.ConfigParser()
   App.__conf.read('example.ini')
  return App.__conf

if __name__ == '__main__':
 App.config()['DEFAULT']['MYSQL_PORT']
 # or, better:
 App.config().get(section='DEFAULT', option='MYSQL_PORT', fallback=3306)
 ....

Downsides:

  • Uncontrolled global mutable state.

回答 7

请检出通过traitlet实现的IPython配置系统,以实现您正在手动执行的类型强制。

在此处进行剪切和粘贴,以符合SO准则,而不仅仅是随着链接的内容随时间变化而删除链接。

特征文档

这是我们希望我们的配置系统具有的主要要求:

支持分层配置信息。

与命令行选项解析器完全集成。通常,您想读取配置文件,然后使用命令行选项覆盖某些值。我们的配置系统使该过程自动化,并允许将每个命令行选项链接到将被覆盖的配置层次结构中的特定属性。

配置文件本身就是有效的Python代码。这完成了很多事情。首先,可以将逻辑放入配置文件中,以根据操作系统,网络设置,Python版本等设置属性。其次,Python具有用于访问分层数据结构的超简单语法,即常规属性访问(Foo。 Bar.Bam.name)。第三,使用Python可使用户轻松地将配置属性从一个配置文件导入到另一个。第四,即使Python是动态类型的,它也确实具有可以在运行时检查的类型。因此,配置文件中的1是整数’1’,而’1’是字符串。

一种在运行时将配置信息获取到需要它的类的全自动方法。编写遍历配置层次结构以提取特定属性的代码很痛苦。当您具有包含数百个属性的复杂配置信息时,这会让您想哭。

类型检查和验证不需要在运行时之前静态地指定整个配置层次结构。Python是一种非常动态的语言,您并不总是知道程序启动时需要配置的所有内容。

为此,他们基本上定义了3个对象类以及它们之间的关系:

1)配置-基本上是ChainMap /基本dict,具有一些用于合并的增强功能。

2)可配置-基类可将您要配置的所有内容都子类化。

3)应用程序-实例化以执行特定应用程序功能的对象,或用于单一目的软件的主应用程序。

用他们的话说:

应用:应用

应用程序是执行特定工作的过程。最明显的应用是ipython命令行程序。每个应用程序都读取一个或多个配置文件和一组命令行选项,然后为该应用程序生成一个主配置对象。然后,此配置对象将传递到应用程序创建的可配置对象。这些可配置对象实现了应用程序的实际逻辑,并且知道如何在给定配置对象的情况下进行自我配置。

应用程序始终具有配置为Logger的日志属性。这允许对每个应用程序进行集中式日志记录配置。可配置:可配置

可配置的是常规Python类,它充当应用程序中所有主要类的基类。可配置基类是轻量级的,只能做一件事。

此Configurable是HasTraits的子类,它知道如何进行自我配置。具有元数据config = True的类级别特征变为可以从命令行和配置文件配置的值。

开发人员创建可配置的子类,以实现应用程序中的所有逻辑。这些子类中的每一个都有其自己的配置信息,该信息控制如何创建实例。

please check out the IPython configuration system, implemented via traitlets for the type enforcement you are doing manually.

Cut and pasted here to comply with SO guidelines for not just dropping links as the content of links changes over time.

traitlets documentation

Here are the main requirements we wanted our configuration system to have:

Support for hierarchical configuration information.

Full integration with command line option parsers. Often, you want to read a configuration file, but then override some of the values with command line options. Our configuration system automates this process and allows each command line option to be linked to a particular attribute in the configuration hierarchy that it will override.

Configuration files that are themselves valid Python code. This accomplishes many things. First, it becomes possible to put logic in your configuration files that sets attributes based on your operating system, network setup, Python version, etc. Second, Python has a super simple syntax for accessing hierarchical data structures, namely regular attribute access (Foo.Bar.Bam.name). Third, using Python makes it easy for users to import configuration attributes from one configuration file to another. Fourth, even though Python is dynamically typed, it does have types that can be checked at runtime. Thus, a 1 in a config file is the integer ‘1’, while a ‘1’ is a string.

A fully automated method for getting the configuration information to the classes that need it at runtime. Writing code that walks a configuration hierarchy to extract a particular attribute is painful. When you have complex configuration information with hundreds of attributes, this makes you want to cry.

Type checking and validation that doesn’t require the entire configuration hierarchy to be specified statically before runtime. Python is a very dynamic language and you don’t always know everything that needs to be configured when a program starts.

To acheive this they basically define 3 object classes and their relations to each other:

1) Configuration – basically a ChainMap / basic dict with some enhancements for merging.

2) Configurable – base class to subclass all things you’d wish to configure.

3) Application – object that is instantiated to perform a specific application function, or your main application for single purpose software.

In their words:

Application: Application

An application is a process that does a specific job. The most obvious application is the ipython command line program. Each application reads one or more configuration files and a single set of command line options and then produces a master configuration object for the application. This configuration object is then passed to the configurable objects that the application creates. These configurable objects implement the actual logic of the application and know how to configure themselves given the configuration object.

Applications always have a log attribute that is a configured Logger. This allows centralized logging configuration per-application. Configurable: Configurable

A configurable is a regular Python class that serves as a base class for all main classes in an application. The Configurable base class is lightweight and only does one things.

This Configurable is a subclass of HasTraits that knows how to configure itself. Class level traits with the metadata config=True become values that can be configured from the command line and configuration files.

Developers create Configurable subclasses that implement all of the logic in the application. Each of these subclasses has its own configuration information that controls how instances are created.