Python 实用宝典

Question 1

Originally I wanted to ask this question, but then I found it was already thought of before…

Googling around I found this example of extending configparser. The following works with Python 3:

$ python3
Python 3.2.3rc2 (default, Mar 21 2012, 06:59:51) 
[GCC 4.6.3] on linux2
>>> from configparser import  SafeConfigParser
>>> class AmritaConfigParser(SafeConfigParser):
...     def __init_(self):
...         super().__init__()
... 
>>> cfg = AmritaConfigParser()

But not with Python 2:

>>> class AmritaConfigParser(SafeConfigParser):
...       def __init__(self):
...           super(SafeConfigParser).init()
... 
>>> cfg = AmritaConfigParser()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<stdin>", line 3, in __init__
TypeError: must be type, not classob

Then I read a little bit on Python New Class vs. Old Class styles (e.g. here. And now I am wondering, I can do:

class MyConfigParser(ConfigParser.ConfigParser):
      def Write(self, fp):
          """override the module's original write funcition"""
          ....
      def MyWrite(self, fp):
          """Define new function and inherit all others"""

But, shouldn’t I call init? Is this in Python 2 the equivalent:

 class AmritaConfigParser(ConfigParser.SafeConfigParser):
    #def __init__(self):
    #    super().__init__() # Python3 syntax, or rather, new style class syntax ...
    #
    # is this the equivalent of the above ? 
    def __init__(self):
        ConfigParser.SafeConfigParser.__init__(self)

Question 2

super() (without arguments) was introduced in Python 3 (along with __class__):
```
super() -> same as super(__class__, self)
```
so that would be the Python 2 equivalent for new-style classes:
```
super(CurrentClass, self)
```

for old-style classes you can always use:

 class Classname(OldStyleParent):
    def __init__(self, *args, **kwargs):
        OldStyleParent.__init__(self, *args, **kwargs)

Question 3

In a single inheritance case (when you subclass one class only), your new class inherits methods of the base class. This includes __init__. So if you don’t define it in your class, you will get the one from the base.

Things start being complicated if you introduce multiple inheritance (subclassing more than one class at a time). This is because if more than one base class has __init__, your class will inherit the first one only.

In such cases, you should really use super if you can, I’ll explain why. But not always you can. The problem is that all your base classes must also use it (and their base classes as well — the whole tree).

If that is the case, then this will also work correctly (in Python 3 but you could rework it into Python 2 — it also has super):

class A:
    def __init__(self):
        print('A')
        super().__init__()

class B:
    def __init__(self):
        print('B')
        super().__init__()

class C(A, B):
    pass

C()
#prints:
#A
#B

Notice how both base classes use super even though they don’t have their own base classes.

What super does is: it calls the method from the next class in MRO (method resolution order). The MRO for C is: (C, A, B, object). You can print C.__mro__ to see it.

So, C inherits __init__ from A and super in A.__init__ calls B.__init__ (B follows A in MRO).

So by doing nothing in C, you end up calling both, which is what you want.

Now if you were not using super, you would end up inheriting A.__init__ (as before) but this time there’s nothing that would call B.__init__ for you.

class A:
    def __init__(self):
        print('A')

class B:
    def __init__(self):
        print('B')

class C(A, B):
    pass

C()
#prints:
#A

To fix that you have to define C.__init__:

class C(A, B):
    def __init__(self):
        A.__init__(self)
        B.__init__(self)

The problem with that is that in more complicated MI trees, __init__ methods of some classes may end up being called more than once whereas super/MRO guarantee that they’re called just once.

Question 4

In short, they are equivalent. Let’s have a history view:

(1) at first, the function looks like this.

    class MySubClass(MySuperClass):
        def __init__(self):
            MySuperClass.__init__(self)

(2) to make code more abstract (and more portable). A common method to get Super-Class is invented like:

    super(<class>, <instance>)

And init function can be:

    class MySubClassBetter(MySuperClass):
        def __init__(self):
            super(MySubClassBetter, self).__init__()

However requiring an explicit passing of both the class and instance break the DRY (Don’t Repeat Yourself) rule a bit.

(3) in V3. It is more smart,

    super()

is enough in most case. You can refer to http://www.python.org/dev/peps/pep-3135/

Question 5

Just to have a simple and complete example for Python 3, which most people seem to be using now.

class MySuper(object):
    def __init__(self,a):
        self.a = a

class MySub(MySuper):
    def __init__(self,a,b):
        self.b = b
        super().__init__(a)

my_sub = MySub(42,'chickenman')
print(my_sub.a)
print(my_sub.b)

gives

42
chickenman

Question 6

Another python3 implementation that involves the use of Abstract classes with super(). You should remember that

super().__init__(name, 10)

has the same effect as

Person.__init__(self, name, 10)

Remember there’s a hidden ‘self’ in super(), So the same object passes on to the superclass init method and the attributes are added to the object that called it. Hence super()gets translated to Person and then if you include the hidden self, you get the above code frag.

from abc import ABCMeta, abstractmethod
class Person(metaclass=ABCMeta):
    name = ""
    age = 0

    def __init__(self, personName, personAge):
        self.name = personName
        self.age = personAge

    @abstractmethod
    def showName(self):
        pass

    @abstractmethod
    def showAge(self):
        pass


class Man(Person):

    def __init__(self, name, height):
        self.height = height
        # Person.__init__(self, name, 10)
        super().__init__(name, 10)  # same as Person.__init__(self, name, 10)
        # basically used to call the superclass init . This is used incase you want to call subclass init
        # and then also call superclass's init.
        # Since there's a hidden self in the super's parameters, when it's is called,
        # the superclasses attributes are a part of the same object that was sent out in the super() method

    def showIdentity(self):
        return self.name, self.age, self.height

    def showName(self):
        pass

    def showAge(self):
        pass


a = Man("piyush", "179")
print(a.showIdentity())

Question 7

The typical ConfigParser generated file looks like:

[Section]
bar=foo
[Section 2]
bar2= baz

Now, is there a way to index lists like, for instance:

[Section 3]
barList={
    item1,
    item2
}

Related question: Python’s ConfigParser unique keys per section

Question 8

There is nothing stopping you from packing the list into a delimited string and then unpacking it once you get the string from the config. If you did it this way your config section would look like:

[Section 3]
barList=item1,item2

It’s not pretty but it’s functional for most simple lists.

Question 9

Also a bit late, but maybe helpful for some. I am using a combination of ConfigParser and JSON:

[Foo]
fibs: [1,1,2,3,5,8,13]

just read it with:

>>> json.loads(config.get("Foo","fibs"))
[1, 1, 2, 3, 5, 8, 13]

You can even break lines if your list is long (thanks @peter-smit):

[Bar]
files_to_check = [
     "/path/to/file1",
     "/path/to/file2",
     "/path/to/another file with space in the name"
     ]

Of course i could just use JSON, but i find config files much more readable, and the [DEFAULT] Section very handy.

Question 10

Coming late to this party, but I recently implemented this with a dedicated section in a config file for a list:

[paths]
path1           = /some/path/
path2           = /another/path/
...

and using config.items( "paths" ) to get an iterable list of path items, like so:

path_items = config.items( "paths" )
for key, path in path_items:
    #do something with path

Hope this helps other folk Googling this question ;)

Question 11

One thing a lot of people don’t know is that multi-line configuration-values are allowed. For example:

;test.ini
[hello]
barlist = 
    item1
    item2

The value of config.get('hello','barlist') will now be:

"\nitem1\nitem2"

Which you easily can split with the splitlines method (don’t forget to filter empty items).

If we look to a big framework like Pyramid they are using this technique:

def aslist_cronly(value):
    if isinstance(value, string_types):
        value = filter(None, [x.strip() for x in value.splitlines()])
    return list(value)

def aslist(value, flatten=True):
    """ Return a list of strings, separating the input based on newlines
    and, if flatten=True (the default), also split on spaces within
    each line."""
    values = aslist_cronly(value)
    if not flatten:
        return values
    result = []
    for value in values:
        subvalues = value.split()
        result.extend(subvalues)
    return result

Source

Myself, I would maybe extend the ConfigParser if this is a common thing for you:

class MyConfigParser(ConfigParser):
    def getlist(self,section,option):
        value = self.get(section,option)
        return list(filter(None, (x.strip() for x in value.splitlines())))

    def getlistint(self,section,option):
        return [int(x) for x in self.getlist(section,option)]

Note that there are a few things to look out for when using this technique

New lines that are items should start with whitespace (e.g. a space or a tab)
All following lines that start with whitespace are considered to be part of the previous item. Also if it has an = sign or if it starts with a ; following the whitespace.

Question 12

If you want to literally pass in a list then you can use:

ast.literal_eval()

For example configuration:

[section]
option=["item1","item2","item3"]

The code is:

import ConfigParser
import ast

my_list = ast.literal_eval(config.get("section", "option"))
print(type(my_list))
print(my_list)

output:

<type'list'>
["item1","item2","item3"]

Question 13

No mention of the converters kwarg for ConfigParser() in any of these answers was rather disappointing.

According to the documentation you can pass a dictionary to ConfigParser that will add a get method for both the parser and section proxies. So for a list:

example.ini

[Germ]
germs: a,list,of,names, and,1,2, 3,numbers

Parser example:

cp = ConfigParser(converters={'list': lambda x: [i.strip() for i in x.split(',')]})
cp.read('example.ini')
cp.getlist('Germ', 'germs')
['a', 'list', 'of', 'names', 'and', '1', '2', '3', 'numbers']
cp['Germ'].getlist('germs')
['a', 'list', 'of', 'names', 'and', '1', '2', '3', 'numbers']

This is my personal favorite as no subclassing is necessary and I don’t have to rely on an end user to perfectly write JSON or a list that can be interpreted by ast.literal_eval.

Question 14

I landed here seeking to consume this…

[global]
spys = richard.sorge@cccp.gov, mata.hari@deutschland.gov

The answer is to split it on the comma and strip the spaces:

SPYS = [e.strip() for e in parser.get('global', 'spys').split(',')]

To get a list result:

['richard.sorge@cccp.gov', 'mata.hari@deutschland.gov']

It may not answer the OP’s question exactly but might be the simple answer some people are looking for.

Question 15

This is what I use for lists:

config file content:

[sect]
alist = a
        b
        c

code :

l = config.get('sect', 'alist').split('\n')

it work for strings

in case of numbers

config content:

nlist = 1
        2
        3

code:

nl = config.get('sect', 'alist').split('\n')
l = [int(nl) for x in nl]

thanks.

Question 16

So another way, which I prefer, is to just split the values, for example:

#/path/to/config.cfg
[Numbers]
first_row = 1,2,4,8,12,24,36,48

Could be loaded like this into a list of strings or integers, as follows:

import configparser

config = configparser.ConfigParser()
config.read('/path/to/config.cfg')

# Load into a list of strings
first_row_strings = config.get('Numbers', 'first_row').split(',')

# Load into a list of integers
first_row_integers = [int(x) for x in config.get('Numbers', 'first_row').split(',')]

This method prevents you from needing to wrap your values in brackets to load as JSON.

Question 17

Only primitive types are supported for serialization by config parser. I would use JSON or YAML for that kind of requirement.

Question 18

I faced the same problem in the past. If you need more complex lists, consider creating your own parser by inheriting from ConfigParser. Then you would overwrite the get method with that:

    def get(self, section, option):
    """ Get a parameter
    if the returning value is a list, convert string value to a python list"""
    value = SafeConfigParser.get(self, section, option)
    if (value[0] == "[") and (value[-1] == "]"):
        return eval(value)
    else:
        return value

With this solution you will also be able to define dictionaries in your config file.

But be careful! This is not as safe: this means anyone could run code through your config file. If security is not an issue in your project, I would consider using directly python classes as config files. The following is much more powerful and expendable than a ConfigParser file:

class Section
    bar = foo
class Section2
    bar2 = baz
class Section3
    barList=[ item1, item2 ]

Question 19

import ConfigParser
import os

class Parser(object):
    """attributes may need additional manipulation"""
    def __init__(self, section):
        """section to retun all options on, formatted as an object
        transforms all comma-delimited options to lists
        comma-delimited lists with colons are transformed to dicts
        dicts will have values expressed as lists, no matter the length
        """
        c = ConfigParser.RawConfigParser()
        c.read(os.path.join(os.path.dirname(__file__), 'config.cfg'))

        self.section_name = section

        self.__dict__.update({k:v for k, v in c.items(section)})

        #transform all ',' into lists, all ':' into dicts
        for key, value in self.__dict__.items():
            if value.find(':') > 0:
                #dict
                vals = value.split(',')
                dicts = [{k:v} for k, v in [d.split(':') for d in vals]]
                merged = {}
                for d in dicts:
                    for k, v in d.items():
                        merged.setdefault(k, []).append(v)
                self.__dict__[key] = merged
            elif value.find(',') > 0:
                #list
                self.__dict__[key] = value.split(',')

So now my config.cfg file, which could look like this:

[server]
credentials=username:admin,password:$3<r3t
loggingdirs=/tmp/logs,~/logs,/var/lib/www/logs
timeoutwait=15

Can be parsed into fine-grained-enough objects for my small project.

>>> import config
>>> my_server = config.Parser('server')
>>> my_server.credentials
{'username': ['admin'], 'password', ['$3<r3t']}
>>> my_server.loggingdirs:
['/tmp/logs', '~/logs', '/var/lib/www/logs']
>>> my_server.timeoutwait
'15'

This is for very quick parsing of simple configs, you lose all ability to fetch ints, bools, and other types of output without either transforming the object returned from Parser, or re-doing the parsing job accomplished by the Parser class elsewhere.

Question 20

I completed similar task in my project with section with keys without values:

import configparser

# allow_no_value param says that no value keys are ok
config = configparser.ConfigParser(allow_no_value=True)

# overwrite optionxform method for overriding default behaviour (I didn't want lowercased keys)
config.optionxform = lambda optionstr: optionstr

config.read('./app.config')

features = list(config['FEATURES'].keys())

print(features)

Output:

['BIOtag', 'TextPosition', 'IsNoun', 'IsNomn']

app.config:

[FEATURES]
BIOtag
TextPosition
IsNoun
IsNomn

Question 21

json.loads & ast.literal_eval seems to be working but simple list within config is treating each character as byte so returning even square bracket….

meaning if config has fieldvalue = [1,2,3,4,5]

then config.read(*.cfg) config['fieldValue'][0] returning [ in place of 1

Question 22

As mentioned by Peter Smit (https://stackoverflow.com/a/11866695/7424596) You might want to extend ConfigParser, in addition, an Interpolator can be used to automatically convert into and from the list.

For reference at the bottom you can find code which automatically converts config like:

[DEFAULT]
keys = [
    Overall cost structure, Capacity, RAW MATERIALS,
    BY-PRODUCT CREDITS, UTILITIES, PLANT GATE COST,
    PROCESS DESCRIPTION, AT 50% CAPACITY, PRODUCTION COSTS,
    INVESTMENT, US$ MILLION, PRODUCTION COSTS, US ¢/LB,
    VARIABLE COSTS, PRODUCTION COSTS, MAINTENANCE MATERIALS
  ]

So if you request keys you will get:

<class 'list'>: ['Overall cost structure', 'Capacity', 'RAW MATERIALS', 'BY-PRODUCT CREDITS', 'UTILITIES', 'PLANT GATE COST', 'PROCESS DESCRIPTION', 'AT 50% CAPACITY', 'PRODUCTION COSTS', 'INVESTMENT', 'US$ MILLION', 'PRODUCTION COSTS', 'US ¢/LB', 'VARIABLE COSTS', 'PRODUCTION COSTS', 'MAINTENANCE MATERIALS']

Code:

class AdvancedInterpolator(Interpolation):
    def before_get(self, parser, section, option, value, defaults):
        is_list = re.search(parser.LIST_MATCHER, value)
        if is_list:
            return parser.getlist(section, option, raw=True)
        return value


class AdvancedConfigParser(ConfigParser):

    _DEFAULT_INTERPOLATION = AdvancedInterpolator()

    LIST_SPLITTER = '\s*,\s*'
    LIST_MATCHER = '^\[([\s\S]*)\]$'

    def _to_list(self, str):
        is_list = re.search(self.LIST_MATCHER, str)
        if is_list:
            return re.split(self.LIST_SPLITTER, is_list.group(1))
        else:
            return re.split(self.LIST_SPLITTER, str)


    def getlist(self, section, option, conv=lambda x:x.strip(), *, raw=False, vars=None,
                  fallback=_UNSET, **kwargs):
        return self._get_conv(
                section, option,
                lambda value: [conv(x) for x in self._to_list(value)],
                raw=raw,
                vars=vars,
                fallback=fallback,
                **kwargs
        )

    def getlistint(self, section, option, *, raw=False, vars=None,
            fallback=_UNSET, **kwargs):
        return self.getlist(section, option, int, raw=raw, vars=vars,
                fallback=fallback, **kwargs)

    def getlistfloat(self, section, option, *, raw=False, vars=None,
            fallback=_UNSET, **kwargs):
        return self.getlist(section, option, float, raw=raw, vars=vars,
                fallback=fallback, **kwargs)

    def getlistboolean(self, section, option, *, raw=False, vars=None,
            fallback=_UNSET, **kwargs):
        return self.getlist(section, option, self._convert_to_boolean,
                raw=raw, vars=vars, fallback=fallback, **kwargs)

Ps keep in mind importance of indentdation. As reads in ConfigParser doc string:

Values can span multiple lines, as long as they are indented deeper than the first line of the value. Depending on the parser’s mode, blank lines may be treated as parts of multiline values or ignored.

Python 实用宝典

标签归档：configparser

Python扩展-使用super（）Python 3 vs Python 2

问题：Python扩展-使用super（）Python 3 vs Python 2

回答 0

回答 1

回答 2

回答 3

回答 4

ConfigParser中的列表

问题：ConfigParser中的列表

回答 0

回答 1

回答 2

回答 3

回答 4

回答 5

回答 6

回答 7

回答 8

回答 9

回答 10

回答 11

回答 12

回答 13

回答 14

有趣好用的Python教程