Python 实用宝典

Question 1

If I define a class method with a keyword argument thus:

class foo(object):
  def foodo(thing=None, thong='not underwear'):
    print thing if thing else "nothing" 
    print 'a thong is',thong

calling the method generates a TypeError:

myfoo = foo()
myfoo.foodo(thing="something")

...
TypeError: foodo() got multiple values for keyword argument 'thing'

What’s going on?

Question 2

The problem is that the first argument passed to class methods in python is always a copy of the class instance on which the method is called, typically labelled self. If the class is declared thus:

class foo(object):
  def foodo(self, thing=None, thong='not underwear'):
    print thing if thing else "nothing" 
    print 'a thong is',thong

it behaves as expected.

Explanation:

Without self as the first parameter, when myfoo.foodo(thing="something") is executed, the foodo method is called with arguments (myfoo, thing="something"). The instance myfoo is then assigned to thing (since thing is the first declared parameter), but python also attempts to assign "something" to thing, hence the Exception.

To demonstrate, try running this with the original code:

myfoo.foodo("something")
print
print myfoo

You’ll output like:

<__main__.foo object at 0x321c290>
a thong is something

<__main__.foo object at 0x321c290>

You can see that ‘thing’ has been assigned a reference to the instance ‘myfoo’ of the class ‘foo’. This section of the docs explains how function arguments work a bit more.

Question 3

Thanks for the instructive posts. I’d just like to keep a note that if you’re getting “TypeError: foodo() got multiple values for keyword argument ‘thing'”, it may also be that you’re mistakenly passing the ‘self’ as a parameter when calling the function (probably because you copied the line from the class declaration – it’s a common error when one’s in a hurry).

Question 4

This might be obvious, but it might help someone who has never seen it before. This also happens for regular functions if you mistakenly assign a parameter by position and explicitly by name.

>>> def foodo(thing=None, thong='not underwear'):
...     print thing if thing else "nothing"
...     print 'a thong is',thong
...
>>> foodo('something', thing='everything')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: foodo() got multiple values for keyword argument 'thing'

Question 5

just add ‘staticmethod’ decorator to function and problem is fixed

class foo(object):
    @staticmethod
    def foodo(thing=None, thong='not underwear'):
        print thing if thing else "nothing" 
        print 'a thong is',thong

Question 6

I want to add one more answer :

It happens when you try to pass positional parameter with wrong position order along with keyword argument in calling function.

there is difference between parameter and argument you can read in detail about here Arguments and Parameter in python

def hello(a,b=1, *args):
   print(a, b, *args)


hello(1, 2, 3, 4,a=12)

since we have three parameters :

a is positional parameter

b=1 is keyword and default parameter

*args is variable length parameter

so we first assign a as positional parameter , means we have to provide value to positional argument in its position order, here order matter. but we are passing argument 1 at the place of a in calling function and then we are also providing value to a , treating as keyword argument. now a have two values :

one is positional value: a=1

second is keyworded value which is a=12

Solution

We have to change hello(1, 2, 3, 4,a=12) to hello(1, 2, 3, 4,12) so now a will get only one positional value which is 1 and b will get value 2 and rest of values will get *args (variable length parameter)

additional information

if we want that *args should get 2,3,4 and a should get 1 and b should get 12

then we can do like this
def hello(a,*args,b=1): pass hello(1, 2, 3, 4,b=12)

Something more :

def hello(a,*c,b=1,**kwargs):
    print(b)
    print(c)
    print(a)
    print(kwargs)

hello(1,2,1,2,8,9,c=12)

output :

1

(2, 1, 2, 8, 9)

1

{'c': 12}

Question 7

This error can also happen if you pass a key word argument for which one of the keys is similar (has same string name) to a positional argument.

>>> class Foo():
...     def bar(self, bar, **kwargs):
...             print(bar)
... 
>>> kwgs = {"bar":"Barred", "jokes":"Another key word argument"}
>>> myfoo = Foo()
>>> myfoo.bar("fire", **kwgs)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: bar() got multiple values for argument 'bar'
>>>

“fire” has been accepted into the ‘bar’ argument. And yet there is another ‘bar’ argument present in kwargs.

You would have to remove the keyword argument from the kwargs before passing it to the method.

Question 8

Also this can happen in Django if you are using jquery ajax to url that reverses to a function that doesn’t contain ‘request’ parameter

$.ajax({
  url: '{{ url_to_myfunc }}',
});


def myfunc(foo, bar):
    ...

Question 9

Is there a built-in method in Python to get an array of all a class’ instance variables? For example, if I have this code:

class hi:
  def __init__(self):
    self.ii = "foo"
    self.kk = "bar"

Is there a way for me to do this:

>>> mystery_method(hi)
["ii", "kk"]

Edit: I originally had asked for class variables erroneously.

Question 10

Every object has a __dict__ variable containing all the variables and its values in it.

Try this

>>> hi_obj = hi()
>>> hi_obj.__dict__.keys()

Question 11

Use vars()

class Foo(object):
    def __init__(self):
        self.a = 1
        self.b = 2

vars(Foo()) #==> {'a': 1, 'b': 2}
vars(Foo()).keys() #==> ['a', 'b']

Question 12

You normally can’t get instance attributes given just a class, at least not without instantiating the class. You can get instance attributes given an instance, though, or class attributes given a class. See the ‘inspect’ module. You can’t get a list of instance attributes because instances really can have anything as attribute, and — as in your example — the normal way to create them is to just assign to them in the __init__ method.

An exception is if your class uses slots, which is a fixed list of attributes that the class allows instances to have. Slots are explained in http://www.python.org/2.2.3/descrintro.html, but there are various pitfalls with slots; they affect memory layout, so multiple inheritance may be problematic, and inheritance in general has to take slots into account, too.

Question 13

Both the Vars() and dict methods will work for the example the OP posted, but they won’t work for “loosely” defined objects like:

class foo:
  a = 'foo'
  b = 'bar'

To print all non-callable attributes, you can use the following function:

def printVars(object):
    for i in [v for v in dir(object) if not callable(getattr(object,v))]:
        print '\n%s:' % i
        exec('print object.%s\n\n') % i

Question 14

You can also test if an object has a specific variable with:

>>> hi_obj = hi()
>>> hasattr(hi_obj, "some attribute")

Question 15

Your example shows “instance variables”, not really class variables.

Look in hi_obj.__class__.__dict__.items() for the class variables, along with other other class members like member functions and the containing module.

class Hi( object ):
    class_var = ( 23, 'skidoo' ) # class variable
    def __init__( self ):
        self.ii = "foo" # instance variable
        self.jj = "bar"

Class variables are shared by all instances of the class.

Question 16

Suggest

>>> print vars.__doc__
vars([object]) -> dictionary

Without arguments, equivalent to locals().
With an argument, equivalent to object.__dict__.

In otherwords, it essentially just wraps __dict__

Question 17

Although not directly an answer to the OP question, there is a pretty sweet way of finding out what variables are in scope in a function. take a look at this code:

>>> def f(x, y):
    z = x**2 + y**2
    sqrt_z = z**.5
    return sqrt_z

>>> f.func_code.co_varnames
('x', 'y', 'z', 'sqrt_z')
>>>

The func_code attribute has all kinds of interesting things in it. It allows you todo some cool stuff. Here is an example of how I have have used this:

def exec_command(self, cmd, msg, sig):

    def message(msg):
        a = self.link.process(self.link.recieved_message(msg))
        self.exec_command(*a)

    def error(msg):
        self.printer.printInfo(msg)

    def set_usrlist(msg):
        self.client.connected_users = msg

    def chatmessage(msg):
        self.printer.printInfo(msg)

    if not locals().has_key(cmd): return
    cmd = locals()[cmd]

    try:
        if 'sig' in cmd.func_code.co_varnames and \
                       'msg' in cmd.func_code.co_varnames: 
            cmd(msg, sig)
        elif 'msg' in cmd.func_code.co_varnames: 
            cmd(msg)
        else:
            cmd()
    except Exception, e:
        print '\n-----------ERROR-----------'
        print 'error: ', e
        print 'Error proccessing: ', cmd.__name__
        print 'Message: ', msg
        print 'Sig: ', sig
        print '-----------ERROR-----------\n'

Question 18

built on dmark’s answer to get the following, which is useful if you want the equiv of sprintf and hopefully will help someone…

def sprint(object):
    result = ''
    for i in [v for v in dir(object) if not callable(getattr(object, v)) and v[0] != '_']:
        result += '\n%s:' % i + str(getattr(object, i, ''))
    return result

Question 19

Sometimes you want to filter the list based on public/private vars. E.g.

def pub_vars(self):
    """Gives the variable names of our instance we want to expose
    """
    return [k for k in vars(self) if not k.startswith('_')]

Question 20

The Zen of Python states that there should only be one way to do things- yet frequently I run into the problem of deciding when to use a function versus when to use a method.

Let’s take a trivial example- a ChessBoard object. Let’s say we need some way to get all the legal King moves available on the board. Do we write ChessBoard.get_king_moves() or get_king_moves(chess_board)?

Here are some related questions I looked at:

The answers I got were largely inconclusive:

Why does Python use methods for some functionality (e.g. list.index()) but functions for other (e.g. len(list))?

The major reason is history. Functions were used for those operations that were generic for a group of types and which were intended to work even for objects that didn’t have methods at all (e.g. tuples). It is also convenient to have a function that can readily be applied to an amorphous collection of objects when you use the functional features of Python (map(), apply() et al).

In fact, implementing len(), max(), min() as a built-in function is actually less code than implementing them as methods for each type. One can quibble about individual cases but it’s a part of Python, and it’s too late to make such fundamental changes now. The functions have to remain to avoid massive code breakage.

While interesting, the above doesn’t really say much as to what strategy to adopt.

This is one of the reasons – with custom methods, developers would be free to choose a different method name, like getLength(), length(), getlength() or whatsoever. Python enforces strict naming so that the common function len() can be used.

Slightly more interesting. My take is that functions are in a sense, the Pythonic version of interfaces.

Lastly, from Guido himself:

Talking about the Abilities/Interfaces made me think about some of our “rogue” special method names. In the Language Reference, it says, “A class can implement certain operations that are invoked by special syntax (such as arithmetic operations or subscripting and slicing) by defining methods with special names.” But there are all these methods with special names like __len__ or __unicode__ which seem to be provided for the benefit of built-in functions, rather than for support of syntax. Presumably in an interface-based Python, these methods would turn into regularly-named methods on an ABC, so that __len__ would become
class container:
  ...
  def len(self):
    raise NotImplemented
Though, thinking about it some more, I don’t see why all syntactic operations wouldn’t just invoke the appropriate normally-named method on a specific ABC. “<“, for instance, would presumably invoke “object.lessthan” (or perhaps “comparable.lessthan“). So another benefit would be the ability to wean Python away from this mangled-name oddness, which seems to me an HCI improvement.
Hm. I’m not sure I agree (figure that :-).

There are two bits of “Python rationale” that I’d like to explain first.

First of all, I chose len(x) over x.len() for HCI reasons (def __len__() came much later). There are two intertwined reasons actually, both HCI:

(a) For some operations, prefix notation just reads better than postfix — prefix (and infix!) operations have a long tradition in mathematics which likes notations where the visuals help the mathematician thinking about a problem. Compare the easy with which we rewrite a formula like x*(a+b) into x*a + x*b to the clumsiness of doing the same thing using a raw OO notation.

(b) When I read code that says len(x) I know that it is asking for the length of something. This tells me two things: the result is an integer, and the argument is some kind of container. To the contrary, when I read x.len(), I have to already know that x is some kind of container implementing an interface or inheriting from a class that has a standard len(). Witness the confusion we occasionally have when a class that is not implementing a mapping has a get() or keys() method, or something that isn’t a file has a write() method.

Saying the same thing in another way, I see ‘len’ as a built-in operation. I’d hate to lose that. I can’t say for sure whether you meant that or not, but ‘def len(self): …’ certainly sounds like you want to demote it to an ordinary method. I’m strongly -1 on that.

The second bit of Python rationale I promised to explain is the reason why I chose special methods to look __special__ and not merely special. I was anticipating lots of operations that classes might want to override, some standard (e.g. __add__ or __getitem__), some not so standard (e.g. pickle’s __reduce__ for a long time had no support in C code at all). I didn’t want these special operations to use ordinary method names, because then pre-existing classes, or classes written by users without an encyclopedic memory for all the special methods, would be liable to accidentally define operations they didn’t mean to implement, with possibly disastrous consequences. Ivan Krstić explained this more concise in his message, which arrived after I’d written all this up.

— –Guido van Rossum (home page: http://www.python.org/~guido/)

My understanding of this is that in certain cases, prefix notation just makes more sense (ie, Duck.quack makes more sense than quack(Duck) from a linguistic standpoint.) and again, the functions allow for “interfaces”.

In such a case, my guess would be to implement get_king_moves based solely on Guido’s first point. But that still leaves a lot of open questions regarding say, implementing a stack and queue class with similar push and pop methods- should they be functions or methods? (here I would guess functions, because I really want to signal a push-pop interface)

TLDR: Can someone explain what the strategy for deciding when to use functions vs. methods should be?

Question 21

My general rule is this – is the operation performed on the object or by the object?

if it is done by the object, it should be a member operation. If it could apply to other things too, or is done by something else to the object then it should be a function (or perhaps a member of something else).

When introducing programming, it is traditional (albeit implementation incorrect) to describe objects in terms of real-world objects such as cars. You mention a duck, so let’s go with that.

class duck: 
    def __init__(self):pass
    def eat(self, o): pass 
    def crap(self) : pass
    def die(self)
    ....

In the context of the “objects are real things” analogy, it is “correct” to add a class method for anything which the object can do. So say I want to kill off a duck, do I add a .kill() to the duck? No… as far as I know animals do not commit suicide. Therefore if I want to kill a duck I should do this:

def kill(o):
    if isinstance(o, duck):
        o.die()
    elif isinstance(o, dog):
        print "WHY????"
        o.die()
    elif isinstance(o, nyancat):
        raise Exception("NYAN "*9001)
    else:
       print "can't kill it."

Moving away from this analogy, why do we use methods and classes? Because we want to contain data and hopefully structure our code in a manner such that it will be reusable and extensible in the future. This brings us to the notion of encapsulation which is so dear to OO design.

The encapsulation principal is really what this comes down to: as a designer you should hide everything about the implementation and class internals which it is not absolutely necessarily for any user or other developer to access. Because we deal with instances of classes, this reduces to “what operations are crucial on this instance“. If an operation is not instance specific, then it should not be a member function.

TL;DR: what @Bryan said. If it operates on an instance and needs to access data which is internal to the class instance, it should be a member function.

Question 22

Use a class when you want to:

1) Isolate calling code from implementation details — taking advantage of abstraction and encapsulation.

2) When you want to be substitutable for other objects — taking advantage of polymorphism.

3) When you want to reuse code for similar objects — taking advantage of inheritance.

Use a function for calls that make sense across many different object types — for example, the builtin len and repr functions apply to many kinds of objects.

That being said, the choice sometimes comes down to a matter of taste. Think in terms of what is most convenient and readable for typical calls. For example, which would be better (x.sin()**2 + y.cos()**2).sqrt() or sqrt(sin(x)**2 + cos(y)**2)?

Question 23

Here’s a simple rule of thumb: if the code acts upon a single instance of an object, use a method. Even better: use a method unless there is a compelling reason to write it as a function.

In your specific example, you want it to look like this:

chessboard = Chessboard()
...
chessboard.get_king_moves()

Don’t over think it. Always use methods until the point comes where you say to yourself “it doesn’t make sense to make this a method”, in which case you can make a function.

Question 24

I usually think of an object like a person.

Attributes are the person’s name, height, shoe size, etc.

Methods and functions are operations that the person can perform.

If the operation could be done by just any ol’ person, without requiring anything unique to this one specific person (and without changing anything on this one specific person), then it’s a function and should be written as such.

If an operation is acting upon the person (e.g. eating, walking, …) or requires something unique to this person to get involved (like dancing, writing a book, …), then it should be a method.

Of course, it is not always trivial to translate this into the specific object you’re working with, but I find it is a good way to think of it.

Question 25

Generally I use classes to implement a logical set of capabilities for some thing, so that in the rest of my program I can reason about the thing, not having to worry about all the little concerns that make up its implementation.

Anything that’s part of that core abstraction of “what you can do with a thing” should usually be a method. This generally includes everything that can alter a thing, as the internal data state is usually considered private and not part of the logical idea of “what you can do with a thing“.

When you come to higher level operations, especially if they involve multiple things, I find they are usually most naturally expressed as functions, if they can be built out of the public abstraction of a thing without needing special access to the internals (unless they’re methods of some other object). This has the big advantage that when I decide to completely rewrite the internals of how my thing works (without changing the interface), I just have a small core set of methods to rewrite, and then all the external functions written in terms of those methods will Just Work. I find that insisting that all operations to do with class X are methods on class X leads to over-complicated classes.

It depends on the code I’m writing though. For some programs I model them as a collection of objects whose interactions give rise to the behavior of the program; here most important functionality is closely coupled to a single object, and so is implemented in methods, with a scattering of utility functions. For other programs the most important stuff is a set of functions that manipulate data, and classes are in use only to implement the natural “duck types” that are manipulated by the functions.

Question 26

You may say that, “in the face of ambiguity, refuse the temptation to guess”.

However, it’s not even a guess. You’re absolutely sure that the outcomes of both approaches are the same in that they solve your problem.

I believe it is only a good thing to have multiple ways to accomplishing goals. I’d humbly tell you, as other users did already, to employ whichever “tastes better” / feels more intuitive, in terms of language.

Question 27

In Python, is there a way to bind an unbound method without calling it?

I am writing a wxPython program, and for a certain class I decided it’d be nice to group the data of all of my buttons together as a class-level list of tuples, like so:

class MyWidget(wx.Window):
    buttons = [("OK", OnOK),
               ("Cancel", OnCancel)]

    # ...

    def Setup(self):
        for text, handler in MyWidget.buttons:

            # This following line is the problem line.
            b = wx.Button(parent, label=text).Bind(wx.EVT_BUTTON, handler)

The problem is, since all of the values of handler are unbound methods, my program explodes in a spectacular blaze and I weep.

I was looking around online for a solution to what seems like should be a relatively straightforward, solvable problem. Unfortunately I couldn’t find anything. Right now, I’m using functools.partial to work around this, but does anyone know if there’s a clean-feeling, healthy, Pythonic way to bind an unbound method to an instance and continue passing it around without calling it?

Question 28

All functions are also descriptors, so you can bind them by calling their __get__ method:

bound_handler = handler.__get__(self, MyWidget)

Here’s R. Hettinger’s excellent guide to descriptors.

As a self-contained example pulled from Keith’s comment:

def bind(instance, func, as_name=None):
    """
    Bind the function *func* to *instance*, with either provided name *as_name*
    or the existing name of *func*. The provided *func* should accept the 
    instance as the first argument, i.e. "self".
    """
    if as_name is None:
        as_name = func.__name__
    bound_method = func.__get__(instance, instance.__class__)
    setattr(instance, as_name, bound_method)
    return bound_method

class Thing:
    def __init__(self, val):
        self.val = val

something = Thing(21)

def double(self):
    return 2 * self.val

bind(something, double)
something.double()  # returns 42

Question 29

This can be done cleanly with types.MethodType. Example:

import types

def f(self): print self

class C(object): pass

meth = types.MethodType(f, C(), C) # Bind f to an instance of C
print meth # prints <bound method C.f of <__main__.C object at 0x01255E90>>

Question 30

Creating a closure with self in it will not technically bind the function, but it is an alternative way of solving the same (or very similar) underlying problem. Here’s a trivial example:

self.method = (lambda self: lambda args: self.do(args))(self)

Question 31

This will bind self to handler:

bound_handler = lambda *args, **kwargs: handler(self, *args, **kwargs)

This works by passing self as the first argument to the function. object.function() is just syntactic sugar for function(object).

Question 32

Late to the party, but I came here with a similar question: I have a class method and an instance, and want to apply the instance to the method.

At the risk of oversimplifying the OP’s question, I ended up doing something less mysterious that may be useful to others who arrive here (caveat: I’m working in Python 3 — YMMV).

Consider this simple class:

class Foo(object):

    def __init__(self, value):
        self._value = value

    def value(self):
        return self._value

    def set_value(self, value):
        self._value = value

Here’s what you can do with it:

>>> meth = Foo.set_value   # the method
>>> a = Foo(12)            # a is an instance with value 12
>>> meth(a, 33)            # apply instance and method
>>> a.value()              # voila - the method was called
33

Question 33

I know that __call__ method in a class is triggered when the instance of a class is called. However, I have no idea when I can use this special method, because one can simply create a new method and perform the same operation done in __call__ method and instead of calling the instance, you can call the method.

I would really appreciate it if someone gives me a practical usage of this special method.

Question 34

Django forms module uses __call__ method nicely to implement a consistent API for form validation. You can write your own validator for a form in Django as a function.

def custom_validator(value):
    #your validation logic

Django has some default built-in validators such as email validators, url validators etc., which broadly fall under the umbrella of RegEx validators. To implement these cleanly, Django resorts to callable classes (instead of functions). It implements default Regex Validation logic in a RegexValidator and then extends these classes for other validations.

class RegexValidator(object):
    def __call__(self, value):
        # validation logic

class URLValidator(RegexValidator):
    def __call__(self, value):
        super(URLValidator, self).__call__(value)
        #additional logic

class EmailValidator(RegexValidator):
    # some logic

Now both your custom function and built-in EmailValidator can be called with the same syntax.

for v in [custom_validator, EmailValidator()]:
    v(value) # <-----

As you can see, this implementation in Django is similar to what others have explained in their answers below. Can this be implemented in any other way? You could, but IMHO it will not be as readable or as easily extensible for a big framework like Django.

Question 35

This example uses memoization, basically storing values in a table (dictionary in this case) so you can look them up later instead of recalculating them.

Here we use a simple class with a __call__ method to calculate factorials (through a callable object) instead of a factorial function that contains a static variable (as that’s not possible in Python).

class Factorial:
    def __init__(self):
        self.cache = {}
    def __call__(self, n):
        if n not in self.cache:
            if n == 0:
                self.cache[n] = 1
            else:
                self.cache[n] = n * self.__call__(n-1)
        return self.cache[n]

fact = Factorial()

Now you have a fact object which is callable, just like every other function. For example

for i in xrange(10):                                                             
    print("{}! = {}".format(i, fact(i)))

# output
0! = 1
1! = 1
2! = 2
3! = 6
4! = 24
5! = 120
6! = 720
7! = 5040
8! = 40320
9! = 362880

And it is also stateful.

Question 36

I find it useful because it allows me to create APIs that are easy to use (you have some callable object that requires some specific arguments), and are easy to implement because you can use Object Oriented practices.

The following is code I wrote yesterday that makes a version of the hashlib.foo methods that hash entire files rather than strings:

# filehash.py
import hashlib


class Hasher(object):
    """
    A wrapper around the hashlib hash algorithms that allows an entire file to
    be hashed in a chunked manner.
    """
    def __init__(self, algorithm):
        self.algorithm = algorithm

    def __call__(self, file):
        hash = self.algorithm()
        with open(file, 'rb') as f:
            for chunk in iter(lambda: f.read(4096), ''):
                hash.update(chunk)
        return hash.hexdigest()


md5    = Hasher(hashlib.md5)
sha1   = Hasher(hashlib.sha1)
sha224 = Hasher(hashlib.sha224)
sha256 = Hasher(hashlib.sha256)
sha384 = Hasher(hashlib.sha384)
sha512 = Hasher(hashlib.sha512)

This implementation allows me to use the functions in a similar fashion to the hashlib.foo functions:

from filehash import sha1
print sha1('somefile.txt')

Of course I could have implemented it a different way, but in this case it seemed like a simple approach.

Question 37

__call__ is also used to implement decorator classes in python. In this case the instance of the class is called when the method with the decorator is called.

class EnterExitParam(object):

    def __init__(self, p1):
        self.p1 = p1

    def __call__(self, f):
        def new_f():
            print("Entering", f.__name__)
            print("p1=", self.p1)
            f()
            print("Leaving", f.__name__)
        return new_f


@EnterExitParam("foo bar")
def hello():
    print("Hello")


if __name__ == "__main__":
    hello()

Question 38

Yes, when you know you’re dealing with objects, it’s perfectly possible (and in many cases advisable) to use an explicit method call. However, sometimes you deal with code that expects callable objects – typically functions, but thanks to __call__ you can build more complex objects, with instance data and more methods to delegate repetitive tasks, etc. that are still callable.

Also, sometimes you’re using both objects for complex tasks (where it makes sense to write a dedicated class) and objects for simple tasks (that already exist in functions, or are more easily written as functions). To have a common interface, you either have to write tiny classes wrapping those functions with the expected interface, or you keep the functions functions and make the more complex objects callable. Let’s take threads as example. The Thread objects from the standard libary module threading want a callable as target argument (i.e. as action to be done in the new thread). With a callable object, you are not restricted to functions, you can pass other objects as well, such as a relatively complex worker that gets tasks to do from other threads and executes them sequentially:

class Worker(object):
    def __init__(self, *args, **kwargs):
        self.queue = queue.Queue()
        self.args = args
        self.kwargs = kwargs

    def add_task(self, task):
        self.queue.put(task)

    def __call__(self):
        while True:
            next_action = self.queue.get()
            success = next_action(*self.args, **self.kwargs)
            if not success:
               self.add_task(next_action)

This is just an example off the top of my head, but I think it is already complex enough to warrant the class. Doing this only with functions is hard, at least it requires returning two functions and that’s slowly getting complex. One could rename __call__ to something else and pass a bound method, but that makes the code creating the thread slightly less obvious, and doesn’t add any value.

Question 39

Class-based decorators use __call__ to reference the wrapped function. E.g.:

class Deco(object):
    def __init__(self,f):
        self.f = f
    def __call__(self, *args, **kwargs):
        print args
        print kwargs
        self.f(*args, **kwargs)

There is a good description of the various options here at Artima.com

Question 40

IMHO __call__ method and closures give us a natural way to create STRATEGY design pattern in Python. We define a family of algorithms, encapsulate each one, make them interchangeable and in the end we can execute a common set of steps and, for example, calculate a hash for a file.

Question 41

I just stumbled upon a usage of __call__() in concert with __getattr__() which I think is beautiful. It allows you to hide multiple levels of a JSON/HTTP/(however_serialized) API inside an object.

The __getattr__() part takes care of iteratively returning a modified instance of the same class, filling in one more attribute at a time. Then, after all information has been exhausted, __call__() takes over with whatever arguments you passed in.

Using this model, you can for example make a call like api.v2.volumes.ssd.update(size=20), which ends up in a PUT request to https://some.tld/api/v2/volumes/ssd/update.

The particular code is a block storage driver for a certain volume backend in OpenStack, you can check it out here: https://github.com/openstack/cinder/blob/master/cinder/volume/drivers/nexenta/jsonrpc.py

EDIT: Updated the link to point to master revision.

Question 42

Specify a __metaclass__ and override the __call__ method, and have the specified meta classes’ __new__ method return an instance of the class, viola you have a “function” with methods.

Question 43

We can use __call__ method to use other class methods as static methods.

    class _Callable:
        def __init__(self, anycallable):
            self.__call__ = anycallable

    class Model:

        def get_instance(conn, table_name):

            """ do something"""

        get_instance = _Callable(get_instance)

    provs_fac = Model.get_instance(connection, "users")

Question 44

One common example is the __call__ in functools.partial, here is a simplified version (with Python >= 3.5):

class partial:
    """New function with partial application of the given arguments and keywords."""

    def __new__(cls, func, *args, **kwargs):
        if not callable(func):
            raise TypeError("the first argument must be callable")
        self = super().__new__(cls)

        self.func = func
        self.args = args
        self.kwargs = kwargs
        return self

    def __call__(self, *args, **kwargs):
        return self.func(*self.args, *args, **self.kwargs, **kwargs)

Usage:

def add(x, y):
    return x + y

inc = partial(add, y=1)
print(inc(41))  # 42

Question 45

The function call operator.

class Foo:
    def __call__(self, a, b, c):
        # do something

x = Foo()
x(1, 2, 3)

The __call__ method can be used to redefined/re-initialize the same object. It also facilitates the use of instances/objects of a class as functions by passing arguments to the objects.

Question 46

I find a good place to use callable objects, those that define __call__(), is when using the functional programming capabilities in Python, such as map(), filter(), reduce().

The best time to use a callable object over a plain function or a lambda function is when the logic is complex and needs to retain some state or uses other info that in not passed to the __call__() function.

Here’s some code that filters file names based upon their filename extension using a callable object and filter().

Callable:

import os

class FileAcceptor(object):
    def __init__(self, accepted_extensions):
        self.accepted_extensions = accepted_extensions

    def __call__(self, filename):
        base, ext = os.path.splitext(filename)
        return ext in self.accepted_extensions

class ImageFileAcceptor(FileAcceptor):
    def __init__(self):
        image_extensions = ('.jpg', '.jpeg', '.gif', '.bmp')
        super(ImageFileAcceptor, self).__init__(image_extensions)

Usage:

filenames = [
    'me.jpg',
    'me.txt',
    'friend1.jpg',
    'friend2.bmp',
    'you.jpeg',
    'you.xml']

acceptor = ImageFileAcceptor()
image_filenames = filter(acceptor, filenames)
print image_filenames

Output:

['me.jpg', 'friend1.jpg', 'friend2.bmp', 'you.jpeg']

Question 47

This is too late but I’m giving an example. Imagine you have a Vector class and a Point class. Both take x, y as positional args. Let’s imagine you want to create a function that moves the point to be put on the vector.

4 Solutions

put_point_on_vec(point, vec)
Make it a method on the vector class. e.g my_vec.put_point(point)
Make it a method on the Point class. my_point.put_on_vec(vec)
Vector implements __call__, So you can use it like my_vec_instance(point)

This is actually part of some examples I’m working on for a guide for dunder methods explained with Maths that I’m gonna release sooner or later.

I left the logic of moving the point itself because this is not what this question is about

Question 48

I have a nested dictionary. Is there only one way to get values out safely?

try:
    example_dict['key1']['key2']
except KeyError:
    pass

Or maybe python has a method like get() for nested dictionary ?

Question 49

You could use get twice:

example_dict.get('key1', {}).get('key2')

This will return None if either key1 or key2 does not exist.

Note that this could still raise an AttributeError if example_dict['key1'] exists but is not a dict (or a dict-like object with a get method). The try..except code you posted would raise a TypeError instead if example_dict['key1'] is unsubscriptable.

Another difference is that the try...except short-circuits immediately after the first missing key. The chain of get calls does not.

If you wish to preserve the syntax, example_dict['key1']['key2'] but do not want it to ever raise KeyErrors, then you could use the Hasher recipe:

class Hasher(dict):
    # https://stackoverflow.com/a/3405143/190597
    def __missing__(self, key):
        value = self[key] = type(self)()
        return value

example_dict = Hasher()
print(example_dict['key1'])
# {}
print(example_dict['key1']['key2'])
# {}
print(type(example_dict['key1']['key2']))
# <class '__main__.Hasher'>

Note that this returns an empty Hasher when a key is missing.

Since Hasher is a subclass of dict you can use a Hasher in much the same way you could use a dict. All the same methods and syntax is available, Hashers just treat missing keys differently.

You can convert a regular dict into a Hasher like this:

hasher = Hasher(example_dict)

and convert a Hasher to a regular dict just as easily:

regular_dict = dict(hasher)

Another alternative is to hide the ugliness in a helper function:

def safeget(dct, *keys):
    for key in keys:
        try:
            dct = dct[key]
        except KeyError:
            return None
    return dct

So the rest of your code can stay relatively readable:

safeget(example_dict, 'key1', 'key2')

Question 50

You could also use python reduce:

def deep_get(dictionary, *keys):
    return reduce(lambda d, key: d.get(key) if d else None, keys, dictionary)

Question 51

By combining all of these answer here and small changes that I made, I think this function would be useful. its safe, quick, easily maintainable.

def deep_get(dictionary, keys, default=None):
    return reduce(lambda d, key: d.get(key, default) if isinstance(d, dict) else default, keys.split("."), dictionary)

Example :

>>> from functools import reduce
>>> def deep_get(dictionary, keys, default=None):
...     return reduce(lambda d, key: d.get(key, default) if isinstance(d, dict) else default, keys.split("."), dictionary)
...
>>> person = {'person':{'name':{'first':'John'}}}
>>> print (deep_get(person, "person.name.first"))
John
>>> print (deep_get(person, "person.name.lastname"))
None
>>> print (deep_get(person, "person.name.lastname", default="No lastname"))
No lastname
>>>

Question 52

Building up on Yoav’s answer, an even safer approach:

def deep_get(dictionary, *keys):
    return reduce(lambda d, key: d.get(key, None) if isinstance(d, dict) else None, keys, dictionary)

Question 53

A recursive solution. It’s not the most efficient but I find it a bit more readable than the other examples and it doesn’t rely on functools.

def deep_get(d, keys):
    if not keys or d is None:
        return d
    return deep_get(d.get(keys[0]), keys[1:])

Example

d = {'meta': {'status': 'OK', 'status_code': 200}}
deep_get(d, ['meta', 'status_code'])     # => 200
deep_get(d, ['garbage', 'status_code'])  # => None

A more polished version

def deep_get(d, keys, default=None):
    """
    Example:
        d = {'meta': {'status': 'OK', 'status_code': 200}}
        deep_get(d, ['meta', 'status_code'])          # => 200
        deep_get(d, ['garbage', 'status_code'])       # => None
        deep_get(d, ['meta', 'garbage'], default='-') # => '-'
    """
    assert type(keys) is list
    if d is None:
        return default
    if not keys:
        return d
    return deep_get(d.get(keys[0]), keys[1:], default)

Question 54

While the reduce approach is neat and short, I think a simple loop is easier to grok. I’ve also included a default parameter.

def deep_get(_dict, keys, default=None):
    for key in keys:
        if isinstance(_dict, dict):
            _dict = _dict.get(key, default)
        else:
            return default
    return _dict

As an exercise to understand how the reduce one-liner worked, I did the following. But ultimately the loop approach seems more intuitive to me.

def deep_get(_dict, keys, default=None):

    def _reducer(d, key):
        if isinstance(d, dict):
            return d.get(key, default)
        return default

    return reduce(_reducer, keys, _dict)

Usage

nested = {'a': {'b': {'c': 42}}}

print deep_get(nested, ['a', 'b'])
print deep_get(nested, ['a', 'b', 'z', 'z'], default='missing')

Question 55

I suggest you to try python-benedict.

It is a dict subclass that provides keypath support and much more.

Installation: pip install python-benedict

from benedict import benedict

example_dict = benedict(example_dict, keypath_separator='.')

now you can access nested values using keypath:

val = example_dict['key1.key2']

# using 'get' method to avoid a possible KeyError:
val = example_dict.get('key1.key2')

or access nested values using keys list:

val = example_dict['key1', 'key2']

# using get to avoid a possible KeyError:
val = example_dict.get(['key1', 'key2'])

It is well tested and open-source on GitHub:

https://github.com/fabiocaccamo/python-benedict

Note: I am the author of this project

Question 56

A simple class that can wrap a dict, and retrieve based on a key:

class FindKey(dict):
    def get(self, path, default=None):
        keys = path.split(".")
        val = None

        for key in keys:
            if val:
                if isinstance(val, list):
                    val = [v.get(key, default) if v else None for v in val]
                else:
                    val = val.get(key, default)
            else:
                val = dict.get(self, key, default)

            if not val:
                break

        return val

For example:

person = {'person':{'name':{'first':'John'}}}
FindDict(person).get('person.name.first') # == 'John'

If the key doesn’t exist, it returns None by default. You can override that using a default= key in the FindDict wrapper — for example`:

FindDict(person, default='').get('person.name.last') # == doesn't exist, so ''

Question 57

for a second level key retrieving, you can do this:

key2_value = (example_dict.get('key1') or {}).get('key2')

Question 58

After seeing this for deeply getting attributes, I made the following to safely get nested dict values using dot notation. This works for me because my dicts are deserialized MongoDB objects, so I know the key names don’t contain .s. Also, in my context, I can specify a falsy fallback value (None) that I don’t have in my data, so I can avoid the try/except pattern when calling the function.

from functools import reduce # Python 3
def deepgetitem(obj, item, fallback=None):
    """Steps through an item chain to get the ultimate value.

    If ultimate value or path to value does not exist, does not raise
    an exception and instead returns `fallback`.

    >>> d = {'snl_final': {'about': {'_icsd': {'icsd_id': 1}}}}
    >>> deepgetitem(d, 'snl_final.about._icsd.icsd_id')
    1
    >>> deepgetitem(d, 'snl_final.about._sandbox.sbx_id')
    >>>
    """
    def getitem(obj, name):
        try:
            return obj[name]
        except (KeyError, TypeError):
            return fallback
    return reduce(getitem, item.split('.'), obj)

Question 59

Yet another function for the same thing, also returns a boolean to represent whether the key was found or not and handles some unexpected errors.

'''
json : json to extract value from if exists
path : details.detail.first_name
            empty path represents root

returns a tuple (boolean, object)
        boolean : True if path exists, otherwise False
        object : the object if path exists otherwise None

'''
def get_json_value_at_path(json, path=None, default=None):

    if not bool(path):
        return True, json
    if type(json) is not dict :
        raise ValueError(f'json={json}, path={path} not supported, json must be a dict')
    if type(path) is not str and type(path) is not list:
        raise ValueError(f'path format {path} not supported, path can be a list of strings like [x,y,z] or a string like x.y.z')

    if type(path) is str:
        path = path.strip('.').split('.')
    key = path[0]
    if key in json.keys():
        return get_json_value_at_path(json[key], path[1:], default)
    else:
        return False, default

example usage:

my_json = {'details' : {'first_name' : 'holla', 'last_name' : 'holla'}}
print(get_json_value_at_path(my_json, 'details.first_name', ''))
print(get_json_value_at_path(my_json, 'details.phone', ''))

(True, ‘holla’)

(False, ”)

Question 60

You can use pydash:

import pydash as _

_.get(example_dict, 'key1.key2', default='Default')

https://pydash.readthedocs.io/en/latest/api.html

Question 61

An adaptation of unutbu’s answer that I found useful in my own code:

example_dict.setdefaut('key1', {}).get('key2')

It generates a dictionary entry for key1 if it does not have that key already so that you avoid the KeyError. If you want to end up a nested dictionary that includes that key pairing anyway like I did, this seems like the easiest solution.

Question 62

Since raising an key error if one of keys is missing is a reasonable thing to do, we can even not check for it and get it as single as that:

def get_dict(d, kl):
  cur = d[kl[0]]
  return get_dict(cur, kl[1:]) if len(kl) > 1 else cur

Question 63

Little improvement to reduce approach to make it work with list. Also using data path as string divided by dots instead of array.

def deep_get(dictionary, path):
    keys = path.split('.')
    return reduce(lambda d, key: d[int(key)] if isinstance(d, list) else d.get(key) if d else None, keys, dictionary)

Question 64

A solution I’ve used that is similar to the double get but with the additional ability to avoid a TypeError using if else logic:

    value = example_dict['key1']['key2'] if example_dict.get('key1') and example_dict['key1'].get('key2') else default_value

However, the more nested the dictionary the more cumbersome this becomes.

问题：类方法生成“ TypeError：…为关键字参数获得了多个值……”

回答 0

回答 1

回答 2

回答 3

回答 4

解

附加信息

Solution

additional information

回答 5

回答 6

问题：如何在Python中获取实例变量？

回答 0

回答 1

回答 2

回答 3

回答 4

回答 5

回答 6

回答 7

回答 8

回答 9

问题：在Python中，什么时候应该使用函数而不是方法？

为什么Python使用方法来实现某些功能（例如list.index（）），却使用其他方法（例如len（list））呢？

Why does Python use methods for some functionality (e.g. list.index()) but functions for other (e.g. len(list))?

回答 0

回答 1

回答 2

回答 3

回答 4

回答 5

问题：Python：绑定未绑定方法？

回答 0

回答 1

回答 2

回答 3

回答 4

问题：Python __call__特殊方法的实际示例

回答 0

回答 1

回答 2

回答 3

回答 4

回答 5

回答 6

回答 7

回答 8

回答 9

回答 10

回答 11

回答 12

回答 13

4个解决方案

4 Solutions

问题：获取嵌套字典值的Python安全方法

回答 0

回答 1

回答 2

回答 3

回答 4

回答 5

回答 6

回答 7

回答 8

回答 9

回答 10

回答 11

回答 12

回答 13

回答 14

回答 15

回答 16

回答 17

回答 18

问题：未绑定方法f（）必须以fibo_实例作为第一个参数调用（取而代之的是class classobj实例）

回答 0

回答 1

如何用尽可能少的行来重现此错误：

How to reproduce this error with as few lines as possible:

问题：Python call特殊方法的实际示例