获取嵌套字典值的Python安全方法

I have a nested dictionary. Is there only one way to get values out safely?

try:
    example_dict['key1']['key2']
except KeyError:
    pass

Or maybe python has a method like get() for nested dictionary ?

You could use get twice:

example_dict.get('key1', {}).get('key2')

This will return None if either key1 or key2 does not exist.

Note that this could still raise an AttributeError if example_dict['key1'] exists but is not a dict (or a dict-like object with a get method). The try..except code you posted would raise a TypeError instead if example_dict['key1'] is unsubscriptable.

Another difference is that the try...except short-circuits immediately after the first missing key. The chain of get calls does not.

If you wish to preserve the syntax, example_dict['key1']['key2'] but do not want it to ever raise KeyErrors, then you could use the Hasher recipe:

class Hasher(dict):
    # https://stackoverflow.com/a/3405143/190597
    def __missing__(self, key):
        value = self[key] = type(self)()
        return value

example_dict = Hasher()
print(example_dict['key1'])
# {}
print(example_dict['key1']['key2'])
# {}
print(type(example_dict['key1']['key2']))
# <class '__main__.Hasher'>

Note that this returns an empty Hasher when a key is missing.

Since Hasher is a subclass of dict you can use a Hasher in much the same way you could use a dict. All the same methods and syntax is available, Hashers just treat missing keys differently.

You can convert a regular dict into a Hasher like this:

hasher = Hasher(example_dict)

and convert a Hasher to a regular dict just as easily:

regular_dict = dict(hasher)

Another alternative is to hide the ugliness in a helper function:

def safeget(dct, *keys):
    for key in keys:
        try:
            dct = dct[key]
        except KeyError:
            return None
    return dct

So the rest of your code can stay relatively readable:

safeget(example_dict, 'key1', 'key2')

You could also use python reduce:

def deep_get(dictionary, *keys):
    return reduce(lambda d, key: d.get(key) if d else None, keys, dictionary)

By combining all of these answer here and small changes that I made, I think this function would be useful. its safe, quick, easily maintainable.

def deep_get(dictionary, keys, default=None):
    return reduce(lambda d, key: d.get(key, default) if isinstance(d, dict) else default, keys.split("."), dictionary)

Example :

>>> from functools import reduce
>>> def deep_get(dictionary, keys, default=None):
...     return reduce(lambda d, key: d.get(key, default) if isinstance(d, dict) else default, keys.split("."), dictionary)
...
>>> person = {'person':{'name':{'first':'John'}}}
>>> print (deep_get(person, "person.name.first"))
John
>>> print (deep_get(person, "person.name.lastname"))
None
>>> print (deep_get(person, "person.name.lastname", default="No lastname"))
No lastname
>>>

Building up on Yoav’s answer, an even safer approach:

def deep_get(dictionary, *keys):
    return reduce(lambda d, key: d.get(key, None) if isinstance(d, dict) else None, keys, dictionary)

A recursive solution. It’s not the most efficient but I find it a bit more readable than the other examples and it doesn’t rely on functools.

def deep_get(d, keys):
    if not keys or d is None:
        return d
    return deep_get(d.get(keys[0]), keys[1:])

Example

d = {'meta': {'status': 'OK', 'status_code': 200}}
deep_get(d, ['meta', 'status_code'])     # => 200
deep_get(d, ['garbage', 'status_code'])  # => None

A more polished version

def deep_get(d, keys, default=None):
    """
    Example:
        d = {'meta': {'status': 'OK', 'status_code': 200}}
        deep_get(d, ['meta', 'status_code'])          # => 200
        deep_get(d, ['garbage', 'status_code'])       # => None
        deep_get(d, ['meta', 'garbage'], default='-') # => '-'
    """
    assert type(keys) is list
    if d is None:
        return default
    if not keys:
        return d
    return deep_get(d.get(keys[0]), keys[1:], default)

While the reduce approach is neat and short, I think a simple loop is easier to grok. I’ve also included a default parameter.

def deep_get(_dict, keys, default=None):
    for key in keys:
        if isinstance(_dict, dict):
            _dict = _dict.get(key, default)
        else:
            return default
    return _dict

As an exercise to understand how the reduce one-liner worked, I did the following. But ultimately the loop approach seems more intuitive to me.

def deep_get(_dict, keys, default=None):

    def _reducer(d, key):
        if isinstance(d, dict):
            return d.get(key, default)
        return default

    return reduce(_reducer, keys, _dict)

Usage

nested = {'a': {'b': {'c': 42}}}

print deep_get(nested, ['a', 'b'])
print deep_get(nested, ['a', 'b', 'z', 'z'], default='missing')

I suggest you to try python-benedict.

It is a dict subclass that provides keypath support and much more.

Installation: pip install python-benedict

from benedict import benedict

example_dict = benedict(example_dict, keypath_separator='.')

now you can access nested values using keypath:

val = example_dict['key1.key2']

# using 'get' method to avoid a possible KeyError:
val = example_dict.get('key1.key2')

or access nested values using keys list:

val = example_dict['key1', 'key2']

# using get to avoid a possible KeyError:
val = example_dict.get(['key1', 'key2'])

It is well tested and open-source on GitHub:

https://github.com/fabiocaccamo/python-benedict

Note: I am the author of this project

A simple class that can wrap a dict, and retrieve based on a key:

class FindKey(dict):
    def get(self, path, default=None):
        keys = path.split(".")
        val = None

        for key in keys:
            if val:
                if isinstance(val, list):
                    val = [v.get(key, default) if v else None for v in val]
                else:
                    val = val.get(key, default)
            else:
                val = dict.get(self, key, default)

            if not val:
                break

        return val

For example:

person = {'person':{'name':{'first':'John'}}}
FindDict(person).get('person.name.first') # == 'John'

If the key doesn’t exist, it returns None by default. You can override that using a default= key in the FindDict wrapper — for example`:

FindDict(person, default='').get('person.name.last') # == doesn't exist, so ''

for a second level key retrieving, you can do this:

key2_value = (example_dict.get('key1') or {}).get('key2')

After seeing this for deeply getting attributes, I made the following to safely get nested dict values using dot notation. This works for me because my dicts are deserialized MongoDB objects, so I know the key names don’t contain .s. Also, in my context, I can specify a falsy fallback value (None) that I don’t have in my data, so I can avoid the try/except pattern when calling the function.

from functools import reduce # Python 3
def deepgetitem(obj, item, fallback=None):
    """Steps through an item chain to get the ultimate value.

    If ultimate value or path to value does not exist, does not raise
    an exception and instead returns `fallback`.

    >>> d = {'snl_final': {'about': {'_icsd': {'icsd_id': 1}}}}
    >>> deepgetitem(d, 'snl_final.about._icsd.icsd_id')
    1
    >>> deepgetitem(d, 'snl_final.about._sandbox.sbx_id')
    >>>
    """
    def getitem(obj, name):
        try:
            return obj[name]
        except (KeyError, TypeError):
            return fallback
    return reduce(getitem, item.split('.'), obj)

Yet another function for the same thing, also returns a boolean to represent whether the key was found or not and handles some unexpected errors.

'''
json : json to extract value from if exists
path : details.detail.first_name
            empty path represents root

returns a tuple (boolean, object)
        boolean : True if path exists, otherwise False
        object : the object if path exists otherwise None

'''
def get_json_value_at_path(json, path=None, default=None):

    if not bool(path):
        return True, json
    if type(json) is not dict :
        raise ValueError(f'json={json}, path={path} not supported, json must be a dict')
    if type(path) is not str and type(path) is not list:
        raise ValueError(f'path format {path} not supported, path can be a list of strings like [x,y,z] or a string like x.y.z')

    if type(path) is str:
        path = path.strip('.').split('.')
    key = path[0]
    if key in json.keys():
        return get_json_value_at_path(json[key], path[1:], default)
    else:
        return False, default

example usage:

my_json = {'details' : {'first_name' : 'holla', 'last_name' : 'holla'}}
print(get_json_value_at_path(my_json, 'details.first_name', ''))
print(get_json_value_at_path(my_json, 'details.phone', ''))

(True, ‘holla’)

(False, ”)

You can use pydash:

import pydash as _

_.get(example_dict, 'key1.key2', default='Default')

https://pydash.readthedocs.io/en/latest/api.html

An adaptation of unutbu’s answer that I found useful in my own code:

example_dict.setdefaut('key1', {}).get('key2')

It generates a dictionary entry for key1 if it does not have that key already so that you avoid the KeyError. If you want to end up a nested dictionary that includes that key pairing anyway like I did, this seems like the easiest solution.

Since raising an key error if one of keys is missing is a reasonable thing to do, we can even not check for it and get it as single as that:

def get_dict(d, kl):
  cur = d[kl[0]]
  return get_dict(cur, kl[1:]) if len(kl) > 1 else cur

Little improvement to reduce approach to make it work with list. Also using data path as string divided by dots instead of array.

def deep_get(dictionary, path):
    keys = path.split('.')
    return reduce(lambda d, key: d[int(key)] if isinstance(d, list) else d.get(key) if d else None, keys, dictionary)

A solution I’ve used that is similar to the double get but with the additional ability to avoid a TypeError using if else logic:

    value = example_dict['key1']['key2'] if example_dict.get('key1') and example_dict['key1'].get('key2') else default_value

However, the more nested the dictionary the more cumbersome this becomes.

For nested dictionary/JSON lookups, you can use dictor

pip install dictor

dict object

{
    "characters": {
        "Lonestar": {
            "id": 55923,
            "role": "renegade",
            "items": [
                "space winnebago",
                "leather jacket"
            ]
        },
        "Barfolomew": {
            "id": 55924,
            "role": "mawg",
            "items": [
                "peanut butter jar",
                "waggy tail"
            ]
        },
        "Dark Helmet": {
            "id": 99999,
            "role": "Good is dumb",
            "items": [
                "Shwartz",
                "helmet"
            ]
        },
        "Skroob": {
            "id": 12345,
            "role": "Spaceballs CEO",
            "items": [
                "luggage"
            ]
        }
    }
}

to get Lonestar’s items, simply provide a dot-separated path, ie

import json
from dictor import dictor

with open('test.json') as data: 
    data = json.load(data)

print dictor(data, 'characters.Lonestar.items')

>> [u'space winnebago', u'leather jacket']

you can provide fallback value in case the key isnt in path

theres tons more options you can do, like ignore letter casing and using other characters besides ‘.’ as a path separator,

https://github.com/perfecto25/dictor

I little changed this answer. I added checking if we’re using list with numbers. So now we can use it whichever way. deep_get(allTemp, [0], {}) or deep_get(getMinimalTemp, [0, minimalTemperatureKey], 26) etc

def deep_get(_dict, keys, default=None):
    def _reducer(d, key):
        if isinstance(d, dict):
            return d.get(key, default)
        if isinstance(d, list):
            return d[key] if len(d) > 0 else default
        return default
    return reduce(_reducer, keys, _dict)

There are already lots of good answers but I have come up with a function called get similar to lodash get in JavaScript land that also supports reaching into lists by index:

def get(value, keys, default_value = None):
'''
    Useful for reaching into nested JSON like data
    Inspired by JavaScript lodash get and Clojure get-in etc.
'''
  if value is None or keys is None:
      return None
  path = keys.split('.') if isinstance(keys, str) else keys
  result = value
  def valid_index(key):
      return re.match('^([1-9][0-9]*|[0-9])$', key) and int(key) >= 0
  def is_dict_like(v):
      return hasattr(v, '__getitem__') and hasattr(v, '__contains__')
  for key in path:
      if isinstance(result, list) and valid_index(key) and int(key) < len(result):
          result = result[int(key)] if int(key) < len(result) else None
      elif is_dict_like(result) and key in result:
          result = result[key]
      else:
          result = default_value
          break
  return result

def test_get():
  assert get(None, ['foo']) == None
  assert get({'foo': 1}, None) == None
  assert get(None, None) == None
  assert get({'foo': 1}, []) == {'foo': 1}
  assert get({'foo': 1}, ['foo']) == 1
  assert get({'foo': 1}, ['bar']) == None
  assert get({'foo': 1}, ['bar'], 'the default') == 'the default'
  assert get({'foo': {'bar': 'hello'}}, ['foo', 'bar']) == 'hello'
  assert get({'foo': {'bar': 'hello'}}, 'foo.bar') == 'hello'
  assert get({'foo': [{'bar': 'hello'}]}, 'foo.0.bar') == 'hello'
  assert get({'foo': [{'bar': 'hello'}]}, 'foo.1') == None
  assert get({'foo': [{'bar': 'hello'}]}, 'foo.1.bar') == None
  assert get(['foo', 'bar'], '1') == 'bar'
  assert get(['foo', 'bar'], '2') == None