I need to merge multiple dictionaries, here’s what I have for instance:

dict1 = {1:{"a":{A}}, 2:{"b":{B}}}dict2 = {2:{"c":{C}}, 3:{"d":{D}}

With A B C and D being leaves of the tree, like {"info1":"value", "info2":"value2"}

There is an unknown level(depth) of dictionaries, it could be {2:{"c":{"z":{"y":{C}}}}}

In my case it represents a directory/files structure with nodes being docs and leaves being files.

I want to merge them to obtain:

 dict3 = {1:{"a":{A}}, 2:{"b":{B},"c":{C}}, 3:{"d":{D}}}

I’m not sure how I could do that easily with Python.

this is actually quite tricky – particularly if you want a useful error message when things are inconsistent, while correctly accepting duplicate but consistent entries (something no other answer here does….)

assuming you don’t have huge numbers of entries a recursive function is easiest:

def merge(a, b, path=None):    "merges b into a"    if path is None: path = []    for key in b:        if key in a:            if isinstance(a[key], dict) and isinstance(b[key], dict):                merge(a[key], b[key], path + [str(key)])            elif a[key] == b[key]:                pass # same leaf value            else:                raise Exception('Conflict at %s' % '.'.join(path + [str(key)]))        else:            a[key] = b[key]    return a# worksprint(merge({1:{"a":"A"},2:{"b":"B"}}, {2:{"c":"C"},3:{"d":"D"}}))# has conflictmerge({1:{"a":"A"},2:{"b":"B"}}, {1:{"a":"A"},2:{"b":"C"}})

note that this mutates a – the contents of b are added to a (which is also returned). if you want to keep a you could call it like merge(dict(a), b).

agf pointed out (below) that you may have more than two dicts, in which case you can use:

reduce(merge, [dict1, dict2, dict3...])

where everything will be added to dict1.

[note – i edited my initial answer to mutate the first argument; that makes the “reduce” easier to explain]

ps in python 3, you will also need from functools import reduce

Here’s an easy way to do it using generators:

def mergedicts(dict1, dict2):    for k in set(dict1.keys()).union(dict2.keys()):        if k in dict1 and k in dict2:            if isinstance(dict1[k], dict) and isinstance(dict2[k], dict):                yield (k, dict(mergedicts(dict1[k], dict2[k])))            else:                # If one of the values is not a dict, you can't continue merging it.                # Value from second dict overrides one in first and we move on.                yield (k, dict2[k])                # Alternatively, replace this with exception raiser to alert you of value conflicts        elif k in dict1:            yield (k, dict1[k])        else:            yield (k, dict2[k])dict1 = {1:{"a":"A"},2:{"b":"B"}}dict2 = {2:{"c":"C"},3:{"d":"D"}}print dict(mergedicts(dict1,dict2))

This prints:

{1: {'a': 'A'}, 2: {'c': 'C', 'b': 'B'}, 3: {'d': 'D'}}

One issue with this question is that the values of the dict can be arbitrarily complex pieces of data. Based upon these and other answers I came up with this code:

class YamlReaderError(Exception):    passdef data_merge(a, b):    """merges b into a and return merged result    NOTE: tuples and arbitrary objects are not handled as it is totally ambiguous what should happen"""    key = None    # ## debug output    # sys.stderr.write("DEBUG: %s to %s\n" %(b,a))    try:        if a is None or isinstance(a, str) or isinstance(a, unicode) or isinstance(a, int) or isinstance(a, long) or isinstance(a, float):            # border case for first run or if a is a primitive            a = b        elif isinstance(a, list):            # lists can be only appended            if isinstance(b, list):                # merge lists                a.extend(b)            else:                # append to list                a.append(b)        elif isinstance(a, dict):            # dicts must be merged            if isinstance(b, dict):                for key in b:                    if key in a:                        a[key] = data_merge(a[key], b[key])                    else:                        a[key] = b[key]            else:                raise YamlReaderError('Cannot merge non-dict "%s" into dict "%s"' % (b, a))        else:            raise YamlReaderError('NOT IMPLEMENTED "%s" into "%s"' % (b, a))    except TypeError, e:        raise YamlReaderError('TypeError "%s" in key "%s" when merging "%s" into "%s"' % (e, key, b, a))    return a

My use case is merging YAML files where I only have to deal with a subset of possible data types. Hence I can ignore tuples and other objects. For me a sensible merge logic means

• replace scalars
• append lists
• merge dicts by adding missing keys and updating existing keys

Everything else and the unforeseens results in an error.

Dictionaries of dictionaries merge

As this is the canonical question (in spite of certain non-generalities) I’m providing the canonical Pythonic approach to solving this issue.

Simplest Case: “leaves are nested dicts that end in empty dicts”:

d1 = {'a': {1: {'foo': {}}, 2: {}}}d2 = {'a': {1: {}, 2: {'bar': {}}}}d3 = {'b': {3: {'baz': {}}}}d4 = {'a': {1: {'quux': {}}}}

This is the simplest case for recursion, and I would recommend two naive approaches:

def rec_merge1(d1, d2):    '''return new merged dict of dicts'''    for k, v in d1.items(): # in Python 2, use .iteritems()!        if k in d2:            d2[k] = rec_merge1(v, d2[k])    d3 = d1.copy()    d3.update(d2)    return d3def rec_merge2(d1, d2):    '''update first dict with second recursively'''    for k, v in d1.items(): # in Python 2, use .iteritems()!        if k in d2:            d2[k] = rec_merge2(v, d2[k])    d1.update(d2)    return d1

I believe I would prefer the second to the first, but keep in mind that the original state of the first would have to be rebuilt from its origin. Here’s the usage:

>>> from functools import reduce # only required for Python 3.>>> reduce(rec_merge1, (d1, d2, d3, d4)){'a': {1: {'quux': {}, 'foo': {}}, 2: {'bar': {}}}, 'b': {3: {'baz': {}}}}>>> reduce(rec_merge2, (d1, d2, d3, d4)){'a': {1: {'quux': {}, 'foo': {}}, 2: {'bar': {}}}, 'b': {3: {'baz': {}}}}

Complex Case: “leaves are of any other type:”

So if they end in dicts, it’s a simple case of merging the end empty dicts. If not, it’s not so trivial. If strings, how do you merge them? Sets can be updated similarly, so we could give that treatment, but we lose the order in which they were merged. So does order matter?

So in lieu of more information, the simplest approach will be to give them the standard update treatment if both values are not dicts: i.e. the second dict’s value will overwrite the first, even if the second dict’s value is None and the first’s value is a dict with a lot of info.

d1 = {'a': {1: 'foo', 2: None}}d2 = {'a': {1: None, 2: 'bar'}}d3 = {'b': {3: 'baz'}}d4 = {'a': {1: 'quux'}}from collections.abc import MutableMappingdef rec_merge(d1, d2):    '''    Update two dicts of dicts recursively,     if either mapping has leaves that are non-dicts,     the second's leaf overwrites the first's.    '''    for k, v in d1.items():        if k in d2:            # this next check is the only difference!            if all(isinstance(e, MutableMapping) for e in (v, d2[k])):                d2[k] = rec_merge(v, d2[k])            # we could further check types and merge as appropriate here.    d3 = d1.copy()    d3.update(d2)    return d3

And now

from functools import reducereduce(rec_merge, (d1, d2, d3, d4))

returns

{'a': {1: 'quux', 2: 'bar'}, 'b': {3: 'baz'}}

Application to the original question:

I’ve had to remove the curly braces around the letters and put them in single quotes for this to be legit Python (else they would be set literals in Python 2.7+) as well as append a missing brace:

dict1 = {1:{"a":'A'}, 2:{"b":'B'}}dict2 = {2:{"c":'C'}, 3:{"d":'D'}}

and rec_merge(dict1, dict2) now returns:

{1: {'a': 'A'}, 2: {'c': 'C', 'b': 'B'}, 3: {'d': 'D'}}

Which matches the desired outcome of the original question (after changing, e.g. the {A} to 'A'.)

Based on @andrew cooke. This version handles nested lists of dicts and also allows the option to update the values

def merge(a, b, path=None, update=True):    "http://stackoverflow.com/questions/7204805/python-dictionaries-of-dictionaries-merge"    "merges b into a"    if path is None: path = []    for key in b:        if key in a:            if isinstance(a[key], dict) and isinstance(b[key], dict):                merge(a[key], b[key], path + [str(key)])            elif a[key] == b[key]:                pass # same leaf value            elif isinstance(a[key], list) and isinstance(b[key], list):                for idx, val in enumerate(b[key]):                    a[key][idx] = merge(a[key][idx], b[key][idx], path + [str(key), str(idx)], update=update)            elif update:                a[key] = b[key]            else:                raise Exception('Conflict at %s' % '.'.join(path + [str(key)]))        else:            a[key] = b[key]    return a

This simple recursive procedure will merge one dictionary into another while overriding conflicting keys:

#!/usr/bin/env python2.7def merge_dicts(dict1, dict2):    """ Recursively merges dict2 into dict1 """    if not isinstance(dict1, dict) or not isinstance(dict2, dict):        return dict2    for k in dict2:        if k in dict1:            dict1[k] = merge_dicts(dict1[k], dict2[k])        else:            dict1[k] = dict2[k]    return dict1print (merge_dicts({1:{"a":"A"}, 2:{"b":"B"}}, {2:{"c":"C"}, 3:{"d":"D"}}))print (merge_dicts({1:{"a":"A"}, 2:{"b":"B"}}, {1:{"a":"A"}, 2:{"b":"C"}}))

Output:

{1: {'a': 'A'}, 2: {'c': 'C', 'b': 'B'}, 3: {'d': 'D'}}{1: {'a': 'A'}, 2: {'b': 'C'}}

Based on answers from @andrew cooke. It takes care of nested lists in a better way.

def deep_merge_lists(original, incoming):    """    Deep merge two lists. Modifies original.    Recursively call deep merge on each correlated element of list.     If item type in both elements are     a. dict: Call deep_merge_dicts on both values.     b. list: Recursively call deep_merge_lists on both values.     c. any other type: Value is overridden.     d. conflicting types: Value is overridden.    If length of incoming list is more that of original then extra values are appended.    """    common_length = min(len(original), len(incoming))    for idx in range(common_length):        if isinstance(original[idx], dict) and isinstance(incoming[idx], dict):            deep_merge_dicts(original[idx], incoming[idx])        elif isinstance(original[idx], list) and isinstance(incoming[idx], list):            deep_merge_lists(original[idx], incoming[idx])        else:            original[idx] = incoming[idx]    for idx in range(common_length, len(incoming)):        original.append(incoming[idx])def deep_merge_dicts(original, incoming):    """    Deep merge two dictionaries. Modifies original.    For key conflicts if both values are:     a. dict: Recursively call deep_merge_dicts on both values.     b. list: Call deep_merge_lists on both values.     c. any other type: Value is overridden.     d. conflicting types: Value is overridden.    """    for key in incoming:        if key in original:            if isinstance(original[key], dict) and isinstance(incoming[key], dict):                deep_merge_dicts(original[key], incoming[key])            elif isinstance(original[key], list) and isinstance(incoming[key], list):                deep_merge_lists(original[key], incoming[key])            else:                original[key] = incoming[key]        else:            original[key] = incoming[key]

If you have an unknown level of dictionaries, then I would suggest a recursive function:

def combineDicts(dictionary1, dictionary2):    output = {}    for item, value in dictionary1.iteritems():        if dictionary2.has_key(item):            if isinstance(dictionary2[item], dict):                output[item] = combineDicts(value, dictionary2.pop(item))        else:            output[item] = value    for item, value in dictionary2.iteritems():         output[item] = value    return output

Overview

The following approach subdivides the problem of a deep merge of dicts into:

1. A parameterized shallow merge function merge(f)(a,b) that uses afunction f to merge two dicts a and b

2. A recursive merger function f to be used together with merge

Implementation

A function for merging two (non nested) dicts can be written in a lot of ways. I personally like

def merge(f):    def merge(a,b):         keys = a.keys() | b.keys()        return {key:f(a.get(key), b.get(key)) for key in keys}    return merge

A nice way of defining an appropriate recursive merger function f is using multipledispatch which allows to define functions that evaluate along different paths depending on the type of their arguments.

from multipledispatch import dispatch#for anything that is not a dict return@dispatch(object, object)def f(a, b):    return b if b is not None else a#for dicts recurse @dispatch(dict, dict)def f(a,b):    return merge(f)(a,b)

Example

To merge two nested dicts simply use merge(f) e.g.:

dict1 = {1:{"a":"A"},2:{"b":"B"}}dict2 = {2:{"c":"C"},3:{"d":"D"}}merge(f)(dict1, dict2)#returns {1: {'a': 'A'}, 2: {'b': 'B', 'c': 'C'}, 3: {'d': 'D'}} 

Notes:

The advantages of this approach are:

• The function is build from smaller functions that each do a single thing which makes the code simpler to reason about and test

• The behaviour is not hard-coded but can be changed and extended as needed which improves code reuse (see example below).

Customization

Some answers also considered dicts that contain lists e.g. of other (potentially nested) dicts. In this case one might want map over the lists and merge them based on position. This can be done by adding another definition to the merger function f:

import itertools@dispatch(list, list)def f(a,b):    return [merge(f)(*arg) for arg in itertools.zip_longest(a, b)]

In case someone wants yet another approach to this problem, here’s my solution.

Virtues: short, declarative, and functional in style (recursive, does no mutation).

Potential Drawback: This might not be the merge you’re looking for. Consult the docstring for semantics.

def deep_merge(a, b):    """    Merge two values, with `b` taking precedence over `a`.    Semantics:    - If either `a` or `b` is not a dictionary, `a` will be returned only if      `b` is `None`. Otherwise `b` will be returned.    - If both values are dictionaries, they are merged as follows:        * Each key that is found only in `a` or only in `b` will be included in          the output collection with its value intact.        * For any key in common between `a` and `b`, the corresponding values          will be merged with the same semantics.    """    if not isinstance(a, dict) or not isinstance(b, dict):        return a if b is None else b    else:        # If we're here, both a and b must be dictionaries or subtypes thereof.        # Compute set of all keys in both dictionaries.        keys = set(a.keys()) | set(b.keys())        # Build output dictionary, merging recursively values with common keys,        # where `None` is used to mean the absence of a value.        return {            key: deep_merge(a.get(key), b.get(key))            for key in keys        }

You could try mergedeep.

Installation

$ pip3 install mergedeep

Usage

from mergedeep import mergea = {"keyA": 1}b = {"keyB": {"sub1": 10}}c = {"keyB": {"sub2": 20}}merge(a, b, c) print(a)# {"keyA": 1, "keyB": {"sub1": 10, "sub2": 20}}

For a full list of options, check out the docs!

There’s a slight problem with andrew cookes answer: In some cases it modifies the second argument b when you modify the returned dict. Specifically it’s because of this line:

if key in a:    ...else:    a[key] = b[key]

If b[key] is a dict, it will simply be assigned to a, meaning any subsequent modifications to that dict will affect both a and b.

a={}b={'1':{'2':'b'}}c={'1':{'3':'c'}}merge(merge(a,b), c) # {'1': {'3': 'c', '2': 'b'}}a # {'1': {'3': 'c', '2': 'b'}} (as expected)b # {'1': {'3': 'c', '2': 'b'}} <----c # {'1': {'3': 'c'}} (unmodified)

To fix this, the line would have to be substituted with this:

if isinstance(b[key], dict):    a[key] = clone_dict(b[key])else:    a[key] = b[key]

Where clone_dict is:

def clone_dict(obj):    clone = {}    for key, value in obj.iteritems():        if isinstance(value, dict):            clone[key] = clone_dict(value)        else:            clone[key] = value    return

Still. This obviously doesn’t account for list, set and other stuff, but I hope it illustrates the pitfalls when trying to merge dicts.

And for completeness sake, here is my version, where you can pass it multiple dicts:

def merge_dicts(*args):    def clone_dict(obj):        clone = {}        for key, value in obj.iteritems():            if isinstance(value, dict):                clone[key] = clone_dict(value)            else:                clone[key] = value        return    def merge(a, b, path=[]):        for key in b:            if key in a:                if isinstance(a[key], dict) and isinstance(b[key], dict):                    merge(a[key], b[key], path + [str(key)])                elif a[key] == b[key]:                    pass                else:                    raise Exception('Conflict at `{path}\''.format(path='.'.join(path + [str(key)])))            else:                if isinstance(b[key], dict):                    a[key] = clone_dict(b[key])                else:                    a[key] = b[key]        return a    return reduce(merge, args, {})