转置/解压缩功能(zip的反函数)?

问题:转置/解压缩功能(zip的反函数)?

我有一个2项元组的列表,我想将它们转换为2个列表,其中第一个包含每个元组中的第一项,第二个包含第二项。

例如:

original = [('a', 1), ('b', 2), ('c', 3), ('d', 4)]
# and I want to become...
result = (['a', 'b', 'c', 'd'], [1, 2, 3, 4])

有内置的功能吗?

I have a list of 2-item tuples and I’d like to convert them to 2 lists where the first contains the first item in each tuple and the second list holds the second item.

For example:

original = [('a', 1), ('b', 2), ('c', 3), ('d', 4)]
# and I want to become...
result = (['a', 'b', 'c', 'd'], [1, 2, 3, 4])

Is there a builtin function that does that?


回答 0

zip是它自己的逆!前提是您使用特殊的*运算符。

>>> zip(*[('a', 1), ('b', 2), ('c', 3), ('d', 4)])
[('a', 'b', 'c', 'd'), (1, 2, 3, 4)]

它的工作方式是通过调用zip参数:

zip(('a', 1), ('b', 2), ('c', 3), ('d', 4))

…除了参数zip直接传递(在转换为元组之后)之外,因此不必担心参数数量太大。

zip is its own inverse! Provided you use the special * operator.

>>> zip(*[('a', 1), ('b', 2), ('c', 3), ('d', 4)])
[('a', 'b', 'c', 'd'), (1, 2, 3, 4)]

The way this works is by calling zip with the arguments:

zip(('a', 1), ('b', 2), ('c', 3), ('d', 4))

… except the arguments are passed to zip directly (after being converted to a tuple), so there’s no need to worry about the number of arguments getting too big.


回答 1

你也可以

result = ([ a for a,b in original ], [ b for a,b in original ])

应该更好地扩展。特别是如果Python除非需要,否则最好不要扩展列表推导。

(顺便说一句,它会组成一个2元组(一对)的列表,而不是一个元组列表,例如 zip。)

如果可以使用生成器而不是实际列表,则可以这样做:

result = (( a for a,b in original ), ( b for a,b in original ))

生成器在您请求每个元素之前不会仔细检查列表,但是另一方面,它们会保留对原始列表的引用。

You could also do

result = ([ a for a,b in original ], [ b for a,b in original ])

It should scale better. Especially if Python makes good on not expanding the list comprehensions unless needed.

(Incidentally, it makes a 2-tuple (pair) of lists, rather than a list of tuples, like zip does.)

If generators instead of actual lists are ok, this would do that:

result = (( a for a,b in original ), ( b for a,b in original ))

The generators don’t munch through the list until you ask for each element, but on the other hand, they do keep references to the original list.


回答 2

如果列表的长度不同,则可能不希望按照Patricks的答案使用zip。这有效:

>>> zip(*[('a', 1), ('b', 2), ('c', 3), ('d', 4)])
[('a', 'b', 'c', 'd'), (1, 2, 3, 4)]

但是使用不同的长度列表,zip会将每个项目截断为最短列表的长度:

>>> zip(*[('a', 1), ('b', 2), ('c', 3), ('d', 4), ('e', )])
[('a', 'b', 'c', 'd', 'e')]

您可以使用不带功能的map来用None填充空白结果:

>>> map(None, *[('a', 1), ('b', 2), ('c', 3), ('d', 4), ('e', )])
[('a', 'b', 'c', 'd', 'e'), (1, 2, 3, 4, None)]

zip()稍快一些。

If you have lists that are not the same length, you may not want to use zip as per Patricks answer. This works:

>>> zip(*[('a', 1), ('b', 2), ('c', 3), ('d', 4)])
[('a', 'b', 'c', 'd'), (1, 2, 3, 4)]

But with different length lists, zip truncates each item to the length of the shortest list:

>>> zip(*[('a', 1), ('b', 2), ('c', 3), ('d', 4), ('e', )])
[('a', 'b', 'c', 'd', 'e')]

You can use map with no function to fill empty results with None:

>>> map(None, *[('a', 1), ('b', 2), ('c', 3), ('d', 4), ('e', )])
[('a', 'b', 'c', 'd', 'e'), (1, 2, 3, 4, None)]

zip() is marginally faster though.


回答 3

我喜欢在程序中使用zip(*iterable)(这是您要查找的代码):

def unzip(iterable):
    return zip(*iterable)

我发现unzip更具可读性。

I like to use zip(*iterable) (which is the piece of code you’re looking for) in my programs as so:

def unzip(iterable):
    return zip(*iterable)

I find unzip more readable.


回答 4

>>> original = [('a', 1), ('b', 2), ('c', 3), ('d', 4)]
>>> tuple([list(tup) for tup in zip(*original)])
(['a', 'b', 'c', 'd'], [1, 2, 3, 4])

给出问题中的列表元组。

list1, list2 = [list(tup) for tup in zip(*original)]

解压缩两个列表。

>>> original = [('a', 1), ('b', 2), ('c', 3), ('d', 4)]
>>> tuple([list(tup) for tup in zip(*original)])
(['a', 'b', 'c', 'd'], [1, 2, 3, 4])

Gives a tuple of lists as in the question.

list1, list2 = [list(tup) for tup in zip(*original)]

Unpacks the two lists.


回答 5

天真的方法

def transpose_finite_iterable(iterable):
    return zip(*iterable)  # `itertools.izip` for Python 2 users

对于(潜在无限)可迭代的有限可迭代(例如list/ tuple/的序列str),效果很好

| |a_00| |a_10| ... |a_n0| |
| |a_01| |a_11| ... |a_n1| |
| |... | |... | ... |... | |
| |a_0i| |a_1i| ... |a_ni| |
| |... | |... | ... |... | |

哪里

  • n in ℕ
  • a_ij对应于-th可迭代的j-th元素i

申请后transpose_finite_iterable我们得到

| |a_00| |a_01| ... |a_0i| ... |
| |a_10| |a_11| ... |a_1i| ... |
| |... | |... | ... |... | ... |
| |a_n0| |a_n1| ... |a_ni| ... |

这种情况的Python示例,其中a_ij == jn == 2

>>> from itertools import count
>>> iterable = [count(), count()]
>>> result = transpose_finite_iterable(iterable)
>>> next(result)
(0, 0)
>>> next(result)
(1, 1)

但是我们不能transpose_finite_iterable再次使用它来返回原始的结构,iterable因为它result是有限迭代的无限迭代(tuple在我们的例子中是s):

>>> transpose_finite_iterable(result)
... hangs ...
Traceback (most recent call last):
  File "...", line 1, in ...
  File "...", line 2, in transpose_finite_iterable
MemoryError

那么我们该如何处理呢?

…这是 deque

看完itertools.teefunction文档后,有一些Python配方可以通过一些修改来帮助解决我们的问题

def transpose_finite_iterables(iterable):
    iterator = iter(iterable)
    try:
        first_elements = next(iterator)
    except StopIteration:
        return ()
    queues = [deque([element])
              for element in first_elements]

    def coordinate(queue):
        while True:
            if not queue:
                try:
                    elements = next(iterator)
                except StopIteration:
                    return
                for sub_queue, element in zip(queues, elements):
                    sub_queue.append(element)
            yield queue.popleft()

    return tuple(map(coordinate, queues))

让我们检查

>>> from itertools import count
>>> iterable = [count(), count()]
>>> result = transpose_finite_iterables(transpose_finite_iterable(iterable))
>>> result
(<generator object transpose_finite_iterables.<locals>.coordinate at ...>, <generator object transpose_finite_iterables.<locals>.coordinate at ...>)
>>> next(result[0])
0
>>> next(result[0])
1

合成

现在我们可以定义通用函数来处理可迭代的可迭代对象,其中一些是有限的,而另一个则可以使用functools.singledispatch装饰器(例如)

from collections import (abc,
                         deque)
from functools import singledispatch


@singledispatch
def transpose(object_):
    """
    Transposes given object.
    """
    raise TypeError('Unsupported object type: {type}.'
                    .format(type=type))


@transpose.register(abc.Iterable)
def transpose_finite_iterables(object_):
    """
    Transposes given iterable of finite iterables.
    """
    iterator = iter(object_)
    try:
        first_elements = next(iterator)
    except StopIteration:
        return ()
    queues = [deque([element])
              for element in first_elements]

    def coordinate(queue):
        while True:
            if not queue:
                try:
                    elements = next(iterator)
                except StopIteration:
                    return
                for sub_queue, element in zip(queues, elements):
                    sub_queue.append(element)
            yield queue.popleft()

    return tuple(map(coordinate, queues))


def transpose_finite_iterable(object_):
    """
    Transposes given finite iterable of iterables.
    """
    yield from zip(*object_)

try:
    transpose.register(abc.Collection, transpose_finite_iterable)
except AttributeError:
    # Python3.5-
    transpose.register(abc.Mapping, transpose_finite_iterable)
    transpose.register(abc.Sequence, transpose_finite_iterable)
    transpose.register(abc.Set, transpose_finite_iterable)

在有限非空可迭代项上的二元运算符类中,可以将其视为自身的逆(数学家称这种函数为“对合”)。


作为singledispatching 的奖励,我们可以处理numpy类似

import numpy as np
...
transpose.register(np.ndarray, np.transpose)

然后像

>>> array = np.arange(4).reshape((2,2))
>>> array
array([[0, 1],
       [2, 3]])
>>> transpose(array)
array([[0, 2],
       [1, 3]])

注意

由于transpose返回迭代器,并且如果有人希望在OP中具有的tuplelist则可以通过map内置函数(例如

>>> original = [('a', 1), ('b', 2), ('c', 3), ('d', 4)]
>>> tuple(map(list, transpose(original)))
(['a', 'b', 'c', 'd'], [1, 2, 3, 4])

广告

我已经添加推广解决方案lz0.5.0版本,可以像使用

>>> from lz.transposition import transpose
>>> list(map(tuple, transpose(zip(range(10), range(10, 20)))))
[(0, 1, 2, 3, 4, 5, 6, 7, 8, 9), (10, 11, 12, 13, 14, 15, 16, 17, 18, 19)]

聚苯乙烯

没有用于处理潜在无限迭代的潜在无限迭代的解决方案(至少很明显),但是这种情况并不常见。

Naive approach

def transpose_finite_iterable(iterable):
    return zip(*iterable)  # `itertools.izip` for Python 2 users

works fine for finite iterable (e.g. sequences like list/tuple/str) of (potentially infinite) iterables which can be illustrated like

| |a_00| |a_10| ... |a_n0| |
| |a_01| |a_11| ... |a_n1| |
| |... | |... | ... |... | |
| |a_0i| |a_1i| ... |a_ni| |
| |... | |... | ... |... | |

where

  • n in ℕ,
  • a_ij corresponds to j-th element of i-th iterable,

and after applying transpose_finite_iterable we get

| |a_00| |a_01| ... |a_0i| ... |
| |a_10| |a_11| ... |a_1i| ... |
| |... | |... | ... |... | ... |
| |a_n0| |a_n1| ... |a_ni| ... |

Python example of such case where a_ij == j, n == 2

>>> from itertools import count
>>> iterable = [count(), count()]
>>> result = transpose_finite_iterable(iterable)
>>> next(result)
(0, 0)
>>> next(result)
(1, 1)

But we can’t use transpose_finite_iterable again to return to structure of original iterable because result is an infinite iterable of finite iterables (tuples in our case):

>>> transpose_finite_iterable(result)
... hangs ...
Traceback (most recent call last):
  File "...", line 1, in ...
  File "...", line 2, in transpose_finite_iterable
MemoryError

So how can we deal with this case?

… and here comes the deque

After we take a look at docs of itertools.tee function, there is Python recipe that with some modification can help in our case

def transpose_finite_iterables(iterable):
    iterator = iter(iterable)
    try:
        first_elements = next(iterator)
    except StopIteration:
        return ()
    queues = [deque([element])
              for element in first_elements]

    def coordinate(queue):
        while True:
            if not queue:
                try:
                    elements = next(iterator)
                except StopIteration:
                    return
                for sub_queue, element in zip(queues, elements):
                    sub_queue.append(element)
            yield queue.popleft()

    return tuple(map(coordinate, queues))

let’s check

>>> from itertools import count
>>> iterable = [count(), count()]
>>> result = transpose_finite_iterables(transpose_finite_iterable(iterable))
>>> result
(<generator object transpose_finite_iterables.<locals>.coordinate at ...>, <generator object transpose_finite_iterables.<locals>.coordinate at ...>)
>>> next(result[0])
0
>>> next(result[0])
1

Synthesis

Now we can define general function for working with iterables of iterables ones of which are finite and another ones are potentially infinite using functools.singledispatch decorator like

from collections import (abc,
                         deque)
from functools import singledispatch


@singledispatch
def transpose(object_):
    """
    Transposes given object.
    """
    raise TypeError('Unsupported object type: {type}.'
                    .format(type=type))


@transpose.register(abc.Iterable)
def transpose_finite_iterables(object_):
    """
    Transposes given iterable of finite iterables.
    """
    iterator = iter(object_)
    try:
        first_elements = next(iterator)
    except StopIteration:
        return ()
    queues = [deque([element])
              for element in first_elements]

    def coordinate(queue):
        while True:
            if not queue:
                try:
                    elements = next(iterator)
                except StopIteration:
                    return
                for sub_queue, element in zip(queues, elements):
                    sub_queue.append(element)
            yield queue.popleft()

    return tuple(map(coordinate, queues))


def transpose_finite_iterable(object_):
    """
    Transposes given finite iterable of iterables.
    """
    yield from zip(*object_)

try:
    transpose.register(abc.Collection, transpose_finite_iterable)
except AttributeError:
    # Python3.5-
    transpose.register(abc.Mapping, transpose_finite_iterable)
    transpose.register(abc.Sequence, transpose_finite_iterable)
    transpose.register(abc.Set, transpose_finite_iterable)

which can be considered as its own inverse (mathematicians call this kind of functions “involutions”) in class of binary operators over finite non-empty iterables.


As a bonus of singledispatching we can handle numpy arrays like

import numpy as np
...
transpose.register(np.ndarray, np.transpose)

and then use it like

>>> array = np.arange(4).reshape((2,2))
>>> array
array([[0, 1],
       [2, 3]])
>>> transpose(array)
array([[0, 2],
       [1, 3]])

Note

Since transpose returns iterators and if someone wants to have a tuple of lists like in OP — this can be made additionally with map built-in function like

>>> original = [('a', 1), ('b', 2), ('c', 3), ('d', 4)]
>>> tuple(map(list, transpose(original)))
(['a', 'b', 'c', 'd'], [1, 2, 3, 4])

I’ve added generalized solution to lz package from 0.5.0 version which can be used like

>>> from lz.transposition import transpose
>>> list(map(tuple, transpose(zip(range(10), range(10, 20)))))
[(0, 1, 2, 3, 4, 5, 6, 7, 8, 9), (10, 11, 12, 13, 14, 15, 16, 17, 18, 19)]

P.S.

There is no solution (at least obvious) for handling potentially infinite iterable of potentially infinite iterables, but this case is less common though.


回答 6

这只是另一种实现方式,但是它对我有很大帮助,所以我在这里写下来:

具有以下数据结构:

X=[1,2,3,4]
Y=['a','b','c','d']
XY=zip(X,Y)

导致:

In: XY
Out: [(1, 'a'), (2, 'b'), (3, 'c'), (4, 'd')]

我认为,将其解压缩并返回原始格式的更Python方式是:

x,y=zip(*XY)

但这返回一个元组,因此如果您需要一个列表,则可以使用:

x,y=(list(x),list(y))

It’s only another way to do it but it helped me a lot so I write it here:

Having this data structure:

X=[1,2,3,4]
Y=['a','b','c','d']
XY=zip(X,Y)

Resulting in:

In: XY
Out: [(1, 'a'), (2, 'b'), (3, 'c'), (4, 'd')]

The more pythonic way to unzip it and go back to the original is this one in my opinion:

x,y=zip(*XY)

But this return a tuple so if you need a list you can use:

x,y=(list(x),list(y))

回答 7

考虑使用more_itertools.unzip

>>> from more_itertools import unzip
>>> original = [('a', 1), ('b', 2), ('c', 3), ('d', 4)]
>>> [list(x) for x in unzip(original)]
[['a', 'b', 'c', 'd'], [1, 2, 3, 4]]     

Consider using more_itertools.unzip:

>>> from more_itertools import unzip
>>> original = [('a', 1), ('b', 2), ('c', 3), ('d', 4)]
>>> [list(x) for x in unzip(original)]
[['a', 'b', 'c', 'd'], [1, 2, 3, 4]]     

回答 8

因为它返回元组(并且可以使用大量内存),所以zip(*zipped)对我来说,这个技巧似乎比有用的还要聪明。

这是一个实际上将为您提供zip反函数的函数。

def unzip(zipped):
    """Inverse of built-in zip function.
    Args:
        zipped: a list of tuples

    Returns:
        a tuple of lists

    Example:
        a = [1, 2, 3]
        b = [4, 5, 6]
        zipped = list(zip(a, b))

        assert zipped == [(1, 4), (2, 5), (3, 6)]

        unzipped = unzip(zipped)

        assert unzipped == ([1, 2, 3], [4, 5, 6])

    """

    unzipped = ()
    if len(zipped) == 0:
        return unzipped

    dim = len(zipped[0])

    for i in range(dim):
        unzipped = unzipped + ([tup[i] for tup in zipped], )

    return unzipped

Since it returns tuples (and can use tons of memory), the zip(*zipped) trick seems more clever than useful, to me.

Here’s a function that will actually give you the inverse of zip.

def unzip(zipped):
    """Inverse of built-in zip function.
    Args:
        zipped: a list of tuples

    Returns:
        a tuple of lists

    Example:
        a = [1, 2, 3]
        b = [4, 5, 6]
        zipped = list(zip(a, b))

        assert zipped == [(1, 4), (2, 5), (3, 6)]

        unzipped = unzip(zipped)

        assert unzipped == ([1, 2, 3], [4, 5, 6])

    """

    unzipped = ()
    if len(zipped) == 0:
        return unzipped

    dim = len(zipped[0])

    for i in range(dim):
        unzipped = unzipped + ([tup[i] for tup in zipped], )

    return unzipped

回答 9

先前的答案都没有有效地提供所需的输出,即列表的元组,而不是元组的列表。对于前者,你可以使用与。区别在于:tuplemap

res1 = list(zip(*original))              # [('a', 'b', 'c', 'd'), (1, 2, 3, 4)]
res2 = tuple(map(list, zip(*original)))  # (['a', 'b', 'c', 'd'], [1, 2, 3, 4])

此外,大多数以前的解决方案都假定使用Python 2.7,在Python 2.7中zip返回列表而不是迭代器。

对于Python 3.x,您需要将结果传递给诸如listtuple耗尽迭代器的函数。对于内存高效的迭代器,您可以省略外部list和外部tuple调用各自的解决方案。

None of the previous answers efficiently provide the required output, which is a tuple of lists, rather than a list of tuples. For the former, you can use tuple with map. Here’s the difference:

res1 = list(zip(*original))              # [('a', 'b', 'c', 'd'), (1, 2, 3, 4)]
res2 = tuple(map(list, zip(*original)))  # (['a', 'b', 'c', 'd'], [1, 2, 3, 4])

In addition, most of the previous solutions assume Python 2.7, where zip returns a list rather than an iterator.

For Python 3.x, you will need to pass the result to a function such as list or tuple to exhaust the iterator. For memory-efficient iterators, you can omit the outer list and tuple calls for the respective solutions.


回答 10

虽然zip(*seq)非常有用,但可能不适用于很长的序列,因为它将创建要传递的值的元组。例如,我一直在使用具有超过一百万个条目的坐标系,并且发现创建它的速度明显更快序列直接。

通用方法如下所示:

from collections import deque
seq = ((a1, b1, …), (a2, b2, …), …)
width = len(seq[0])
output = [deque(len(seq))] * width # preallocate memory
for element in seq:
    for s, item in zip(output, element):
        s.append(item)

但是,根据您要对结果执行的操作,收集的选择可能会产生很大的不同。在我的实际用例中,使用集而不使用内部循环比所有其他方法明显更快。

而且,正如其他人指出的那样,如果您要对数据集执行此操作,则可以改用Numpy或Pandas集合。

While zip(*seq) is very useful, it may be unsuitable for very long sequences as it will create a tuple of values to be passed in. For example, I’ve been working with a coordinate system with over a million entries and find it signifcantly faster to create the sequences directly.

A generic approach would be something like this:

from collections import deque
seq = ((a1, b1, …), (a2, b2, …), …)
width = len(seq[0])
output = [deque(len(seq))] * width # preallocate memory
for element in seq:
    for s, item in zip(output, element):
        s.append(item)

But, depending on what you want to do with the result, the choice of collection can make a big difference. In my actual use case, using sets and no internal loop, is noticeably faster than all other approaches.

And, as others have noted, if you are doing this with datasets, it might make sense to use Numpy or Pandas collections instead.


回答 11

虽然numpy数组和熊猫可能是更可取的,但此函数模仿zip(*args)as时的行为unzip(args)

允许在args迭代值时传递生成器。装饰cls和/或main_cls微管理容器初始化。

def unzip(items, cls=list, main_cls=tuple):
    """Zip function in reverse.

    :param items: Zipped-like iterable.
    :type  items: iterable

    :param cls: Callable that returns iterable with callable append attribute.
        Defaults to `list`.
    :type  cls: callable, optional

    :param main_cls: Callable that returns iterable with callable append
        attribute. Defaults to `tuple`.
    :type  main_cls: callable, optional

    :returns: Unzipped items in instances returned from `cls`, in an instance
        returned from `main_cls`.

    :Example:

        assert unzip(zip(["a","b","c"],[1,2,3])) == (["a","b",c"],[1,2,3])
        assert unzip([("a",1),("b",2),("c",3)]) == (["a","b","c"],[1,2,3])
        assert unzip([("a",1)], deque, list) == [deque(["a"]),deque([1])]
        assert unzip((["a"],["b"]), lambda i: deque(i,1)) == (deque(["b"]),)
    """
    items = iter(items)

    try:
        i = next(items)
    except StopIteration:
        return main_cls()

    unzipped = main_cls(cls([v]) for v in i)

    for i in items:
        for c,v in zip(unzipped,i):
            c.append(v)

    return unzipped

While numpy arrays and pandas may be preferrable, this function imitates the behavior of zip(*args) when called as unzip(args).

Allows for generators to be passed as args as it iterates through values. Decorate cls and/or main_cls to micro manage container initialization.

def unzip(items, cls=list, main_cls=tuple):
    """Zip function in reverse.

    :param items: Zipped-like iterable.
    :type  items: iterable

    :param cls: Callable that returns iterable with callable append attribute.
        Defaults to `list`.
    :type  cls: callable, optional

    :param main_cls: Callable that returns iterable with callable append
        attribute. Defaults to `tuple`.
    :type  main_cls: callable, optional

    :returns: Unzipped items in instances returned from `cls`, in an instance
        returned from `main_cls`.

    :Example:

        assert unzip(zip(["a","b","c"],[1,2,3])) == (["a","b",c"],[1,2,3])
        assert unzip([("a",1),("b",2),("c",3)]) == (["a","b","c"],[1,2,3])
        assert unzip([("a",1)], deque, list) == [deque(["a"]),deque([1])]
        assert unzip((["a"],["b"]), lambda i: deque(i,1)) == (deque(["b"]),)
    """
    items = iter(items)

    try:
        i = next(items)
    except StopIteration:
        return main_cls()

    unzipped = main_cls(cls([v]) for v in i)

    for i in items:
        for c,v in zip(unzipped,i):
            c.append(v)

    return unzipped