用python 3解开python 2对象

问题:用python 3解开python 2对象

我想知道是否有一种方法可以加载在Python 2.4和Python 3.4中腌制的对象。

我一直在大量公司遗留代码上运行2to3,以使其保持最新状态。

完成此操作后,在运行文件时出现以下错误:

  File "H:\fixers - 3.4\addressfixer - 3.4\trunk\lib\address\address_generic.py"
, line 382, in read_ref_files
    d = pickle.load(open(mshelffile, 'rb'))
UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position 1: ordinal
not in range(128)

在争用中查看腌制的对象,它dict在中dict,包含键和type值str

所以我的问题是:有没有办法用python 3.4加载最初在python 2.4中腌制的对象?

I’m wondering if there is a way to load an object that was pickled in Python 2.4, with Python 3.4.

I’ve been running 2to3 on a large amount of company legacy code to get it up to date.

Having done this, when running the file I get the following error:

  File "H:\fixers - 3.4\addressfixer - 3.4\trunk\lib\address\address_generic.py"
, line 382, in read_ref_files
    d = pickle.load(open(mshelffile, 'rb'))
UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position 1: ordinal
not in range(128)

Looking at the pickled object in contention, it’s a dict in a dict, containing keys and values of type str.

So my question is: Is there a way to load an object, originally pickled in python 2.4, with python 3.4?


回答 0

您必须告诉pickle.load()如何将Python字节串数据转换为Python 3字符串,或者可以告诉pickle将它们保留为字节。

默认设置是尝试将所有字符串数据解码为ASCII,并且解码失败。请参阅pickle.load()文档

可选的关键字参数是fix_importsencodingerrors,用于控制对Python 2生成的pickle流的兼容性支持。如果fix_imports为true,pickle将尝试将旧的Python 2名称映射到Python 3中使用的新名称。编码错误告诉pickle如何解码Python 2腌制的8位字符串实例;它们分别默认为“ ASCII”和“ strict”。该编码可以是“字节”来读取这些8位串实例作为字节对象。

将编码设置为latin1可以直接导入数据:

with open(mshelffile, 'rb') as f:
    d = pickle.load(f, encoding='latin1') 

但是您需要确认没有使用错误的编解码器对所有字符串进行解码;Latin-1适用于任何输入,因为它将字节值0-255直接映射到前256个Unicode代码点。

另一种选择是使用加载数据encoding='bytes',然后解码所有bytes键和值。

请注意,直到使用3.6.8、3.7.2和3.8.0之前的Python版本,除非使用,否则对Python 2 datetime对象数据的解泄漏都是无效的encoding='bytes'

You’ll have to tell pickle.load() how to convert Python bytestring data to Python 3 strings, or you can tell pickle to leave them as bytes.

The default is to try and decode all string data as ASCII, and that decoding fails. See the pickle.load() documentation:

Optional keyword arguments are fix_imports, encoding and errors, which are used to control compatibility support for pickle stream generated by Python 2. If fix_imports is true, pickle will try to map the old Python 2 names to the new names used in Python 3. The encoding and errors tell pickle how to decode 8-bit string instances pickled by Python 2; these default to ‘ASCII’ and ‘strict’, respectively. The encoding can be ‘bytes’ to read these 8-bit string instances as bytes objects.

Setting the encoding to latin1 allows you to import the data directly:

with open(mshelffile, 'rb') as f:
    d = pickle.load(f, encoding='latin1') 

but you’ll need to verify that none of your strings are decoded using the wrong codec; Latin-1 works for any input as it maps the byte values 0-255 to the first 256 Unicode codepoints directly.

The alternative would be to load the data with encoding='bytes', and decode all bytes keys and values afterwards.

Note that up to Python versions before 3.6.8, 3.7.2 and 3.8.0, unpickling of Python 2 datetime object data is broken unless you use encoding='bytes'.


回答 1

encoding='latin1'当对象中包含numpy数组时,使用会引起一些问题。

使用encoding='bytes'会更好。

请参阅此答案以获取有关使用的完整说明encoding='bytes'

Using encoding='latin1' causes some issues when your object contains numpy arrays in it.

Using encoding='bytes' will be better.

Please see this answer for complete explanation of using encoding='bytes'