保存和加载对象以及使用泡菜

问题:保存和加载对象以及使用泡菜

我正在尝试使用pickle模块保存和加载对象。
首先,我声明我的对象:

>>> class Fruits:pass
...
>>> banana = Fruits()

>>> banana.color = 'yellow'
>>> banana.value = 30

之后,我打开一个名为“ Fruits.obj”的文件(以前,我创建了一个新的.txt文件,并将其重命名为“ Fruits.obj”):

>>> import pickle
>>> filehandler = open(b"Fruits.obj","wb")
>>> pickle.dump(banana,filehandler)

完成此操作后,我关闭了会话,开始了一个新会话,然后放入下一个会话(尝试访问应该保存的对象):

file = open("Fruits.obj",'r')
object_file = pickle.load(file)

但是我有这个信息:

Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "C:\Python31\lib\pickle.py", line 1365, in load
encoding=encoding, errors=errors).load()
ValueError: read() from the underlying stream did notreturn bytes

我不知道该怎么办,因为我不了解此消息。有人知道我如何加载对象“香蕉”吗?谢谢!

编辑: 正如你们中的一些人所说的那样:

>>> import pickle
>>> file = open("Fruits.obj",'rb')

没问题,但是我要讲的是:

>>> object_file = pickle.load(file)

我有错误:

Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "C:\Python31\lib\pickle.py", line 1365, in load
encoding=encoding, errors=errors).load()
EOFError

I´m trying to save and load objects using pickle module.
First I declare my objects:

>>> class Fruits:pass
...
>>> banana = Fruits()

>>> banana.color = 'yellow'
>>> banana.value = 30

After that I open a file called ‘Fruits.obj'(previously I created a new .txt file and I renamed ‘Fruits.obj’):

>>> import pickle
>>> filehandler = open(b"Fruits.obj","wb")
>>> pickle.dump(banana,filehandler)

After do this I close my session and I began a new one and I put the next (trying to access to the object that it supposed to be saved):

file = open("Fruits.obj",'r')
object_file = pickle.load(file)

But I have this message:

Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "C:\Python31\lib\pickle.py", line 1365, in load
encoding=encoding, errors=errors).load()
ValueError: read() from the underlying stream did notreturn bytes

I don´t know what to do because I don´t understand this message. Does anyone know How I can load my object ‘banana’? Thank you!

EDIT: As some of you have sugested I put:

>>> import pickle
>>> file = open("Fruits.obj",'rb')

There were no problem, but the next I put was:

>>> object_file = pickle.load(file)

And I have error:

Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "C:\Python31\lib\pickle.py", line 1365, in load
encoding=encoding, errors=errors).load()
EOFError

回答 0

至于第二个问题:

 Traceback (most recent call last):
 File "<stdin>", line 1, in <module>
 File "C:\Python31\lib\pickle.py", line
 1365, in load encoding=encoding,
 errors=errors).load() EOFError

读取文件内容后,文件指针将位于文件末尾-不再有其他数据可读取。您必须倒带该文件,以便从头开始再次读取它:

file.seek(0)

但是,您通常要使用上下文管理器打开文件并从中读取数据。这样,文件将在块执行完后自动关闭,这也将帮助您将文件操作组织为有意义的块。

最后,cPickle是C语言中pickle模块的更快实现。因此:

In [1]: import cPickle

In [2]: d = {"a": 1, "b": 2}

In [4]: with open(r"someobject.pickle", "wb") as output_file:
   ...:     cPickle.dump(d, output_file)
   ...:

# pickle_file will be closed at this point, preventing your from accessing it any further

In [5]: with open(r"someobject.pickle", "rb") as input_file:
   ...:     e = cPickle.load(input_file)
   ...:

In [7]: print e
------> print(e)
{'a': 1, 'b': 2}

As for your second problem:

 Traceback (most recent call last):
 File "<stdin>", line 1, in <module>
 File "C:\Python31\lib\pickle.py", line
 1365, in load encoding=encoding,
 errors=errors).load() EOFError

After you have read the contents of the file, the file pointer will be at the end of the file – there will be no further data to read. You have to rewind the file so that it will be read from the beginning again:

file.seek(0)

What you usually want to do though, is to use a context manager to open the file and read data from it. This way, the file will be automatically closed after the block finishes executing, which will also help you organize your file operations into meaningful chunks.

Finally, cPickle is a faster implementation of the pickle module in C. So:

In [1]: import cPickle

In [2]: d = {"a": 1, "b": 2}

In [4]: with open(r"someobject.pickle", "wb") as output_file:
   ...:     cPickle.dump(d, output_file)
   ...:

# pickle_file will be closed at this point, preventing your from accessing it any further

In [5]: with open(r"someobject.pickle", "rb") as input_file:
   ...:     e = cPickle.load(input_file)
   ...:

In [7]: print e
------> print(e)
{'a': 1, 'b': 2}

回答 1

以下对我有用:

class Fruits: pass

banana = Fruits()

banana.color = 'yellow'
banana.value = 30

import pickle

filehandler = open("Fruits.obj","wb")
pickle.dump(banana,filehandler)
filehandler.close()

file = open("Fruits.obj",'rb')
object_file = pickle.load(file)
file.close()

print(object_file.color, object_file.value, sep=', ')
# yellow, 30

The following works for me:

class Fruits: pass

banana = Fruits()

banana.color = 'yellow'
banana.value = 30

import pickle

filehandler = open("Fruits.obj","wb")
pickle.dump(banana,filehandler)
filehandler.close()

file = open("Fruits.obj",'rb')
object_file = pickle.load(file)
file.close()

print(object_file.color, object_file.value, sep=', ')
# yellow, 30

回答 2

您也忘记将其读取为二进制文件。

在您的写作部分中,您有:

open(b"Fruits.obj","wb") # Note the wb part (Write Binary)

在阅读部分中,您有:

file = open("Fruits.obj",'r') # Note the r part, there should be a b too

因此,将其替换为:

file = open("Fruits.obj",'rb')

它将起作用:)


至于第二个错误,很可能是由于未正确关闭/同步文件而引起的。

尝试这段代码来编写:

>>> import pickle
>>> filehandler = open(b"Fruits.obj","wb")
>>> pickle.dump(banana,filehandler)
>>> filehandler.close()

这(不变)为:

>>> import pickle
>>> file = open("Fruits.obj",'rb')
>>> object_file = pickle.load(file)

更整洁的版本将使用该with语句。

写作:

>>> import pickle
>>> with open('Fruits.obj', 'wb') as fp:
>>>     pickle.dump(banana, fp)

阅读:

>>> import pickle
>>> with open('Fruits.obj', 'rb') as fp:
>>>     banana = pickle.load(fp)

You’re forgetting to read it as binary too.

In your write part you have:

open(b"Fruits.obj","wb") # Note the wb part (Write Binary)

In the read part you have:

file = open("Fruits.obj",'r') # Note the r part, there should be a b too

So replace it with:

file = open("Fruits.obj",'rb')

And it will work :)


As for your second error, it is most likely cause by not closing/syncing the file properly.

Try this bit of code to write:

>>> import pickle
>>> filehandler = open(b"Fruits.obj","wb")
>>> pickle.dump(banana,filehandler)
>>> filehandler.close()

And this (unchanged) to read:

>>> import pickle
>>> file = open("Fruits.obj",'rb')
>>> object_file = pickle.load(file)

A neater version would be using the with statement.

For writing:

>>> import pickle
>>> with open('Fruits.obj', 'wb') as fp:
>>>     pickle.dump(banana, fp)

For reading:

>>> import pickle
>>> with open('Fruits.obj', 'rb') as fp:
>>>     banana = pickle.load(fp)

回答 3

在这种情况下,始终以二进制模式打开

file = open("Fruits.obj",'rb')

Always open in binary mode, in this case

file = open("Fruits.obj",'rb')

回答 4

您没有以二进制模式打开文件。

open("Fruits.obj",'rb')

应该管用。

对于第二个错误,文件很可能为空,这意味着您无意中清空了文件或使用了错误的文件名或其他名称。

(这是假设您确实确实关闭了会话。如果没有,则是因为您没有在写入和读取之间关闭文件)。

我测试了您的代码,它可以正常工作。

You didn’t open the file in binary mode.

open("Fruits.obj",'rb')

Should work.

For your second error, the file is most likely empty, which mean you inadvertently emptied it or used the wrong filename or something.

(This is assuming you really did close your session. If not, then it’s because you didn’t close the file between the write and the read).

I tested your code, and it works.


回答 5

看来您想跨会话保存您的类实例,并且使用pickle是一种不错的方法。但是,有一个名为的程序包klepto,将对象的保存抽象到字典接口,因此您可以选择腌制对象并将其保存到文件(如下所示),或腌制对象并将其保存到数据库,或者选择使用pickle使用json或其他许多选项。有趣的klepto是,通过抽象到通用接口,它很容易,因此您不必记住如何通过酸洗保存到文件等其他底层细节。

请注意,它适用于动态添加的类属性,而pickle无法做到这一点…

dude@hilbert>$ python
Python 2.7.6 (default, Nov 12 2013, 13:26:39) 
[GCC 4.2.1 Compatible Apple Clang 4.1 ((tags/Apple/clang-421.11.66))] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> from klepto.archives import file_archive 
>>> db = file_archive('fruits.txt')
>>> class Fruits: pass
... 
>>> banana = Fruits()
>>> banana.color = 'yellow'
>>> banana.value = 30
>>> 
>>> db['banana'] = banana 
>>> db.dump()
>>> 

然后我们重新启动…

dude@hilbert>$ python
Python 2.7.6 (default, Nov 12 2013, 13:26:39) 
[GCC 4.2.1 Compatible Apple Clang 4.1 ((tags/Apple/clang-421.11.66))] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> from klepto.archives import file_archive
>>> db = file_archive('fruits.txt')
>>> db.load()
>>> 
>>> db['banana'].color
'yellow'
>>> 

Klepto 适用于python2和python3。

在此处获取代码:https : //github.com/uqfoundation

It seems you want to save your class instances across sessions, and using pickle is a decent way to do this. However, there’s a package called klepto that abstracts the saving of objects to a dictionary interface, so you can choose to pickle objects and save them to a file (as shown below), or pickle the objects and save them to a database, or instead of use pickle use json, or many other options. The nice thing about klepto is that by abstracting to a common interface, it makes it easy so you don’t have to remember the low-level details of how to save via pickling to a file, or otherwise.

Note that It works for dynamically added class attributes, which pickle cannot do…

dude@hilbert>$ python
Python 2.7.6 (default, Nov 12 2013, 13:26:39) 
[GCC 4.2.1 Compatible Apple Clang 4.1 ((tags/Apple/clang-421.11.66))] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> from klepto.archives import file_archive 
>>> db = file_archive('fruits.txt')
>>> class Fruits: pass
... 
>>> banana = Fruits()
>>> banana.color = 'yellow'
>>> banana.value = 30
>>> 
>>> db['banana'] = banana 
>>> db.dump()
>>> 

Then we restart…

dude@hilbert>$ python
Python 2.7.6 (default, Nov 12 2013, 13:26:39) 
[GCC 4.2.1 Compatible Apple Clang 4.1 ((tags/Apple/clang-421.11.66))] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> from klepto.archives import file_archive
>>> db = file_archive('fruits.txt')
>>> db.load()
>>> 
>>> db['banana'].color
'yellow'
>>> 

Klepto works on python2 and python3.

Get the code here: https://github.com/uqfoundation


回答 6

您可以使用anycache为您完成这项工作。假设您有一个myfunc创建实例的函数:

from anycache import anycache

class Fruits:pass

@anycache(cachedir='/path/to/your/cache')    
def myfunc()
    banana = Fruits()
    banana.color = 'yellow'
    banana.value = 30
return banana

Anycache会myfunc在第一次调用时,cachedir使用唯一的标识符(取决于函数名和参数)作为文件名,将结果腌制到文件中。在任何连续运行中,将加载已腌制的对象。

如果在cachedir两次python运行之间保留了,则腌制的对象将从先前的python运行中获取。

函数参数也被考虑在内。重构的实现也是如此:

from anycache import anycache

class Fruits:pass

@anycache(cachedir='/path/to/your/cache')    
def myfunc(color, value)
    fruit = Fruits()
    fruit.color = color
    fruit.value = value
return fruit

You can use anycache to do the job for you. Assuming you have a function myfunc which creates the instance:

from anycache import anycache

class Fruits:pass

@anycache(cachedir='/path/to/your/cache')    
def myfunc()
    banana = Fruits()
    banana.color = 'yellow'
    banana.value = 30
return banana

Anycache calls myfunc at the first time and pickles the result to a file in cachedir using an unique identifier (depending on the the function name and the arguments) as filename. On any consecutive run, the pickled object is loaded.

If the cachedir is preserved between python runs, the pickled object is taken from the previous python run.

The function arguments are also taken into account. A refactored implementation works likewise:

from anycache import anycache

class Fruits:pass

@anycache(cachedir='/path/to/your/cache')    
def myfunc(color, value)
    fruit = Fruits()
    fruit.color = color
    fruit.value = value
return fruit