RuntimeWarning:numpy.dtype大小已更改,可能表明二进制不兼容

问题:RuntimeWarning:numpy.dtype大小已更改,可能表明二进制不兼容

我尝试加载已保存的SVM模型时遇到此错误。我尝试卸载sklearn,NumPy和SciPy,然后再次重新安装最新版本(使用pip)。我仍然收到此错误。为什么?

In [1]: import sklearn; print sklearn.__version__
0.18.1
In [3]: import numpy; print numpy.__version__
1.11.2
In [5]: import scipy; print scipy.__version__
0.18.1
In [7]: import pandas; print pandas.__version__
0.19.1

In [10]: clf = joblib.load('model/trained_model.pkl')
---------------------------------------------------------------------------
RuntimeWarning                            Traceback (most recent call last)
<ipython-input-10-5e5db1331757> in <module>()
----> 1 clf = joblib.load('sentiment_classification/model/trained_model.pkl')

/usr/local/lib/python2.7/dist-packages/sklearn/externals/joblib/numpy_pickle.pyc in load(filename, mmap_mode)
    573                     return load_compatibility(fobj)
    574
--> 575                 obj = _unpickle(fobj, filename, mmap_mode)
    576
    577     return obj

/usr/local/lib/python2.7/dist-packages/sklearn/externals/joblib/numpy_pickle.pyc in _unpickle(fobj, filename, mmap_mode)
    505     obj = None
    506     try:
--> 507         obj = unpickler.load()
    508         if unpickler.compat_mode:
    509             warnings.warn("The file '%s' has been generated with a "

/usr/lib/python2.7/pickle.pyc in load(self)
    862             while 1:
    863                 key = read(1)
--> 864                 dispatch[key](self)
    865         except _Stop, stopinst:
    866             return stopinst.value

/usr/lib/python2.7/pickle.pyc in load_global(self)
   1094         module = self.readline()[:-1]
   1095         name = self.readline()[:-1]
-> 1096         klass = self.find_class(module, name)
   1097         self.append(klass)
   1098     dispatch[GLOBAL] = load_global

/usr/lib/python2.7/pickle.pyc in find_class(self, module, name)
   1128     def find_class(self, module, name):
   1129         # Subclasses may override this
-> 1130         __import__(module)
   1131         mod = sys.modules[module]
   1132         klass = getattr(mod, name)

/usr/local/lib/python2.7/dist-packages/sklearn/svm/__init__.py in <module>()
     11 # License: BSD 3 clause (C) INRIA 2010
     12
---> 13 from .classes import SVC, NuSVC, SVR, NuSVR, OneClassSVM, LinearSVC, \
     14         LinearSVR
     15 from .bounds import l1_min_c

/usr/local/lib/python2.7/dist-packages/sklearn/svm/classes.py in <module>()
      2 import numpy as np
      3
----> 4 from .base import _fit_liblinear, BaseSVC, BaseLibSVM
      5 from ..base import BaseEstimator, RegressorMixin
      6 from ..linear_model.base import LinearClassifierMixin, SparseCoefMixin, \

/usr/local/lib/python2.7/dist-packages/sklearn/svm/base.py in <module>()
      6 from abc import ABCMeta, abstractmethod
      7
----> 8 from . import libsvm, liblinear
      9 from . import libsvm_sparse
     10 from ..base import BaseEstimator, ClassifierMixin

__init__.pxd in init sklearn.svm.libsvm (sklearn/svm/libsvm.c:10207)()

RuntimeWarning: numpy.dtype size changed, may indicate binary incompatibility. Expected 96, got 80

更新:确定,请按照此处

pip uninstall -y scipy scikit-learn
pip install --no-binary scipy scikit-learn

该错误现在已经消失了,尽管我仍然不知道为什么会首先发生它。

I have this error for trying to load a saved SVM model. I have tried uninstalling sklearn, NumPy and SciPy, reinstalling the latest versions all-together again (using pip). I am still getting this error. Why?

In [1]: import sklearn; print sklearn.__version__
0.18.1
In [3]: import numpy; print numpy.__version__
1.11.2
In [5]: import scipy; print scipy.__version__
0.18.1
In [7]: import pandas; print pandas.__version__
0.19.1

In [10]: clf = joblib.load('model/trained_model.pkl')
---------------------------------------------------------------------------
RuntimeWarning                            Traceback (most recent call last)
<ipython-input-10-5e5db1331757> in <module>()
----> 1 clf = joblib.load('sentiment_classification/model/trained_model.pkl')

/usr/local/lib/python2.7/dist-packages/sklearn/externals/joblib/numpy_pickle.pyc in load(filename, mmap_mode)
    573                     return load_compatibility(fobj)
    574
--> 575                 obj = _unpickle(fobj, filename, mmap_mode)
    576
    577     return obj

/usr/local/lib/python2.7/dist-packages/sklearn/externals/joblib/numpy_pickle.pyc in _unpickle(fobj, filename, mmap_mode)
    505     obj = None
    506     try:
--> 507         obj = unpickler.load()
    508         if unpickler.compat_mode:
    509             warnings.warn("The file '%s' has been generated with a "

/usr/lib/python2.7/pickle.pyc in load(self)
    862             while 1:
    863                 key = read(1)
--> 864                 dispatch[key](self)
    865         except _Stop, stopinst:
    866             return stopinst.value

/usr/lib/python2.7/pickle.pyc in load_global(self)
   1094         module = self.readline()[:-1]
   1095         name = self.readline()[:-1]
-> 1096         klass = self.find_class(module, name)
   1097         self.append(klass)
   1098     dispatch[GLOBAL] = load_global

/usr/lib/python2.7/pickle.pyc in find_class(self, module, name)
   1128     def find_class(self, module, name):
   1129         # Subclasses may override this
-> 1130         __import__(module)
   1131         mod = sys.modules[module]
   1132         klass = getattr(mod, name)

/usr/local/lib/python2.7/dist-packages/sklearn/svm/__init__.py in <module>()
     11 # License: BSD 3 clause (C) INRIA 2010
     12
---> 13 from .classes import SVC, NuSVC, SVR, NuSVR, OneClassSVM, LinearSVC, \
     14         LinearSVR
     15 from .bounds import l1_min_c

/usr/local/lib/python2.7/dist-packages/sklearn/svm/classes.py in <module>()
      2 import numpy as np
      3
----> 4 from .base import _fit_liblinear, BaseSVC, BaseLibSVM
      5 from ..base import BaseEstimator, RegressorMixin
      6 from ..linear_model.base import LinearClassifierMixin, SparseCoefMixin, \

/usr/local/lib/python2.7/dist-packages/sklearn/svm/base.py in <module>()
      6 from abc import ABCMeta, abstractmethod
      7
----> 8 from . import libsvm, liblinear
      9 from . import libsvm_sparse
     10 from ..base import BaseEstimator, ClassifierMixin

__init__.pxd in init sklearn.svm.libsvm (sklearn/svm/libsvm.c:10207)()

RuntimeWarning: numpy.dtype size changed, may indicate binary incompatibility. Expected 96, got 80

UPDATE: OK, by following here, and

pip uninstall -y scipy scikit-learn
pip install --no-binary scipy scikit-learn

The error has now gone, though I still have no idea why it occurred in the first place…


回答 0

根据MAINT:沉默有关更改dtype / ufunc大小的Cython警告。-numpy / numpy

每当您导入针对比已安装的numpy早的numpy编译的scipy(或其他软件包)时,这些警告都是可见的。

支票由Cython插入(因此,在任何使用它编译的模块中都存在)。

长话短说,这些警告在的特定情况下应该是良性的numpy,并且numpy 1.8(此提交进入的分支)开始这些消息就会被过滤掉。虽然scikit-learn 0.18.1针对编译numpy 1.6.1

要自己过滤这些警告,您可以执行与补丁程序相同的操作

import warnings
warnings.filterwarnings("ignore", message="numpy.dtype size changed")
warnings.filterwarnings("ignore", message="numpy.ufunc size changed")

当然,你可以重新编译从源代码的所有受影响的模块对当地numpypip install --no-binary :all:¹ ,而不是如果你有该工具。


更长的故事:补丁程序的支持者声称专门针对numpy,应该没有任何风险,并且第3方软件包是有意针对较早版本构建的:

[针对当前的numpy重建所有内容]不是可行的解决方案,当然也没有必要。Scipy(与许多其他软件包一样)与numpy的许多版本兼容。因此,当我们分发scipy二进制文件时,我们将根据支持的最低numpy版本(截至目前为1.5.1)构建它们,并且它们也可与1.6.x,1.7.x和numpy master一起使用。

真正的正确之处在于,Cython仅在dtypes / ufuncs的大小发生变化而导致破坏ABI时发出警告,否则保持沉默。

结果,Cython的开发人员同意信任numpy团队手工维护二进制兼容性,因此我们可以期望使用具有破坏性的ABI更改的版本会产生特制的异常或某些其他显式的阻止程序。


¹ 自以来,先前可用的--no-use-wheel选项已被删除。pip 10.0.0

According to MAINT: silence Cython warnings about changes dtype/ufunc size. – numpy/numpy:

These warnings are visible whenever you import scipy (or another package) that was compiled against an older numpy than is installed.

and the checks are inserted by Cython (hence are present in any module compiled with it).

Long story short, these warnings should be benign in the particular case of numpy, and these messages are filtered out since numpy 1.8 (the branch this commit went onto). While scikit-learn 0.18.1 is compiled against numpy 1.6.1.

To filter these warnings yourself, you can do the same as the patch does:

import warnings
warnings.filterwarnings("ignore", message="numpy.dtype size changed")
warnings.filterwarnings("ignore", message="numpy.ufunc size changed")

Of course, you can just recompile all affected modules from source against your local numpy with pip install --no-binary :all:¹ instead if you have the balls tools for that.


Longer story: the patch’s proponent claims there should be no risk specifically with numpy, and 3rd-party packages are intentionally built against older versions:

[Rebuilding everything against current numpy is] not a feasible solution, and certainly shouldn’t be necessary. Scipy (as many other packages) is compatible with a number of versions of numpy. So when we distribute scipy binaries, we build them against the lowest supported numpy version (1.5.1 as of now) and they work with 1.6.x, 1.7.x and numpy master as well.

The real correct would be for Cython only to issue warnings when the size of dtypes/ufuncs has changes in a way that breaks the ABI, and be silent otherwise.

As a result, Cython’s devs agreed to trust the numpy team with maintaining binary compatibility by hand, so we can probably expect that using versions with breaking ABI changes would yield a specially-crafted exception or some other explicit show-stopper.


¹The previously available --no-use-wheel option has been removed since pip 10.0.0.


回答 1

这是新的numpy版本(1.15.0)的问题

您可以降级numpy,此问题将得到解决:

sudo pip uninstall numpy
sudo pip install numpy==1.14.5

最终发布了numpy 1.15.1版本,从而解决了警告问题。

须藤点安装numpy == 1.15.1

这正在工作..

It’s the issue of new numpy version (1.15.0)

You can downgrade numpy and this problem will be fixed:

sudo pip uninstall numpy
sudo pip install numpy==1.14.5

Finally numpy 1.15.1 version is released so the warning issues are fixed.

sudo pip install numpy==1.15.1

This is working..


回答 2

如果您在anaconda环境中,请使用:

conda update --all

if you are in an anaconda environment use:

conda update --all

回答 3

我已经尝试了上述方法,但是没有任何效果。但是在我通过apt install安装库之后,问题就消失了,

对于Python3,

pip3 uninstall -y numpy scipy pandas scikit-learn
sudo apt update
sudo apt install python3-numpy python3-scipy python3-pandas python3-sklearn 

对于Python2,

pip uninstall -y numpy scipy pandas scikit-learn
sudo apt update
sudo apt install python-numpy python-scipy python-pandas python-sklearn 

希望有帮助。

I’ve tried the above-mentioned ways, but nothing worked. But the issue was gone after I installed the libraries through apt install,

For Python3,

pip3 uninstall -y numpy scipy pandas scikit-learn
sudo apt update
sudo apt install python3-numpy python3-scipy python3-pandas python3-sklearn 

For Python2,

pip uninstall -y numpy scipy pandas scikit-learn
sudo apt update
sudo apt install python-numpy python-scipy python-pandas python-sklearn 

Hope that helps.


回答 4

只需升级您的numpy模块,现在它是1.15.4。对于窗户

pip install numpy --upgrade

Just upgrade your numpy module, right now it is 1.15.4. For windows

pip install numpy --upgrade

回答 5

发生此错误是因为已安装的软件包是numpy的另一个版本。
我们需要针对本地重建scipy和scikit-learnnumpy

对于新的pip(在我的情况下pip 18.0),它起作用:

pip uninstall -y scipy scikit-learn
pip install --no-binary scipy,scikit-learn -I scipy scikit-learn

--no-binary列出您要忽略二进制文件的软件包的名称列表。在这种情况下,我们通过--no-binary scipy,scikit-learn了它将忽略软件包scipy,scikit-learn的二进制文件。没帮我

This error occurs because the installed packages were build agains different version of numpy.
We need to rebuild scipy and scikit-learn against the local numpy.

For new pip (in my case pip 18.0) this worked:

pip uninstall -y scipy scikit-learn
pip install --no-binary scipy,scikit-learn -I scipy scikit-learn

--no-binary takes a list of names of packages that you want to ignore binaries for. In this case we passed --no-binary scipy,scikit-learn which will ignore binaries for packages scipy,scikit-learn. Didn’t help me


回答 6

元信息:安装sklearn的推荐方法

如果您已经可以正常安装numpy和scipy,则安装scikit-learn的最简单方法是使用 pip

pip install -U scikit-learn 

conda

conda install scikit-learn

[…不要使用pip从源代码编译]

如果您还没有一个Python安装与numpy的和SciPy的,我们建议通过你的包管理器,或通过任何安装一个 python的 。这些带有numpy,scipy,scikit-learn,matplotlib和许多其他有用的科学和数据处理库。

Meta-information: The recommended way to install sklearn

If you already have a working installation of numpy and scipy, the easiest way to install scikit-learn is using pip

pip install -U scikit-learn 

or conda:

conda install scikit-learn

[… do not compile from source using pip]

If you don’t already have a python installation with numpy and scipy, we recommend to install either via your package manager or via a python bundle. These come with numpy, scipy, scikit-learn, matplotlib and many other helpful scientific and data processing libraries.


回答 7

请注意,从cython 0.29开始,有一个新的check_size选项可以消除源代码中的警告,因此,一旦该版本渗入到各种软件包中,则无需任何解决方法

Note that as of cython 0.29 there is a new check_size option that eliminates the warning at the source, so no work-arounds should be needed once that version percolates to the various packages


回答 8

我的环境是Python 2.7.15

我尝试

pip uninstall
pip install --no-use-wheel

但它不起作用。它显示错误:

没有这样的选择:–no-use-wheel

然后我尝试:

pip uninstall
pip install --user --install-option="--prefix=" -U scikit-learn

而且有效:不会显示无用的警告。

My enviroment is Python 2.7.15

I try

pip uninstall
pip install --no-use-wheel

but it does not work. It shows the error:

no such option: –no-use-wheel

Then I try:

pip uninstall
pip install --user --install-option="--prefix=" -U scikit-learn

And it works: the useless warnings do not show.


回答 9

导入scipy时,错误信息显示:RuntimeWarning:内置 .type大小已更改,可能表示二进制不兼容。预期的zd,得到了zd

我通过将python版本从2.7.2更新到2.7.13解决了这个问题

When import scipy, error info shows: RuntimeWarning: builtin.type size changed, may indicate binary incompatibility. Expected zd, got zd

I solved this problem by updating python version from 2.7.2 to 2.7.13