Python请求抛出SSLError

问题:Python请求抛出SSLError

我正在研究一个简单的脚本,涉及CAS,jspring安全检查,重定向等。我想使用Kenneth Reitz的python请求,因为这是一项很棒的工作!但是,CAS需要通过SSL进行验证,因此我必须首先通过该步骤。我不知道想要什么Python请求吗?该SSL证书应该存放在哪里?

Traceback (most recent call last):
  File "./test.py", line 24, in <module>
  response = requests.get(url1, headers=headers)
  File "build/bdist.linux-x86_64/egg/requests/api.py", line 52, in get
  File "build/bdist.linux-x86_64/egg/requests/api.py", line 40, in request
  File "build/bdist.linux-x86_64/egg/requests/sessions.py", line 209, in request 
  File "build/bdist.linux-x86_64/egg/requests/models.py", line 624, in send
  File "build/bdist.linux-x86_64/egg/requests/models.py", line 300, in _build_response
  File "build/bdist.linux-x86_64/egg/requests/models.py", line 611, in send
requests.exceptions.SSLError: [Errno 1] _ssl.c:503: error:14090086:SSL routines:SSL3_GET_SERVER_CERTIFICATE:certificate verify failed

I’m working on a simple script that involves CAS, jspring security check, redirection, etc. I would like to use Kenneth Reitz’s python requests because it’s a great piece of work! However, CAS requires getting validated via SSL so I have to get past that step first. I don’t know what Python requests is wanting? Where is this SSL certificate supposed to reside?

Traceback (most recent call last):
  File "./test.py", line 24, in <module>
  response = requests.get(url1, headers=headers)
  File "build/bdist.linux-x86_64/egg/requests/api.py", line 52, in get
  File "build/bdist.linux-x86_64/egg/requests/api.py", line 40, in request
  File "build/bdist.linux-x86_64/egg/requests/sessions.py", line 209, in request 
  File "build/bdist.linux-x86_64/egg/requests/models.py", line 624, in send
  File "build/bdist.linux-x86_64/egg/requests/models.py", line 300, in _build_response
  File "build/bdist.linux-x86_64/egg/requests/models.py", line 611, in send
requests.exceptions.SSLError: [Errno 1] _ssl.c:503: error:14090086:SSL routines:SSL3_GET_SERVER_CERTIFICATE:certificate verify failed

回答 0

您遇到的问题是由不受信任的SSL证书引起的。

就像之前评论中提到的@dirk一样,最快的解决方法是设置verify=False

requests.get('https://example.com', verify=False)

请注意,这将导致证书不被验证。这会使您的应用程序面临安全风险,例如中间人攻击。

当然要运用判断力。正如在评论中提到的,这可能是为快速/一次性应用程序/脚本可以接受的,但真的不应该去制作软件

如果在特定情况下仅跳过证书检查是不可接受的,请考虑以下选项,最好的选择是将verify参数设置为字符串,该字符串是.pem证书文件的路径(应通过某种安全方法获取该字符串)手段)。

因此,从2.0版开始,该verify参数接受以下值及其各自的语义:

  • True:使证书根据库自身的可信证书颁发机构进行验证(注意:您可以通过Certifi库查看哪些根证书请求使用,Certifi库是从Requests:Certifi-Human Trust Database中提取的RC的信任数据库)。
  • False完全绕过证书验证。
  • CA_BUNDLE文件的路径,供请求用于验证证书。

来源:请求-SSL证书验证

还要看一下cert同一链接上的参数。

The problem you are having is caused by an untrusted SSL certificate.

Like @dirk mentioned in a previous comment, the quickest fix is setting verify=False:

requests.get('https://example.com', verify=False)

Please note that this will cause the certificate not to be verified. This will expose your application to security risks, such as man-in-the-middle attacks.

Of course, apply judgment. As mentioned in the comments, this may be acceptable for quick/throwaway applications/scripts, but really should not go to production software.

If just skipping the certificate check is not acceptable in your particular context, consider the following options, your best option is to set the verify parameter to a string that is the path of the .pem file of the certificate (which you should obtain by some sort of secure means).

So, as of version 2.0, the verify parameter accepts the following values, with their respective semantics:

  • True: causes the certificate to validated against the library’s own trusted certificate authorities (Note: you can see which Root Certificates Requests uses via the Certifi library, a trust database of RCs extracted from Requests: Certifi – Trust Database for Humans).
  • False: bypasses certificate validation completely.
  • Path to a CA_BUNDLE file for Requests to use to validate the certificates.

Source: Requests – SSL Cert Verification

Also take a look at the cert parameter on the same link.


回答 1

关于SSL验证的请求文档中

就像网络浏览器一样,请求可以验证HTTPS请求的SSL证书。要检查主机的SSL证书,可以使用verify参数:

>>> requests.get('https://kennethreitz.com', verify=True)

如果您不想验证自己的SSL证书,请输入 verify=False

From requests documentation on SSL verification:

Requests can verify SSL certificates for HTTPS requests, just like a web browser. To check a host’s SSL certificate, you can use the verify argument:

>>> requests.get('https://kennethreitz.com', verify=True)

If you don’t want to verify your SSL certificate, make verify=False


回答 2

要使用的CA文件名可以通过以下方式传递verify

cafile = 'cacert.pem' # http://curl.haxx.se/ca/cacert.pem
r = requests.get(url, verify=cafile)

如果使用,verify=Truerequests使用它自己的CA集,该CA集可能没有用于签署服务器证书的CA。

The name of CA file to use you could pass via verify:

cafile = 'cacert.pem' # http://curl.haxx.se/ca/cacert.pem
r = requests.get(url, verify=cafile)

If you use verify=True then requests uses its own CA set that might not have CA that signed your server certificate.


回答 3

$ pip install -U requests[security]

  • 已在Python 2.7.6 @ Ubuntu 14.04.4 LTS上测试
  • 在Python 2.7.5 @ MacOSX 10.9.5(Mavericks)上测试

打开此问题时(2012-05),请求版本为0.13.1。在版本2.4.1(2014-09)上,引入了“安全”附加功能,并使用certifi软件包(如果有)。

目前(2016-09)主版本为2.11.1,如果没有 ,则可以正常使用verify=False。无需使用requests.get(url, verify=False),如果已安装requests[security]其他功能。

$ pip install -U requests[security]

  • Tested on Python 2.7.6 @ Ubuntu 14.04.4 LTS
  • Tested on Python 2.7.5 @ MacOSX 10.9.5 (Mavericks)

When this question was opened (2012-05) the Requests version was 0.13.1. On version 2.4.1 (2014-09) the “security” extras were introduced, using certifi package if available.

Right now (2016-09) the main version is 2.11.1, that works good without verify=False. No need to use requests.get(url, verify=False), if installed with requests[security] extras.


回答 4

使用aws boto3时遇到相同的问题,并且ssl证书验证失败的问题,通过查看boto3代码,我发现REQUESTS_CA_BUNDLE未设置,因此我通过手动设置解决了这两个问题:

from boto3.session import Session
import os

# debian
os.environ['REQUESTS_CA_BUNDLE'] = os.path.join(
    '/etc/ssl/certs/',
    'ca-certificates.crt')
# centos
#   'ca-bundle.crt')

对于aws-cli,我想将REQUESTS_CA_BUNDLE设置为~/.bashrc可以解决此问题(未经测试,因为我的aws-cli没有它就可以工作)。

REQUESTS_CA_BUNDLE=/etc/ssl/certs/ca-certificates.crt # ca-bundle.crt
export REQUESTS_CA_BUNDLE

I encountered the same issue and ssl certificate verify failed issue when using aws boto3, by review boto3 code, I found the REQUESTS_CA_BUNDLE is not set, so I fixed the both issue by setting it manually:

from boto3.session import Session
import os

# debian
os.environ['REQUESTS_CA_BUNDLE'] = os.path.join(
    '/etc/ssl/certs/',
    'ca-certificates.crt')
# centos
#   'ca-bundle.crt')

For aws-cli, I guess setting REQUESTS_CA_BUNDLE in ~/.bashrc will fix this issue (not tested because my aws-cli works without it).

REQUESTS_CA_BUNDLE=/etc/ssl/certs/ca-certificates.crt # ca-bundle.crt
export REQUESTS_CA_BUNDLE

回答 5

如果您有一个依赖的库requests并且不能修改验证路径(如pyvmomi),则必须找到cacert.pem与请求捆绑在一起的文件,然后在其中附加您的CA。这是找到cacert.pem位置的通用方法:

视窗

C:\>python -c "import requests; print requests.certs.where()"
c:\Python27\lib\site-packages\requests-2.8.1-py2.7.egg\requests\cacert.pem

linux

#  (py2.7.5,requests 2.7.0, verify not enforced)
root@host:~/# python -c "import requests; print requests.certs.where()"
/usr/lib/python2.7/dist-packages/certifi/cacert.pem

#  (py2.7.10, verify enforced)
root@host:~/# python -c "import requests; print requests.certs.where()"
/usr/local/lib/python2.7/dist-packages/requests/cacert.pem

顺便说一句 @ requests-devs,将自己的cacerts与请求捆绑在一起确实非常烦人……尤其是您似乎没有先使用ca ca系统存储这一事实,并且在任何地方都没有记录。

更新

在使用库且无法控制ca-bundle位置的情况下,还可以将ca-bundle位置显式设置为主机范围的ca-bundle:

REQUESTS_CA_BUNDLE=/etc/ssl/certs/ca-bundle.crt python -c "import requests; requests.get('https://somesite.com';)"

In case you have a library that relies on requests and you cannot modify the verify path (like with pyvmomi) then you’ll have to find the cacert.pem bundled with requests and append your CA there. Here’s a generic approach to find the cacert.pem location:

windows

C:\>python -c "import requests; print requests.certs.where()"
c:\Python27\lib\site-packages\requests-2.8.1-py2.7.egg\requests\cacert.pem

linux

#  (py2.7.5,requests 2.7.0, verify not enforced)
root@host:~/# python -c "import requests; print requests.certs.where()"
/usr/lib/python2.7/dist-packages/certifi/cacert.pem

#  (py2.7.10, verify enforced)
root@host:~/# python -c "import requests; print requests.certs.where()"
/usr/local/lib/python2.7/dist-packages/requests/cacert.pem

btw. @requests-devs, bundling your own cacerts with request is really, really annoying… especially the fact that you do not seem to use the system ca store first and this is not documented anywhere.

update

in situations, where you’re using a library and have no control over the ca-bundle location you could also explicitly set the ca-bundle location to be your host-wide ca-bundle:

REQUESTS_CA_BUNDLE=/etc/ssl/certs/ca-bundle.crt python -c "import requests; requests.get('https://somesite.com';)"

回答 6

使用gspread会遇到相同的问题,这些命令对我有用:

sudo pip uninstall -y certifi
sudo pip install certifi==2015.04.28

I face the same problem using gspread and these commands works for me:

sudo pip uninstall -y certifi
sudo pip install certifi==2015.04.28

回答 7

如果要删除警告,请使用以下代码。

import urllib3

urllib3.disable_warnings()

以及verify=Falsewith request.getpostmethod

If you want to remove the warnings, use the code below.

import urllib3

urllib3.disable_warnings()

and verify=False with request.get or post method


回答 8

我找到了解决类似问题的特定方法。这个想法是指向存储在系统上的cacert文件,并由另一个基于ssl的应用程序使用。

在Debian中(我不确定其他发行版中是否相同),证书文件(.pem)存储在/etc/ssl/certs/So,这是对我有用的代码:

import requests
verify='/etc/ssl/certs/cacert.org.pem'
response = requests.get('https://lists.cacert.org', verify=verify)

为了猜测pem选择哪个文件,我浏览了该URL,然后检查哪个证书颁发机构(CA)生成了证书。

编辑:如果您不能编辑代码(因为正在运行第三个应用程序),则可以尝试将pem证书直接添加到其中/usr/local/lib/python2.7/dist-packages/requests/cacert.pem(例如,将证书复制到文件末尾)。

I have found an specific approach for solving a similar issue. The idea is pointing the cacert file stored at the system and used by another ssl based applications.

In Debian (I’m not sure if same in other distributions) the certificate files (.pem) are stored at /etc/ssl/certs/ So, this is the code that work for me:

import requests
verify='/etc/ssl/certs/cacert.org.pem'
response = requests.get('https://lists.cacert.org', verify=verify)

For guessing what pem file choose, I have browse to the url and check which Certificate Authority (CA) has generated the certificate.

EDIT: if you cannot edit the code (because you are running a third app) you can try to add the pem certificate directly into /usr/local/lib/python2.7/dist-packages/requests/cacert.pem (e.g. copying it to the end of the file).


回答 9

如果您不关心证书,请使用verify=False

import requests

url = "Write your url here"

returnResponse = requests.get(url, verify=False)

If you don’t bother about certificate just use verify=False.

import requests

url = "Write your url here"

returnResponse = requests.get(url, verify=False)

回答 10

经过数小时的调试,我只能使用以下软件包来使其工作:

requests[security]==2.7.0  # not 2.18.1
cryptography==1.9  # not 2.0

使用 OpenSSL 1.0.2g 1 Mar 2016

没有这些软件包将verify=False无法正常工作。

我希望这可以帮助别人。

After hours of debugging I could only get this to work using the following packages:

requests[security]==2.7.0  # not 2.18.1
cryptography==1.9  # not 2.0

using OpenSSL 1.0.2g 1 Mar 2016

Without these packages verify=False was not working.

I hope this helps someone.


回答 11

我遇到了同样的问题。原来我没有在服务器上安装中间证书(只需将其附加到证书的底部,如下所示)。

https://www.digicert.com/ssl-support/pem-ssl-creation.htm

确保已安装ca-certificates软件包:

sudo apt-get install ca-certificates

更新时间也可以解决此问题:

sudo apt-get install ntpdate
sudo ntpdate -u ntp.ubuntu.com

如果您使用的是自签名证书,则可能必须手动将其添加到系统中。

I ran into the same issue. Turns out I hadn’t installed the intermediate certificate on my server (just append it to the bottom of your certificate as seen below).

https://www.digicert.com/ssl-support/pem-ssl-creation.htm

Make sure you have the ca-certificates package installed:

sudo apt-get install ca-certificates

Updating the time may also resolve this:

sudo apt-get install ntpdate
sudo ntpdate -u ntp.ubuntu.com

If you’re using a self-signed certificate, you’ll probably have to add it to your system manually.


回答 12

如果请求调用被埋在代码的深处,并且您不想安装服务器证书,则仅出于调试目的,可以对请求进行monkeypatch:

import requests.api
import warnings


def requestspatch(method, url, **kwargs):
    kwargs['verify'] = False
    return _origcall(method, url, **kwargs)

_origcall = requests.api.request
requests.api.request = requestspatch
warnings.warn('Patched requests: SSL verification disabled!')

切勿在生产中使用!

If the request calls are buried somewhere deep in the code and you do not want to install the server certificate, then, just for debug purposes only, it’s possible to monkeypatch requests:

import requests.api
import warnings


def requestspatch(method, url, **kwargs):
    kwargs['verify'] = False
    return _origcall(method, url, **kwargs)

_origcall = requests.api.request
requests.api.request = requestspatch
warnings.warn('Patched requests: SSL verification disabled!')

Never use in production!


回答 13

我想参加聚会太晚了,但我想为像我这样的流浪者粘贴修复程序!所以以下内容在Python 3.7.x上为我解决了

在终端中输入以下内容

pip install --upgrade certifi      # hold your breath..

尝试再次运行您的脚本/请求,看看它是否有效(我确定它不会被修复!)。如果不起作用,请尝试直接在终端中运行以下命令

open /Applications/Python\ 3.6/Install\ Certificates.command  # please replace 3.6 here with your suitable python version

Too late to the party I guess but I wanted to paste the fix for fellow wanderers like myself! So the following worked out for me on Python 3.7.x

Type the following in your terminal

pip install --upgrade certifi      # hold your breath..

Try running your script/requests again and see if it works (I’m sure it won’t be fixed yet!). If it didn’t work then try running the following command in the terminal directly

open /Applications/Python\ 3.6/Install\ Certificates.command  # please replace 3.6 here with your suitable python version

回答 14

我为HOURS争取了这个问题。

我试图更新请求。然后,我更新了证书。我指出了对certifi.where()的验证(无论如何,代码默认情况下都会这样做)。没事。

最后,我将python版本更新为python 2.7.11。我使用的是Python 2.7.5,它与验证证书的方式有些不兼容。一旦我更新了Python(以及其他一些依赖项),它便开始工作。

I fought this problem for HOURS.

I tried to update requests. Then I updated certifi. I pointed verify to certifi.where() (The code does this by default anyways). Nothing worked.

Finally I updated my version of python to python 2.7.11. I was on Python 2.7.5 which had some incompatibilities with the way that the certificates are verified. Once I updated Python (and a handful of other dependencies) it started working.


回答 15

这类似于@ rafael-almeida的答案,但我想指出,从请求2.11+开始,没有3个值verify可以使用,实际上有4个:

  • True:根据请求的内部可信CA进行验证。
  • False完全绕过证书验证。(不建议)
  • CA_BUNDLE文件的路径。请求将使用它来验证服务器的证书。
  • 包含公共证书文件的目录的路径。请求将使用它来验证服务器的证书。

我剩下的答案是关于#4,如何使用包含证书的目录进行验证:

获取所需的公共证书并将其放置在目录中。

严格来说,您可能“应该”使用带外方法来获取证书,但是您也可以仅使用任何浏览器下载它们。

如果服务器使用证书链,请确保获取链中的每个证书。

根据请求文档,必须首先使用“ rehash”实用程序(openssl rehash)处理包含证书的目录。

(这需要openssl 1.1.1+,并且并非所有Windows openssl实施都支持rehash。如果openssl rehash不适合您,则可以尝试在https://github.com/ruby/openssl/blob/master上运行rehash ruby​​脚本。/sample/c_rehash.rb,尽管我还没有尝试过。

我在获取要求识别我的证书的请求时遇到了一些麻烦,但是在使用openssl x509 -outform PEM命令将证书转换为Base64 .pem格式后,一切工作正常。

您也可以只进行懒散的重新哈希处理:

try:
    # As long as the certificates in the certs directory are in the OS's certificate store, `verify=True` is fine.
    return requests.get(url, auth=auth, verify=True)
except requests.exceptions.SSLError:
    subprocess.run(f"openssl rehash -compat -v my_certs_dir", shell=True, check=True)
    return requests.get(url, auth=auth, verify="my_certs_dir")

This is similar to @rafael-almeida ‘s answer, but I want to point out that as of requests 2.11+, there are not 3 values that verify can take, there are actually 4:

  • True: validates against requests’s internal trusted CAs.
  • False: bypasses certificate validation completely. (Not recommended)
  • Path to a CA_BUNDLE file. requests will use this to validate the server’s certificates.
  • Path to a directory containing public certificate files. requests will use this to validate the server’s certificates.

The rest of my answer is about #4, how to use a directory containing certificates to validate:

Obtain the public certificates needed and place them in a directory.

Strictly speaking, you probably “should” use an out-of-band method of obtaining the certificates, but you could also just download them using any browser.

If the server uses a certificate chain, be sure to obtain every single certificate in the chain.

According to the requests documentation, the directory containing the certificates must first be processed with the “rehash” utility (openssl rehash).

(This requires openssl 1.1.1+, and not all Windows openssl implementations support rehash. If openssl rehash won’t work for you, you could try running the rehash ruby script at https://github.com/ruby/openssl/blob/master/sample/c_rehash.rb , though I haven’t tried this. )

I had some trouble with getting requests to recognize my certificates, but after I used the openssl x509 -outform PEM command to convert the certs to Base64 .pem format, everything worked perfectly.

You can also just do lazy rehashing:

try:
    # As long as the certificates in the certs directory are in the OS's certificate store, `verify=True` is fine.
    return requests.get(url, auth=auth, verify=True)
except requests.exceptions.SSLError:
    subprocess.run(f"openssl rehash -compat -v my_certs_dir", shell=True, check=True)
    return requests.get(url, auth=auth, verify="my_certs_dir")

回答 16

目前,请求模块中存在一个导致此错误的问题,存在于v2.6.2至v2.12.4(ATOW)中:https : //github.com/kennethreitz/requests/issues/2573

解决此问题的方法是添加以下行: requests.packages.urllib3.util.ssl_.DEFAULT_CIPHERS = 'ECDH+AESGCM:DH+AESGCM:ECDH+AES256:DH+AES256:ECDH+AES128:DH+AES:ECDH+3DES:DH+3DES:RSA+AESGCM:RSA+AES:RSA+3DES:!aNULL:!MD5:!DSS'

There is currently an issue in the requests module causing this error, present in v2.6.2 to v2.12.4 (ATOW): https://github.com/kennethreitz/requests/issues/2573

Workaround for this issue is adding the following line: requests.packages.urllib3.util.ssl_.DEFAULT_CIPHERS = 'ECDH+AESGCM:DH+AESGCM:ECDH+AES256:DH+AES256:ECDH+AES128:DH+AES:ECDH+3DES:DH+3DES:RSA+AESGCM:RSA+AES:RSA+3DES:!aNULL:!MD5:!DSS'


回答 17

如@Rafael Almeida所述,您遇到的问题是由不受信任的SSL证书引起的。就我而言,我的服务器不信任SSL证书。为了解决此问题而不损害安全性,我下载了证书并将其安装在服务器上(只需双击.crt文件,然后安装证书…)。

As mentioned by @Rafael Almeida, the problem you are having is caused by an untrusted SSL certificate. In my case, the SSL certificate was untrusted by my server. To get around this without compromising security, I downloaded the certificate, and installed it on the server (by simply double clicking on the .crt file and then Install Certificate…).


回答 18

如果正在从另一个包中调用请求,则添加选项是不可行的。在那种情况下,将证书添加到cacert捆绑包是直接的方法,例如,我必须添加“ StartCom Class 1 Primary Intermediate Server CA”,为此,我将根证书下载到StartComClass1.pem中。鉴于我的virtualenv名为caldav,我添加了以下证书:

cat StartComClass1.pem >> .virtualenvs/caldav/lib/python2.7/site-packages/pip/_vendor/requests/cacert.pem
cat temp/StartComClass1.pem >> .virtualenvs/caldav/lib/python2.7/site-packages/requests/cacert.pem

其中之一可能就足够了,我没有检查

It is not feasible to add options if requests is being called from another package. In that case adding certificates to the cacert bundle is the straight path, e.g. I had to add “StartCom Class 1 Primary Intermediate Server CA”, for which I downloaded the root cert into StartComClass1.pem. given my virtualenv is named caldav, I added the certificate with:

cat StartComClass1.pem >> .virtualenvs/caldav/lib/python2.7/site-packages/pip/_vendor/requests/cacert.pem
cat temp/StartComClass1.pem >> .virtualenvs/caldav/lib/python2.7/site-packages/requests/cacert.pem

one of those might be enough, I did not check


回答 19

我遇到了相似或相同的认证验证问题。我读到的OpenSSL版本低于1.0.2,该请求有时取决于验证强证书的困难(请参阅此处)。CentOS 7似乎使用了1.0.1e,这似乎有问题。

我不确定如何在CentOS上解决此问题,因此我决定允许使用较弱的1024位CA证书。

import certifi # This should be already installed as a dependency of 'requests'
requests.get("https://example.com", verify=certifi.old_where())

I was having a similar or the same certification validation problem. I read that OpenSSL versions less than 1.0.2, which requests depends upon sometimes have trouble validating strong certificates (see here). CentOS 7 seems to use 1.0.1e which seems to have the problem.

I wasn’t sure how to get around this problem on CentOS, so I decided to allow weaker 1024bit CA certificates.

import certifi # This should be already installed as a dependency of 'requests'
requests.get("https://example.com", verify=certifi.old_where())

回答 20

我必须从Python 3.4.0升级到3.4.6

pyenv virtualenv 3.4.6 myvenv
pyenv activate myvenv
pip install -r requirements.txt

I had to upgrade from Python 3.4.0 to 3.4.6

pyenv virtualenv 3.4.6 myvenv
pyenv activate myvenv
pip install -r requirements.txt

回答 21

就我而言,原因是无关紧要的。

我知道SSL验证已经进行了几天,实际上是在另一台机器上工作。

我的下一步是比较正在验证的计算机和未进行验证的计算机之间的证书内容和大小。

这很快导致我确定“工作不正确”的计算机上的证书不好,一旦我将其替换为“好”证书,一切就很好了。

In my case the reason was fairly trivial.

I had known that the SSL verification had worked until a few days earlier, and was infact working on a different machine.

My next step was to compare the certificate contents and size between the machine on which verification was working, and the one on which it was not.

This quickly led to me determining that the Certificate on the ‘incorrectly’ working machine was not good, and once I replaced it with the ‘good’ cert, everything was fine.