标签归档:https

如何在Python请求中禁用安全证书检查

问题:如何在Python请求中禁用安全证书检查

我在用

import requests
requests.post(url='https://foo.com', data={'bar':'baz'})

但我收到了request.exceptions.SSLError。该网站的证书已过期,但是我没有发送敏感数据,因此对我来说无关紧要。我可以想象有一个像’verifiy = False’这样的参数可以使用,但是我似乎找不到。

I am using

import requests
requests.post(url='https://foo.com', data={'bar':'baz'})

but I get a request.exceptions.SSLError. The website has an expired certficate, but I am not sending sensitive data, so it doesn’t matter to me. I would imagine there is an argument like ‘verifiy=False’ that I could use, but I can’t seem to find it.


回答 0

文档中

requests如果设置verify为False,也可以忽略验证SSL证书 。

>>> requests.get('https://kennethreitz.com', verify=False)
<Response [200]>

如果您使用的是第三方模块,并且想禁用检查功能,则可以使用上下文管理器来修补requests和更改它,从而使它verify=False成为默认设置并禁止显示警告。

import warnings
import contextlib

import requests
from urllib3.exceptions import InsecureRequestWarning


old_merge_environment_settings = requests.Session.merge_environment_settings

@contextlib.contextmanager
def no_ssl_verification():
    opened_adapters = set()

    def merge_environment_settings(self, url, proxies, stream, verify, cert):
        # Verification happens only once per connection so we need to close
        # all the opened adapters once we're done. Otherwise, the effects of
        # verify=False persist beyond the end of this context manager.
        opened_adapters.add(self.get_adapter(url))

        settings = old_merge_environment_settings(self, url, proxies, stream, verify, cert)
        settings['verify'] = False

        return settings

    requests.Session.merge_environment_settings = merge_environment_settings

    try:
        with warnings.catch_warnings():
            warnings.simplefilter('ignore', InsecureRequestWarning)
            yield
    finally:
        requests.Session.merge_environment_settings = old_merge_environment_settings

        for adapter in opened_adapters:
            try:
                adapter.close()
            except:
                pass

使用方法如下:

with no_ssl_verification():
    requests.get('https://wrong.host.badssl.com/')
    print('It works')

    requests.get('https://wrong.host.badssl.com/', verify=True)
    print('Even if you try to force it to')

requests.get('https://wrong.host.badssl.com/', verify=False)
print('It resets back')

session = requests.Session()
session.verify = True

with no_ssl_verification():
    session.get('https://wrong.host.badssl.com/', verify=True)
    print('Works even here')

try:
    requests.get('https://wrong.host.badssl.com/')
except requests.exceptions.SSLError:
    print('It breaks')

try:
    session.get('https://wrong.host.badssl.com/')
except requests.exceptions.SSLError:
    print('It breaks here again')

请注意,一旦您离开上下文管理器,此代码将关闭处理已打补丁请求的所有打开的适配器。这是因为请求维护每个会话的连接池,并且证书验证每个连接仅进行一次,因此将发生以下意外情况:

>>> import requests
>>> session = requests.Session()
>>> session.get('https://wrong.host.badssl.com/', verify=False)
/usr/local/lib/python3.7/site-packages/urllib3/connectionpool.py:857: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
  InsecureRequestWarning)
<Response [200]>
>>> session.get('https://wrong.host.badssl.com/', verify=True)
/usr/local/lib/python3.7/site-packages/urllib3/connectionpool.py:857: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
  InsecureRequestWarning)
<Response [200]>

From the documentation:

requests can also ignore verifying the SSL certificate if you set verify to False.

>>> requests.get('https://kennethreitz.com', verify=False)
<Response [200]>

If you’re using a third-party module and want to disable the checks, here’s a context manager that monkey patches requests and changes it so that verify=False is the default and suppresses the warning.

import warnings
import contextlib

import requests
from urllib3.exceptions import InsecureRequestWarning


old_merge_environment_settings = requests.Session.merge_environment_settings

@contextlib.contextmanager
def no_ssl_verification():
    opened_adapters = set()

    def merge_environment_settings(self, url, proxies, stream, verify, cert):
        # Verification happens only once per connection so we need to close
        # all the opened adapters once we're done. Otherwise, the effects of
        # verify=False persist beyond the end of this context manager.
        opened_adapters.add(self.get_adapter(url))

        settings = old_merge_environment_settings(self, url, proxies, stream, verify, cert)
        settings['verify'] = False

        return settings

    requests.Session.merge_environment_settings = merge_environment_settings

    try:
        with warnings.catch_warnings():
            warnings.simplefilter('ignore', InsecureRequestWarning)
            yield
    finally:
        requests.Session.merge_environment_settings = old_merge_environment_settings

        for adapter in opened_adapters:
            try:
                adapter.close()
            except:
                pass

Here’s how you use it:

with no_ssl_verification():
    requests.get('https://wrong.host.badssl.com/')
    print('It works')

    requests.get('https://wrong.host.badssl.com/', verify=True)
    print('Even if you try to force it to')

requests.get('https://wrong.host.badssl.com/', verify=False)
print('It resets back')

session = requests.Session()
session.verify = True

with no_ssl_verification():
    session.get('https://wrong.host.badssl.com/', verify=True)
    print('Works even here')

try:
    requests.get('https://wrong.host.badssl.com/')
except requests.exceptions.SSLError:
    print('It breaks')

try:
    session.get('https://wrong.host.badssl.com/')
except requests.exceptions.SSLError:
    print('It breaks here again')

Note that this code closes all open adapters that handled a patched request once you leave the context manager. This is because requests maintains a per-session connection pool and certificate validation happens only once per connection so unexpected things like this will happen:

>>> import requests
>>> session = requests.Session()
>>> session.get('https://wrong.host.badssl.com/', verify=False)
/usr/local/lib/python3.7/site-packages/urllib3/connectionpool.py:857: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
  InsecureRequestWarning)
<Response [200]>
>>> session.get('https://wrong.host.badssl.com/', verify=True)
/usr/local/lib/python3.7/site-packages/urllib3/connectionpool.py:857: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
  InsecureRequestWarning)
<Response [200]>

回答 1

使用requests.packages.urllib3.disable_warnings()verify=Falseon requests方法。

import requests
from urllib3.exceptions import InsecureRequestWarning

# Suppress only the single warning from urllib3 needed.
requests.packages.urllib3.disable_warnings(category=InsecureRequestWarning)

# Set `verify=False` on `requests.post`.
requests.post(url='https://example.com', data={'bar':'baz'}, verify=False)

Use requests.packages.urllib3.disable_warnings() and verify=False on requests methods.

import requests
from urllib3.exceptions import InsecureRequestWarning

# Suppress only the single warning from urllib3 needed.
requests.packages.urllib3.disable_warnings(category=InsecureRequestWarning)

# Set `verify=False` on `requests.post`.
requests.post(url='https://example.com', data={'bar':'baz'}, verify=False)

回答 2

要添加到Blender的答案中,您可以使用禁用所有请求的SSLSession.verify = False

import requests

session = requests.Session()
session.verify = False
session.post(url='https://foo.com', data={'bar':'baz'})

请注意urllib3,(请求使用),强烈建议您不要发送未经验证的HTTPS请求,并会引发InsecureRequestWarning

To add to Blender’s answer, you can disable SSL certificate validation for all requests using Session.verify = False

import requests

session = requests.Session()
session.verify = False
session.post(url='https://example.com', data={'bar':'baz'})

Note that urllib3, (which Requests uses), strongly discourages making unverified HTTPS requests and will raise an InsecureRequestWarning.


回答 3

也可以从环境变量中完成:

export CURL_CA_BUNDLE=""

Also can be done from the environment variable:

export CURL_CA_BUNDLE=""

回答 4

如果您想使用verify = False选项发送准确的发布请求,最快的方法是使用以下代码:

import requests

requests.api.request('post', url, data={'bar':'baz'}, json=None, verify=False)

If you want to send exactly post request with verify=False option, fastest way is to use this code:

import requests

requests.api.request('post', url, data={'bar':'baz'}, json=None, verify=False)

如何查看Python应用程序发送的整个HTTP请求?

问题:如何查看Python应用程序发送的整个HTTP请求?

就我而言,我正在使用该requests库通过HTTPS调用PayPal的API。不幸的是,我从贝宝(PayPal)收到错误消息,贝宝(PayPal)支持人员无法弄清错误是什么或由什么引起的。他们要我“请提供整个请求,包括标头”。

我怎样才能做到这一点?

In my case, I’m using the requests library to call PayPal’s API over HTTPS. Unfortunately, I’m getting an error from PayPal, and PayPal support cannot figure out what the error is or what’s causing it. They want me to “Please provide the entire request, headers included”.

How can I do that?


回答 0

一种简单的方法:启用登录请求的最新版本(1.x及更高版本)。

请求用途http.clientlogging模块配置到控制日志记录级别,如所描述这里

示范

从链接文档中摘录的代码:

import requests
import logging

# These two lines enable debugging at httplib level (requests->urllib3->http.client)
# You will see the REQUEST, including HEADERS and DATA, and RESPONSE with HEADERS but without DATA.
# The only thing missing will be the response.body which is not logged.
try:
    import http.client as http_client
except ImportError:
    # Python 2
    import httplib as http_client
http_client.HTTPConnection.debuglevel = 1

# You must initialize logging, otherwise you'll not see debug output.
logging.basicConfig()
logging.getLogger().setLevel(logging.DEBUG)
requests_log = logging.getLogger("requests.packages.urllib3")
requests_log.setLevel(logging.DEBUG)
requests_log.propagate = True

requests.get('https://httpbin.org/headers')

示例输出

$ python requests-logging.py 
INFO:requests.packages.urllib3.connectionpool:Starting new HTTPS connection (1): httpbin.org
send: 'GET /headers HTTP/1.1\r\nHost: httpbin.org\r\nAccept-Encoding: gzip, deflate, compress\r\nAccept: */*\r\nUser-Agent: python-requests/1.2.0 CPython/2.7.3 Linux/3.2.0-48-generic\r\n\r\n'
reply: 'HTTP/1.1 200 OK\r\n'
header: Content-Type: application/json
header: Date: Sat, 29 Jun 2013 11:19:34 GMT
header: Server: gunicorn/0.17.4
header: Content-Length: 226
header: Connection: keep-alive
DEBUG:requests.packages.urllib3.connectionpool:"GET /headers HTTP/1.1" 200 226

A simple method: enable logging in recent versions of Requests (1.x and higher.)

Requests uses the http.client and logging module configuration to control logging verbosity, as described here.

Demonstration

Code excerpted from the linked documentation:

import requests
import logging

# These two lines enable debugging at httplib level (requests->urllib3->http.client)
# You will see the REQUEST, including HEADERS and DATA, and RESPONSE with HEADERS but without DATA.
# The only thing missing will be the response.body which is not logged.
try:
    import http.client as http_client
except ImportError:
    # Python 2
    import httplib as http_client
http_client.HTTPConnection.debuglevel = 1

# You must initialize logging, otherwise you'll not see debug output.
logging.basicConfig()
logging.getLogger().setLevel(logging.DEBUG)
requests_log = logging.getLogger("requests.packages.urllib3")
requests_log.setLevel(logging.DEBUG)
requests_log.propagate = True

requests.get('https://httpbin.org/headers')

Example Output

$ python requests-logging.py 
INFO:requests.packages.urllib3.connectionpool:Starting new HTTPS connection (1): httpbin.org
send: 'GET /headers HTTP/1.1\r\nHost: httpbin.org\r\nAccept-Encoding: gzip, deflate, compress\r\nAccept: */*\r\nUser-Agent: python-requests/1.2.0 CPython/2.7.3 Linux/3.2.0-48-generic\r\n\r\n'
reply: 'HTTP/1.1 200 OK\r\n'
header: Content-Type: application/json
header: Date: Sat, 29 Jun 2013 11:19:34 GMT
header: Server: gunicorn/0.17.4
header: Content-Length: 226
header: Connection: keep-alive
DEBUG:requests.packages.urllib3.connectionpool:"GET /headers HTTP/1.1" 200 226

回答 1

r = requests.get('https://api.github.com', auth=('user', 'pass'))

r是回应。它具有一个request属性,其中包含您需要的信息。

r.request.allow_redirects  r.request.headers          r.request.register_hook
r.request.auth             r.request.hooks            r.request.response
r.request.cert             r.request.method           r.request.send
r.request.config           r.request.params           r.request.sent
r.request.cookies          r.request.path_url         r.request.session
r.request.data             r.request.prefetch         r.request.timeout
r.request.deregister_hook  r.request.proxies          r.request.url
r.request.files            r.request.redirect         r.request.verify

r.request.headers 给出标题:

{'Accept': '*/*',
 'Accept-Encoding': 'identity, deflate, compress, gzip',
 'Authorization': u'Basic dXNlcjpwYXNz',
 'User-Agent': 'python-requests/0.12.1'}

然后r.request.data将主体作为映射。urllib.urlencode如果他们愿意,可以将其转换为:

import urllib
b = r.request.data
encoded_body = urllib.urlencode(b)

根据响应的类型,.data-attribute可能会丢失,而.body-attribute会在那里。

r = requests.get('https://api.github.com', auth=('user', 'pass'))

r is a response. It has a request attribute which has the information you need.

r.request.allow_redirects  r.request.headers          r.request.register_hook
r.request.auth             r.request.hooks            r.request.response
r.request.cert             r.request.method           r.request.send
r.request.config           r.request.params           r.request.sent
r.request.cookies          r.request.path_url         r.request.session
r.request.data             r.request.prefetch         r.request.timeout
r.request.deregister_hook  r.request.proxies          r.request.url
r.request.files            r.request.redirect         r.request.verify

r.request.headers gives the headers:

{'Accept': '*/*',
 'Accept-Encoding': 'identity, deflate, compress, gzip',
 'Authorization': u'Basic dXNlcjpwYXNz',
 'User-Agent': 'python-requests/0.12.1'}

Then r.request.data has the body as a mapping. You can convert this with urllib.urlencode if they prefer:

import urllib
b = r.request.data
encoded_body = urllib.urlencode(b)

depending on the type of the response the .data-attribute may be missing and a .body-attribute be there instead.


回答 2

您可以使用HTTP Toolkit来做到这一点。

如果您需要在不更改代码的情况下快速执行此操作,则特别有用:您可以从HTTP Toolkit打开终端,从那里正常运行任何Python代码,并且可以查看每个HTTP / HTTPS的全部内容。立即提出要求。

有一个免费版本可以满足您的所有需求,并且它是100%开放源代码。

我是HTTP Toolkit的创建者;我实际上是自己构建的,以便不久前为我解决完全相同的问题!我也曾尝试调试付款集成,但他们的SDK无法正常工作,我无法告知原因,我需要知道实际情况如何对其进行正确修复。这非常令人沮丧,但是能够看到原始流量确实有所帮助。

You can use HTTP Toolkit to do exactly this.

It’s especially useful if you need to do this quickly, with no code changes: you can open a terminal from HTTP Toolkit, run any Python code from there as normal, and you’ll be able to see the full content of every HTTP/HTTPS request immediately.

There’s a free version that can do everything you need, and it’s 100% open source.

I’m the creator of HTTP Toolkit; I actually built it myself to solve the exact same problem for me a while back! I too was trying to debug a payment integration, but their SDK didn’t work, I couldn’t tell why, and I needed to know what was actually going on to properly fix it. It’s very frustrating, but being able to see the raw traffic really helps.


回答 3

如果您使用的是Python 2.x,请尝试安装urllib2打开器。尽管您可能必须将其与您用来击中HTTPS的其他开启器结合使用,但这应该可以打印出标题。

import urllib2
urllib2.install_opener(urllib2.build_opener(urllib2.HTTPHandler(debuglevel=1)))
urllib2.urlopen(url)

If you’re using Python 2.x, try installing a urllib2 opener. That should print out your headers, although you may have to combine that with other openers you’re using to hit the HTTPS.

import urllib2
urllib2.install_opener(urllib2.build_opener(urllib2.HTTPHandler(debuglevel=1)))
urllib2.urlopen(url)

回答 4

verbose配置选项可以让你看到你想要的东西。文档中一个示例

注意:请阅读以下注释:详细的配置选项似乎不再可用。

The verbose configuration option might allow you to see what you want. There is an example in the documentation.

NOTE: Read the comments below: The verbose config options doesn’t seem to be available anymore.