知识问答

TypeError：无法在re.findall（）中的类似字节的对象上使用字符串模式

内容 隐藏
问题：TypeError：无法在re.findall（）中的类似字节的对象上使用字符串模式
回答 0
回答 1

问题：TypeError：无法在re.findall（）中的类似字节的对象上使用字符串模式

我正在尝试学习如何自动从页面获取URL。在下面的代码中，我试图获取网页的标题：

import urllib.request
import re

url = "http://www.google.com"
regex = r'<title>(,+?)</title>'
pattern  = re.compile(regex)

with urllib.request.urlopen(url) as response:
   html = response.read()

title = re.findall(pattern, html)
print(title)

我得到这个意外的错误：

Traceback (most recent call last):
  File "path\to\file\Crawler.py", line 11, in <module>
    title = re.findall(pattern, html)
  File "C:\Python33\lib\re.py", line 201, in findall
    return _compile(pattern, flags).findall(string)
TypeError: can't use a string pattern on a bytes-like object

我究竟做错了什么？

点击查看英文原文

I am trying to learn how to automatically fetch urls from a page. In the following code I am trying to get the title of the webpage:

import urllib.request
import re

url = "http://www.google.com"
regex = r'<title>(,+?)</title>'
pattern  = re.compile(regex)

with urllib.request.urlopen(url) as response:
   html = response.read()

title = re.findall(pattern, html)
print(title)

And I get this unexpected error:

Traceback (most recent call last):
  File "path\to\file\Crawler.py", line 11, in <module>
    title = re.findall(pattern, html)
  File "C:\Python33\lib\re.py", line 201, in findall
    return _compile(pattern, flags).findall(string)
TypeError: can't use a string pattern on a bytes-like object

What am I doing wrong?

回答 0

要转换的HTML（一个字节状物体）到使用字符串.decode，例如 html = response.read().decode('utf-8')。

请参见将字节转换为Python字符串

点击查看英文原文

You want to convert html (a byte-like object) into a string using .decode, e.g. html = response.read().decode('utf-8').

See Convert bytes to a Python String

回答 1

问题是您的正则表达式是一个字符串，但是html是bytes：

>>> type(html)
<class 'bytes'>

由于python不知道这些字节是如何编码的，因此当您尝试在它们上使用字符串正则表达式时会引发异常。

您可以decode将字节存储为字符串：

html = html.decode('ISO-8859-1')  # encoding may vary!
title = re.findall(pattern, html)  # no more error

或使用字节正则表达式：

regex = rb'<title>(,+?)</title>'
#        ^

在此特定上下文中，您可以从响应标头中获取编码：

with urllib.request.urlopen(url) as response:
    encoding = response.info().get_param('charset', 'utf8')
    html = response.read().decode(encoding)

有关更多详细信息，请参见urlopen文档。

点击查看英文原文

The problem is that your regex is a string, but html is bytes:

>>> type(html)
<class 'bytes'>

Since python doesn’t know how those bytes are encoded, it throws an exception when you try to use a string regex on them.

You can either decode the bytes to a string:

html = html.decode('ISO-8859-1')  # encoding may vary!
title = re.findall(pattern, html)  # no more error

Or use a bytes regex:

regex = rb'<title>(,+?)</title>'
#        ^

In this particular context, you can get the encoding from the response headers:

with urllib.request.urlopen(url) as response:
    encoding = response.info().get_param('charset', 'utf8')
    html = response.read().decode(encoding)

See the urlopen documentation for more details.