标签归档:html-encode

如何使用Python / Django执行HTML解码/编码?

问题:如何使用Python / Django执行HTML解码/编码?

我有一个HTML编码的字符串:

'''<img class="size-medium wp-image-113"\
 style="margin-left: 15px;" title="su1"\
 src="http://blah.org/wp-content/uploads/2008/10/su1-300x194.jpg"\
 alt="" width="300" height="194" />'''

我想将其更改为:

<img class="size-medium wp-image-113" style="margin-left: 15px;" 
  title="su1" src="http://blah.org/wp-content/uploads/2008/10/su1-300x194.jpg" 
  alt="" width="300" height="194" /> 

我希望将其注册为HTML,以便浏览器将其呈现为图像,而不是显示为文本。

字符串的存储方式是这样的,因为我正在使用一种名为的网络抓取工具BeautifulSoup,它将“扫描”网页并从中获取某些内容,然后以该格式返回字符串。

我已经找到了如何在C#中而不是在Python中执行此操作。有人可以帮我吗?

有关

I have a string that is HTML encoded:

'''&lt;img class=&quot;size-medium wp-image-113&quot;\
 style=&quot;margin-left: 15px;&quot; title=&quot;su1&quot;\
 src=&quot;http://blah.org/wp-content/uploads/2008/10/su1-300x194.jpg&quot;\
 alt=&quot;&quot; width=&quot;300&quot; height=&quot;194&quot; /&gt;'''

I want to change that to:

<img class="size-medium wp-image-113" style="margin-left: 15px;" 
  title="su1" src="http://blah.org/wp-content/uploads/2008/10/su1-300x194.jpg" 
  alt="" width="300" height="194" /> 

I want this to register as HTML so that it is rendered as an image by the browser instead of being displayed as text.

The string is stored like that because I am using a web-scraping tool called BeautifulSoup, it “scans” a web-page and gets certain content from it, then returns the string in that format.

I’ve found how to do this in C# but not in Python. Can someone help me out?

Related


回答 0

给定Django用例,对此有两个答案。这是它的django.utils.html.escape功能,以供参考:

def escape(html):
    """Returns the given HTML with ampersands, quotes and carets encoded."""
    return mark_safe(force_unicode(html).replace('&', '&amp;').replace('<', '&l
t;').replace('>', '&gt;').replace('"', '&quot;').replace("'", '&#39;'))

为了解决这个问题,Jake的答案中描述的Cheetah函数应该起作用,但是缺少单引号。此版本包含一个更新的元组,并且替换顺序相反,以避免出现对称问题:

def html_decode(s):
    """
    Returns the ASCII decoded version of the given HTML string. This does
    NOT remove normal HTML tags like <p>.
    """
    htmlCodes = (
            ("'", '&#39;'),
            ('"', '&quot;'),
            ('>', '&gt;'),
            ('<', '&lt;'),
            ('&', '&amp;')
        )
    for code in htmlCodes:
        s = s.replace(code[1], code[0])
    return s

unescaped = html_decode(my_string)

但是,这不是一般的解决方案。仅适用于以编码的字符串django.utils.html.escape。更笼统地说,坚持使用标准库是一个好主意:

# Python 2.x:
import HTMLParser
html_parser = HTMLParser.HTMLParser()
unescaped = html_parser.unescape(my_string)

# Python 3.x:
import html.parser
html_parser = html.parser.HTMLParser()
unescaped = html_parser.unescape(my_string)

# >= Python 3.5:
from html import unescape
unescaped = unescape(my_string)

建议:将未转义的HTML存储在数据库中可能更有意义。如果可能的话,值得一探的是从BeautifulSoup获得未转义的结果,并完全避免此过程。

对于Django,转义仅在模板渲染期间发生;因此,为了防止转义,您只需告诉模板引擎不要转义您的字符串即可。为此,请在模板中使用以下选项之一:

{{ context_var|safe }}
{% autoescape off %}
    {{ context_var }}
{% endautoescape %}

Given the Django use case, there are two answers to this. Here is its django.utils.html.escape function, for reference:

def escape(html):
    """Returns the given HTML with ampersands, quotes and carets encoded."""
    return mark_safe(force_unicode(html).replace('&', '&amp;').replace('<', '&l
t;').replace('>', '&gt;').replace('"', '&quot;').replace("'", '&#39;'))

To reverse this, the Cheetah function described in Jake’s answer should work, but is missing the single-quote. This version includes an updated tuple, with the order of replacement reversed to avoid symmetric problems:

def html_decode(s):
    """
    Returns the ASCII decoded version of the given HTML string. This does
    NOT remove normal HTML tags like <p>.
    """
    htmlCodes = (
            ("'", '&#39;'),
            ('"', '&quot;'),
            ('>', '&gt;'),
            ('<', '&lt;'),
            ('&', '&amp;')
        )
    for code in htmlCodes:
        s = s.replace(code[1], code[0])
    return s

unescaped = html_decode(my_string)

This, however, is not a general solution; it is only appropriate for strings encoded with django.utils.html.escape. More generally, it is a good idea to stick with the standard library:

# Python 2.x:
import HTMLParser
html_parser = HTMLParser.HTMLParser()
unescaped = html_parser.unescape(my_string)

# Python 3.x:
import html.parser
html_parser = html.parser.HTMLParser()
unescaped = html_parser.unescape(my_string)

# >= Python 3.5:
from html import unescape
unescaped = unescape(my_string)

As a suggestion: it may make more sense to store the HTML unescaped in your database. It’d be worth looking into getting unescaped results back from BeautifulSoup if possible, and avoiding this process altogether.

With Django, escaping only occurs during template rendering; so to prevent escaping you just tell the templating engine not to escape your string. To do that, use one of these options in your template:

{{ context_var|safe }}
{% autoescape off %}
    {{ context_var }}
{% endautoescape %}

回答 1

使用标准库:

  • HTML转义

    try:
        from html import escape  # python 3.x
    except ImportError:
        from cgi import escape  # python 2.x
    
    print(escape("<"))
    
  • HTML转义

    try:
        from html import unescape  # python 3.4+
    except ImportError:
        try:
            from html.parser import HTMLParser  # python 3.x (<3.4)
        except ImportError:
            from HTMLParser import HTMLParser  # python 2.x
        unescape = HTMLParser().unescape
    
    print(unescape("&gt;"))
    

With the standard library:

  • HTML Escape

    try:
        from html import escape  # python 3.x
    except ImportError:
        from cgi import escape  # python 2.x
    
    print(escape("<"))
    
  • HTML Unescape

    try:
        from html import unescape  # python 3.4+
    except ImportError:
        try:
            from html.parser import HTMLParser  # python 3.x (<3.4)
        except ImportError:
            from HTMLParser import HTMLParser  # python 2.x
        unescape = HTMLParser().unescape
    
    print(unescape("&gt;"))
    

回答 2

对于html编码,标准库中有cgi.escape

>> help(cgi.escape)
cgi.escape = escape(s, quote=None)
    Replace special characters "&", "<" and ">" to HTML-safe sequences.
    If the optional flag quote is true, the quotation mark character (")
    is also translated.

对于html解码,我使用以下代码:

import re
from htmlentitydefs import name2codepoint
# for some reason, python 2.5.2 doesn't have this one (apostrophe)
name2codepoint['#39'] = 39

def unescape(s):
    "unescape HTML code refs; c.f. http://wiki.python.org/moin/EscapingHtml"
    return re.sub('&(%s);' % '|'.join(name2codepoint),
              lambda m: unichr(name2codepoint[m.group(1)]), s)

对于更复杂的事情,我使用BeautifulSoup。

For html encoding, there’s cgi.escape from the standard library:

>> help(cgi.escape)
cgi.escape = escape(s, quote=None)
    Replace special characters "&", "<" and ">" to HTML-safe sequences.
    If the optional flag quote is true, the quotation mark character (")
    is also translated.

For html decoding, I use the following:

import re
from htmlentitydefs import name2codepoint
# for some reason, python 2.5.2 doesn't have this one (apostrophe)
name2codepoint['#39'] = 39

def unescape(s):
    "unescape HTML code refs; c.f. http://wiki.python.org/moin/EscapingHtml"
    return re.sub('&(%s);' % '|'.join(name2codepoint),
              lambda m: unichr(name2codepoint[m.group(1)]), s)

For anything more complicated, I use BeautifulSoup.


回答 3

如果编码字符集受到相对限制,请使用daniel的解决方案。否则,请使用众多HTML解析库之一。

我喜欢BeautifulSoup,因为它可以处理格式错误的XML / HTML:

http://www.crummy.com/software/BeautifulSoup/

对于您的问题,他们的文档中有一个示例

from BeautifulSoup import BeautifulStoneSoup
BeautifulStoneSoup("Sacr&eacute; bl&#101;u!", 
                   convertEntities=BeautifulStoneSoup.HTML_ENTITIES).contents[0]
# u'Sacr\xe9 bleu!'

Use daniel’s solution if the set of encoded characters is relatively restricted. Otherwise, use one of the numerous HTML-parsing libraries.

I like BeautifulSoup because it can handle malformed XML/HTML :

http://www.crummy.com/software/BeautifulSoup/

for your question, there’s an example in their documentation

from BeautifulSoup import BeautifulStoneSoup
BeautifulStoneSoup("Sacr&eacute; bl&#101;u!", 
                   convertEntities=BeautifulStoneSoup.HTML_ENTITIES).contents[0]
# u'Sacr\xe9 bleu!'

回答 4

在Python 3.4+中:

import html

html.unescape(your_string)

In Python 3.4+:

import html

html.unescape(your_string)

回答 5

请参阅此页面底部的Python Wiki,至少有2个选项可以“取消转义” html。

See at the bottom of this page at Python wiki, there are at least 2 options to “unescape” html.


回答 6

丹尼尔的评论作为答案:

“转义仅发生在Django模板渲染期间。因此,不需要进行转义-您只需告诉模板引擎不要转义。{{context_var | safe}}或{%autoescape off%} {{context_var}} { %endautoescape%}”

Daniel’s comment as an answer:

“escaping only occurs in Django during template rendering. Therefore, there’s no need for an unescape – you just tell the templating engine not to escape. either {{ context_var|safe }} or {% autoescape off %}{{ context_var }}{% endautoescape %}”


回答 7

我在以下位置找到了很好的功能:http : //snippets.dzone.com/posts/show/4569

def decodeHtmlentities(string):
    import re
    entity_re = re.compile("&(#?)(\d{1,5}|\w{1,8});")

    def substitute_entity(match):
        from htmlentitydefs import name2codepoint as n2cp
        ent = match.group(2)
        if match.group(1) == "#":
            return unichr(int(ent))
        else:
            cp = n2cp.get(ent)

            if cp:
                return unichr(cp)
            else:
                return match.group()

    return entity_re.subn(substitute_entity, string)[0]

I found a fine function at: http://snippets.dzone.com/posts/show/4569

def decodeHtmlentities(string):
    import re
    entity_re = re.compile("&(#?)(\d{1,5}|\w{1,8});")

    def substitute_entity(match):
        from htmlentitydefs import name2codepoint as n2cp
        ent = match.group(2)
        if match.group(1) == "#":
            return unichr(int(ent))
        else:
            cp = n2cp.get(ent)

            if cp:
                return unichr(cp)
            else:
                return match.group()

    return entity_re.subn(substitute_entity, string)[0]

回答 8

如果有人在寻找通过django模板执行此操作的简单方法,则可以始终使用以下过滤器:

<html>
{{ node.description|safe }}
</html>

我有一些来自供应商的数据,我发布的所有内容实际上都是在呈现的页面上写的html标签,就像您在查看源代码一样。上面的代码极大地帮助了我。希望这对其他人有帮助。

干杯!!

If anyone is looking for a simple way to do this via the django templates, you can always use filters like this:

<html>
{{ node.description|safe }}
</html>

I had some data coming from a vendor and everything I posted had html tags actually written on the rendered page as if you were looking at the source. The above code helped me greatly. Hope this helps others.

Cheers!!


回答 9

即使这是一个非常老的问题,也可能有效。

的Django 1.5.5

In [1]: from django.utils.text import unescape_entities
In [2]: unescape_entities('&lt;img class=&quot;size-medium wp-image-113&quot; style=&quot;margin-left: 15px;&quot; title=&quot;su1&quot; src=&quot;http://blah.org/wp-content/uploads/2008/10/su1-300x194.jpg&quot; alt=&quot;&quot; width=&quot;300&quot; height=&quot;194&quot; /&gt;')
Out[2]: u'<img class="size-medium wp-image-113" style="margin-left: 15px;" title="su1" src="http://blah.org/wp-content/uploads/2008/10/su1-300x194.jpg" alt="" width="300" height="194" />'

Even though this is a really old question, this may work.

Django 1.5.5

In [1]: from django.utils.text import unescape_entities
In [2]: unescape_entities('&lt;img class=&quot;size-medium wp-image-113&quot; style=&quot;margin-left: 15px;&quot; title=&quot;su1&quot; src=&quot;http://blah.org/wp-content/uploads/2008/10/su1-300x194.jpg&quot; alt=&quot;&quot; width=&quot;300&quot; height=&quot;194&quot; /&gt;')
Out[2]: u'<img class="size-medium wp-image-113" style="margin-left: 15px;" title="su1" src="http://blah.org/wp-content/uploads/2008/10/su1-300x194.jpg" alt="" width="300" height="194" />'

回答 10

我在猎豹的源代码中找到了这个(这里

htmlCodes = [
    ['&', '&amp;'],
    ['<', '&lt;'],
    ['>', '&gt;'],
    ['"', '&quot;'],
]
htmlCodesReversed = htmlCodes[:]
htmlCodesReversed.reverse()
def htmlDecode(s, codes=htmlCodesReversed):
    """ Returns the ASCII decoded version of the given HTML string. This does
        NOT remove normal HTML tags like <p>. It is the inverse of htmlEncode()."""
    for code in codes:
        s = s.replace(code[1], code[0])
    return s

不确定为什么要反转列表,我认为它与编码方式有关,因此对于您而言,可能不需要反转。另外,如果我是我,我会将htmlCodes更改为元组列表,而不是列表列表…尽管这将在我的库中进行:)

我也注意到您的标题也要求编码,所以这是猎豹的编码功能。

def htmlEncode(s, codes=htmlCodes):
    """ Returns the HTML encoded version of the given string. This is useful to
        display a plain ASCII text string on a web page."""
    for code in codes:
        s = s.replace(code[0], code[1])
    return s

I found this in the Cheetah source code (here)

htmlCodes = [
    ['&', '&amp;'],
    ['<', '&lt;'],
    ['>', '&gt;'],
    ['"', '&quot;'],
]
htmlCodesReversed = htmlCodes[:]
htmlCodesReversed.reverse()
def htmlDecode(s, codes=htmlCodesReversed):
    """ Returns the ASCII decoded version of the given HTML string. This does
        NOT remove normal HTML tags like <p>. It is the inverse of htmlEncode()."""
    for code in codes:
        s = s.replace(code[1], code[0])
    return s

not sure why they reverse the list, I think it has to do with the way they encode, so with you it may not need to be reversed. Also if I were you I would change htmlCodes to be a list of tuples rather than a list of lists… this is going in my library though :)

i noticed your title asked for encode too, so here is Cheetah’s encode function.

def htmlEncode(s, codes=htmlCodes):
    """ Returns the HTML encoded version of the given string. This is useful to
        display a plain ASCII text string on a web page."""
    for code in codes:
        s = s.replace(code[0], code[1])
    return s

回答 11

您也可以使用django.utils.html.escape

from django.utils.html import escape

something_nice = escape(request.POST['something_naughty'])

You can also use django.utils.html.escape

from django.utils.html import escape

something_nice = escape(request.POST['something_naughty'])

回答 12

以下是使用module的python函数htmlentitydefs。这不是完美的。htmlentitydefs我所拥有的版本不完整,它假设所有实体都解码到一个代码点,这对于像这样的实体是错误的&NotEqualTilde;

http://www.w3.org/TR/html5/named-character-references.html

NotEqualTilde;     U+02242 U+00338    ≂̸

尽管有这些警告,但这里是代码。

def decodeHtmlText(html):
    """
    Given a string of HTML that would parse to a single text node,
    return the text value of that node.
    """
    # Fast path for common case.
    if html.find("&") < 0: return html
    return re.sub(
        '&(?:#(?:x([0-9A-Fa-f]+)|([0-9]+))|([a-zA-Z0-9]+));',
        _decode_html_entity,
        html)

def _decode_html_entity(match):
    """
    Regex replacer that expects hex digits in group 1, or
    decimal digits in group 2, or a named entity in group 3.
    """
    hex_digits = match.group(1)  # '&#10;' -> unichr(10)
    if hex_digits: return unichr(int(hex_digits, 16))
    decimal_digits = match.group(2)  # '&#x10;' -> unichr(0x10)
    if decimal_digits: return unichr(int(decimal_digits, 10))
    name = match.group(3)  # name is 'lt' when '&lt;' was matched.
    if name:
        decoding = (htmlentitydefs.name2codepoint.get(name)
            # Treat &GT; like &gt;.
            # This is wrong for &Gt; and &Lt; which HTML5 adopted from MathML.
            # If htmlentitydefs included mappings for those entities,
            # then this code will magically work.
            or htmlentitydefs.name2codepoint.get(name.lower()))
        if decoding is not None: return unichr(decoding)
    return match.group(0)  # Treat "&noSuchEntity;" as "&noSuchEntity;"

Below is a python function that uses module htmlentitydefs. It is not perfect. The version of htmlentitydefs that I have is incomplete and it assumes that all entities decode to one codepoint which is wrong for entities like &NotEqualTilde;:

http://www.w3.org/TR/html5/named-character-references.html

NotEqualTilde;     U+02242 U+00338    ≂̸

With those caveats though, here’s the code.

def decodeHtmlText(html):
    """
    Given a string of HTML that would parse to a single text node,
    return the text value of that node.
    """
    # Fast path for common case.
    if html.find("&") < 0: return html
    return re.sub(
        '&(?:#(?:x([0-9A-Fa-f]+)|([0-9]+))|([a-zA-Z0-9]+));',
        _decode_html_entity,
        html)

def _decode_html_entity(match):
    """
    Regex replacer that expects hex digits in group 1, or
    decimal digits in group 2, or a named entity in group 3.
    """
    hex_digits = match.group(1)  # '&#10;' -> unichr(10)
    if hex_digits: return unichr(int(hex_digits, 16))
    decimal_digits = match.group(2)  # '&#x10;' -> unichr(0x10)
    if decimal_digits: return unichr(int(decimal_digits, 10))
    name = match.group(3)  # name is 'lt' when '&lt;' was matched.
    if name:
        decoding = (htmlentitydefs.name2codepoint.get(name)
            # Treat &GT; like &gt;.
            # This is wrong for &Gt; and &Lt; which HTML5 adopted from MathML.
            # If htmlentitydefs included mappings for those entities,
            # then this code will magically work.
            or htmlentitydefs.name2codepoint.get(name.lower()))
        if decoding is not None: return unichr(decoding)
    return match.group(0)  # Treat "&noSuchEntity;" as "&noSuchEntity;"

回答 13

这是解决此问题的最简单方法-

{% autoescape on %}
   {{ body }}
{% endautoescape %}

从此页面

This is the easiest solution for this problem –

{% autoescape on %}
   {{ body }}
{% endautoescape %}

From this page.


回答 14

在Django和Python中搜索此问题的最简单解决方案,我发现您可以使用内置函数来转义/转义html代码。

我将您的html代码保存在scraped_html和中clean_html

scraped_html = (
    '&lt;img class=&quot;size-medium wp-image-113&quot; '
    'style=&quot;margin-left: 15px;&quot; title=&quot;su1&quot; '
    'src=&quot;http://blah.org/wp-content/uploads/2008/10/su1-300x194.jpg&quot; '
    'alt=&quot;&quot; width=&quot;300&quot; height=&quot;194&quot; /&gt;'
)
clean_html = (
    '<img class="size-medium wp-image-113" style="margin-left: 15px;" '
    'title="su1" src="http://blah.org/wp-content/uploads/2008/10/su1-300x194.jpg" '
    'alt="" width="300" height="194" />'
)

Django的

您需要Django> = 1.0

逃生

取消抓取的 HTML代码的转义,可以使用django.utils.text.unescape_entities,其中:

将所有命名和数字字符引用转换为相应的unicode字符。

>>> from django.utils.text import unescape_entities
>>> clean_html == unescape_entities(scraped_html)
True

逃逸

要转义干净的html代码,可以使用django.utils.html.escape,其中:

返回给定文本,该文本带有与符号,引号和尖括号,并编码为在HTML中使用。

>>> from django.utils.html import escape
>>> scraped_html == escape(clean_html)
True

Python

您需要Python> = 3.4

逃生

取消抓取的 html代码,可以使用html.unescape,其中:

转换所有命名和数字字符引用(例如&gt;&#62;&x3e;到对应的Unicode字符字符串s)。

>>> from html import unescape
>>> clean_html == unescape(scraped_html)
True

逃逸

要转义干净的html代码,可以使用html.escape,其中:

转换角色&<>在字符串s到HTML安全序列。

>>> from html import escape
>>> scraped_html == escape(clean_html)
True

Searching the simplest solution of this question in Django and Python I found you can use builtin theirs functions to escape/unescape html code.

Example

I saved your html code in scraped_html and clean_html:

scraped_html = (
    '&lt;img class=&quot;size-medium wp-image-113&quot; '
    'style=&quot;margin-left: 15px;&quot; title=&quot;su1&quot; '
    'src=&quot;http://blah.org/wp-content/uploads/2008/10/su1-300x194.jpg&quot; '
    'alt=&quot;&quot; width=&quot;300&quot; height=&quot;194&quot; /&gt;'
)
clean_html = (
    '<img class="size-medium wp-image-113" style="margin-left: 15px;" '
    'title="su1" src="http://blah.org/wp-content/uploads/2008/10/su1-300x194.jpg" '
    'alt="" width="300" height="194" />'
)

Django

You need Django >= 1.0

unescape

To unescape your scraped html code you can use django.utils.text.unescape_entities which:

Convert all named and numeric character references to the corresponding unicode characters.

>>> from django.utils.text import unescape_entities
>>> clean_html == unescape_entities(scraped_html)
True

escape

To escape your clean html code you can use django.utils.html.escape which:

Returns the given text with ampersands, quotes and angle brackets encoded for use in HTML.

>>> from django.utils.html import escape
>>> scraped_html == escape(clean_html)
True

Python

You need Python >= 3.4

unescape

To unescape your scraped html code you can use html.unescape which:

Convert all named and numeric character references (e.g. &gt;, &#62;, &x3e;) in the string s to the corresponding unicode characters.

>>> from html import unescape
>>> clean_html == unescape(scraped_html)
True

escape

To escape your clean html code you can use html.escape which:

Convert the characters &, < and > in string s to HTML-safe sequences.

>>> from html import escape
>>> scraped_html == escape(clean_html)
True