I want this to register as HTML so that it is rendered as an image by the browser instead of being displayed as text.
The string is stored like that because I am using a web-scraping tool called BeautifulSoup, it “scans” a web-page and gets certain content from it, then returns the string in that format.
I’ve found how to do this in C# but not in Python. Can someone help me out?
def escape(html):"""Returns the given HTML with ampersands, quotes and carets encoded."""return mark_safe(force_unicode(html).replace('&','&').replace('<','&l
t;').replace('>','>').replace('"','"').replace("'",'''))
def html_decode(s):"""
Returns the ASCII decoded version of the given HTML string. This does
NOT remove normal HTML tags like <p>.
"""
htmlCodes =(("'",'''),('"','"'),('>','>'),('<','<'),('&','&'))for code in htmlCodes:
s = s.replace(code[1], code[0])return s
unescaped = html_decode(my_string)
Given the Django use case, there are two answers to this. Here is its django.utils.html.escape function, for reference:
def escape(html):
"""Returns the given HTML with ampersands, quotes and carets encoded."""
return mark_safe(force_unicode(html).replace('&', '&').replace('<', '&l
t;').replace('>', '>').replace('"', '"').replace("'", '''))
To reverse this, the Cheetah function described in Jake’s answer should work, but is missing the single-quote. This version includes an updated tuple, with the order of replacement reversed to avoid symmetric problems:
def html_decode(s):
"""
Returns the ASCII decoded version of the given HTML string. This does
NOT remove normal HTML tags like <p>.
"""
htmlCodes = (
("'", '''),
('"', '"'),
('>', '>'),
('<', '<'),
('&', '&')
)
for code in htmlCodes:
s = s.replace(code[1], code[0])
return s
unescaped = html_decode(my_string)
This, however, is not a general solution; it is only appropriate for strings encoded with django.utils.html.escape. More generally, it is a good idea to stick with the standard library:
As a suggestion: it may make more sense to store the HTML unescaped in your database. It’d be worth looking into getting unescaped results back from BeautifulSoup if possible, and avoiding this process altogether.
With Django, escaping only occurs during template rendering; so to prevent escaping you just tell the templating engine not to escape your string. To do that, use one of these options in your template:
try:
from html import escape # python 3.x
except ImportError:
from cgi import escape # python 2.x
print(escape("<"))
HTML Unescape
try:
from html import unescape # python 3.4+
except ImportError:
try:
from html.parser import HTMLParser # python 3.x (<3.4)
except ImportError:
from HTMLParser import HTMLParser # python 2.x
unescape = HTMLParser().unescape
print(unescape(">"))
回答 2
对于html编码，标准库中有cgi.escape：
>> help(cgi.escape)
cgi.escape = escape(s, quote=None)Replace special characters "&","<"and">" to HTML-safe sequences.If the optional flag quote is true, the quotation mark character (")
is also translated.
对于html解码，我使用以下代码：
import refrom htmlentitydefs import name2codepoint# for some reason, python 2.5.2 doesn't have this one (apostrophe)
name2codepoint['#39']=39def unescape(s):"unescape HTML code refs; c.f. http://wiki.python.org/moin/EscapingHtml"return re.sub('&(%s);'%'|'.join(name2codepoint),lambda m: unichr(name2codepoint[m.group(1)]), s)
For html encoding, there’s cgi.escape from the standard library:
>> help(cgi.escape)
cgi.escape = escape(s, quote=None)
Replace special characters "&", "<" and ">" to HTML-safe sequences.
If the optional flag quote is true, the quotation mark character (")
is also translated.
For html decoding, I use the following:
import re
from htmlentitydefs import name2codepoint
# for some reason, python 2.5.2 doesn't have this one (apostrophe)
name2codepoint['#39'] = 39
def unescape(s):
"unescape HTML code refs; c.f. http://wiki.python.org/moin/EscapingHtml"
return re.sub('&(%s);' % '|'.join(name2codepoint),
lambda m: unichr(name2codepoint[m.group(1)]), s)
For anything more complicated, I use BeautifulSoup.
“escaping only occurs in Django during template rendering. Therefore, there’s no need for an unescape – you just tell the templating engine not to escape. either {{ context_var|safe }} or {% autoescape off %}{{ context_var }}{% endautoescape %}”
If anyone is looking for a simple way to do this via the django templates, you can always use filters like this:
<html>
{{ node.description|safe }}
</html>
I had some data coming from a vendor and everything I posted had html tags actually written on the rendered page as if you were looking at the source. The above code helped me greatly.
Hope this helps others.
htmlCodes =[['&','&'],['<','<'],['>','>'],['"','"'],]
htmlCodesReversed = htmlCodes[:]
htmlCodesReversed.reverse()def htmlDecode(s, codes=htmlCodesReversed):""" Returns the ASCII decoded version of the given HTML string. This does
NOT remove normal HTML tags like <p>. It is the inverse of htmlEncode()."""for code in codes:
s = s.replace(code[1], code[0])return s
def htmlEncode(s, codes=htmlCodes):""" Returns the HTML encoded version of the given string. This is useful to
display a plain ASCII text string on a web page."""for code in codes:
s = s.replace(code[0], code[1])return s
htmlCodes = [
['&', '&'],
['<', '<'],
['>', '>'],
['"', '"'],
]
htmlCodesReversed = htmlCodes[:]
htmlCodesReversed.reverse()
def htmlDecode(s, codes=htmlCodesReversed):
""" Returns the ASCII decoded version of the given HTML string. This does
NOT remove normal HTML tags like <p>. It is the inverse of htmlEncode()."""
for code in codes:
s = s.replace(code[1], code[0])
return s
not sure why they reverse the list,
I think it has to do with the way they encode, so with you it may not need to be reversed.
Also if I were you I would change htmlCodes to be a list of tuples rather than a list of lists…
this is going in my library though :)
i noticed your title asked for encode too, so here is Cheetah’s encode function.
def htmlEncode(s, codes=htmlCodes):
""" Returns the HTML encoded version of the given string. This is useful to
display a plain ASCII text string on a web page."""
for code in codes:
s = s.replace(code[0], code[1])
return s
回答 11
您也可以使用django.utils.html.escape
from django.utils.html import escape
something_nice = escape(request.POST['something_naughty'])
def decodeHtmlText(html):"""
Given a string of HTML that would parse to a single text node,
return the text value of that node.
"""# Fast path for common case.if html.find("&")<0:return html
return re.sub('&(?:#(?:x([0-9A-Fa-f]+)|([0-9]+))|([a-zA-Z0-9]+));',
_decode_html_entity,
html)def _decode_html_entity(match):"""
Regex replacer that expects hex digits in group 1, or
decimal digits in group 2, or a named entity in group 3.
"""
hex_digits = match.group(1)# ' ' -> unichr(10)if hex_digits:return unichr(int(hex_digits,16))
decimal_digits = match.group(2)# '' -> unichr(0x10)if decimal_digits:return unichr(int(decimal_digits,10))
name = match.group(3)# name is 'lt' when '<' was matched.if name:
decoding =(htmlentitydefs.name2codepoint.get(name)# Treat > like >.# This is wrong for ≫ and ≪ which HTML5 adopted from MathML.# If htmlentitydefs included mappings for those entities,# then this code will magically work.or htmlentitydefs.name2codepoint.get(name.lower()))if decoding isnotNone:return unichr(decoding)return match.group(0)# Treat "&noSuchEntity;" as "&noSuchEntity;"
Below is a python function that uses module htmlentitydefs. It is not perfect. The version of htmlentitydefs that I have is incomplete and it assumes that all entities decode to one codepoint which is wrong for entities like ≂̸:
def decodeHtmlText(html):
"""
Given a string of HTML that would parse to a single text node,
return the text value of that node.
"""
# Fast path for common case.
if html.find("&") < 0: return html
return re.sub(
'&(?:#(?:x([0-9A-Fa-f]+)|([0-9]+))|([a-zA-Z0-9]+));',
_decode_html_entity,
html)
def _decode_html_entity(match):
"""
Regex replacer that expects hex digits in group 1, or
decimal digits in group 2, or a named entity in group 3.
"""
hex_digits = match.group(1) # ' ' -> unichr(10)
if hex_digits: return unichr(int(hex_digits, 16))
decimal_digits = match.group(2) # '' -> unichr(0x10)
if decimal_digits: return unichr(int(decimal_digits, 10))
name = match.group(3) # name is 'lt' when '<' was matched.
if name:
decoding = (htmlentitydefs.name2codepoint.get(name)
# Treat > like >.
# This is wrong for ≫ and ≪ which HTML5 adopted from MathML.
# If htmlentitydefs included mappings for those entities,
# then this code will magically work.
or htmlentitydefs.name2codepoint.get(name.lower()))
if decoding is not None: return unichr(decoding)
return match.group(0) # Treat "&noSuchEntity;" as "&noSuchEntity;"
When I attempted to connect to a local MySQL server during my test suite, it
fails with the error:
OperationalError: (2002, "Can't connect to local MySQL server through socket '/tmp/mysql.sock' (2)")
However, I’m able to at all times, connect to MySQL by running the command line
mysql program. A ps aux | grep mysql shows the server is running, and
stat /tmp/mysql.sock confirm that the socket exists. Further, if I open a
debugger in except clause of that exception, I’m able to reliably connect
with the exact same parameters.
This issue reproduces fairly reliably, however it doesn’t appear to be 100%,
because every once in a blue moon, my test suite does in fact run without
hitting this error. When I attempted to run with sudo dtruss it did not reproduce.
All the client code is in Python, though I can’t figure how that’d be relevant.
Switching to use host 127.0.0.1 produces the error:
DatabaseError: Can't connect to MySQL server on '127.0.0.1' (61)
The relevant section of the MySQL manual is here. I’d start by going through the debugging steps listed there.
Also, remember that localhost and 127.0.0.1 are not the same thing in this context:
If host is set to localhost, then a socket or pipe is used.
If host is set to 127.0.0.1, then the client is forced to use TCP/IP.
So, for example, you can check if your database is listening for TCP connections vi netstat -nlp. It seems likely that it IS listening for TCP connections because you say that mysql -h 127.0.0.1 works just fine. To check if you can connect to your database via sockets, use mysql -h localhost.
If none of this helps, then you probably need to post more details about your MySQL config, exactly how you’re instantiating the connection, etc.
回答 2
对我来说，问题是我没有运行mysql服务器。首先运行服务器，然后执行mysql。
$ mysql.server start
$ mysql -h localhost -u root -p
I’ve seen this happen at my shop when my devs have a stack manager like MAMP installed that comes preconfigured with MySQL installed in a non standard place.
at your terminal run
mysql_config --socket
that will give you your path to the sock file. take that path and use it in your DATABASES HOST paramater.
To those who upgraded from 5.7 to 8.0 via homebrew, this error is likely caused by the upgrade not being complete. In my case, mysql.server start got me the following error:
ERROR! The server quit without updating PID file
I then checked the log file via cat /usr/local/var/mysql/YOURS.err | tail -n 50, and found the following:
InnoDB: Upgrade after a crash is not supported.
If you are on the same boat, first install mysql@5.7 via homebrew, stop the server, and then start the 8.0 system again.
I think i saw this same behavior some time ago, but can’t remember the details.
In our case, the problem was the moment the testrunner initialises database connections relative to first database interaction required, for instance, by import of a module in settings.py or some __init__.py.
I’ll try to digg up some more info, but this might already ring a bell for your case.
Look into the possibility of not being able to access the /tmp/mysql.sock file. When I setup MySQL databases, I normally let the socket file site in /var/lib/mysql. If you login to mysql as root@localhost, your OS session needs access to the /tmp folder. Make sure /tmp has the correct access rights in the OS. Also, make sure the sudo user can always read file in /tmp.
CONJECTURE #2
Accessing mysql via 127.0.0.1 can cause some confusion if you are not paying attention. How?
From the command line, if you connect to MySQL with 127.0.0.1, you may need to specify the TCP/IP protocol.
mysql -uroot -p -h127.0.0.1 --protocol=tcp
or try the DNS name
mysql -uroot -p -hDNSNAME
This will bypass logging in as root@localhost, but make sure you have root@'127.0.0.1' defined.
Next time you connect to MySQL, run this:
SELECT USER(),CURRENT_USER();
What does this give you?
USER() reports how you attempted to authenticate in MySQL
CURRENT_USER() reports how you were allowed to authenticate in MySQL
If these functions return with the same values, then you are connecting and authenticating as expected. If the values are different, you may need to create the corresponding user root@127.0.0.1.
Check that your mysql has not reached maximum connections, or is not in some sort of booting loop as happens quite often if the settings are incorrect in my.cnf.
Use ps aux | grep mysql to check if the PID is changing.
Looked around online too long not to contribute. After trying to type in the mysql prompt from the command line, I was continuing to receive this message:
ERROR 2002 (HY000): Can’t connect to local MySQL server through socket ‘/tmp/mysql.sock’ (2)
This was due to the fact that my local mysql server was no longer running. In order to restart the server, I navigated to
shell> cd /user/local/bin
where my mysql.server was located. From here, simply type:
shell> mysql.server start
This will relaunch the local mysql server.
From there you can reset the root password if need be..
mysql> UPDATE mysql.user SET Password=PASSWORD('MyNewPass')
-> WHERE User='root';
mysql> FLUSH PRIVILEGES;
The socket is located in /tmp. On Unix system, due to modes & ownerships on /tmp, this could cause some problem. But, as long as you tell us that you CAN use your mysql connexion normally, I guess it is not a problem on your system. A primal check should be to relocate mysql.sock in a more neutral directory.
The fact that the problem occurs “randomly” (or not every time) let me think that it could be a server problem.
Is your /tmp located on a standard disk, or on an exotic mount (like in the RAM) ?
Is your /tmp empty ?
Does iotopshow you something wrong when you encounter the problem ?
According to this other page a socket file is used even if you specify localhost.
A Unix socket file is used if you do not specify a host name or if you
specify the special host name localhost.
It also shows how to check on your server by running these commands:
If a mysqld process is running, you can check it by trying the
following commands. The port number or Unix socket file name might be
different in your setup. host_ip represents the IP address of the
machine where the server is running.
shell> mysqladmin version
shell> mysqladmin variables
shell> mysqladmin -h `hostname` version variables
shell> mysqladmin -h `hostname` --port=3306 version
shell> mysqladmin -h host_ip version
shell> mysqladmin --protocol=SOCKET --socket=/tmp/mysql.sock version
最后，我发现当我使用服务mysqld start启动mysqld服务时，出现了问题（selinux权限问题），当我修复了selinux问题，并使用“ service mysqld start”启动mysqld时，httpd连接问题消失了。但是，当我使用mysqld_safe＆启动mysqld时，可以使用mysqld。（mysql客户端可以正常工作）。但是与httpd连接时仍然存在问题。
For me, I’m sure mysqld is started, and command line mysql can work properly. But the httpd server show the issue(can’t connect to mysql through socket).
I started the service with mysqld_safe&.
finally, I found when I start the mysqld service with service mysqld start, there are issues(selinux permission issue), and when I fix the selinux issue, and start the mysqld with “service mysqld start”, the httpd connection issue disappear. But when I start the mysqld with mysqld_safe&, mysqld can be worked. (mysql client can work properly). But there are still issue when connect with httpd.
回答 23
如果与套接字相关，请阅读此文件
/etc/mysql/my.cnf
并查看什么是标准插座位置。就像这样的一行：
socket =/var/run/mysqld/mysqld.sock
现在为您的shell创建一个别名，例如：
alias mysql="mysql --socket=/var/run/mysqld/mysqld.sock"
Looking to do a very small, quick ‘n dirty side project. I like the fact that the Google App Engine is running on Python with Django built right in – gives me an excuse to try that platform… but my question is this:
Has anyone made use of the app engine for anything other than a toy problem? I see some good example apps out there, so I would assume this is good enough for the real deal, but wanted to get some feedback.
My purpose was to see the capabilities of app engine, so here are the main points:
it doesn’t come by default with Django, it has its own web framework which is pythonic has URL dispatcher like Django and it uses Django templates
So if you have Django exp. you will find it easy to use
You can not execute any long running process on server, what you do is reply to request and which should be quick otherwise appengine will kill it
So if your app needs lots of backend processing appengine is not the best way
otherwise you will have to do processing on a server of your own
My quakewatch app has a subscription feature, it means I had to email latest quakes as they happend, but I can not run a background process in app engine to monitor new quakes
solution here is to use a third part service like pingablity.com which can connect to one of your page and which executes the subscription emailer
but here also you will have to take care that you don’t spend much time here
or break task into several pieces
It provides Django like modeling capabilities but backend is totally different but for a new project it should not matter.
But overall I think it is excellent for creating apps which do not need lot of background processing.
Edit:
Now task queues can be used for running batch processing or scheduled tasks
Edit:
after working/creating a real application on GAE for a year, now my opnion is that unless you are making a application which needs to scale to million and million of users, don’t use GAE. Maintaining and doing trivial tasks in GAE is a headache due to distributed nature, to avoid deadline exceeded errors, count entities or do complex queries requires complex code, so small complex application should stick to LAMP.
Edit:
Models should be specially designed considering all the transactions you wish to have in future, because entities only in same entity group can be used in a transaction and it makes the process of updating two different groups a nightmare e.g. transfer money from user1 to user2 in transaction is impossible unless they are in same entity group, but making them same entity group may not be best for frequent update purposes….
read this http://blog.notdot.net/2009/9/Distributed-Transactions-on-App-Engine
I am using GAE to host several high-traffic applications. Like on the order of 50-100 req/sec. It is great, I can’t recommend it enough.
My previous experience with web development was with Ruby (Rails/Merb). Learning Python was easy. I didn’t mess with Django or Pylons or any other framework, just started from the GAE examples and built what I needed out of the basic webapp libraries that are provided.
If you’re used to the flexibility of SQL the datastore can take some getting used to. Nothing too traumatic! The biggest adjustment is moving away from JOINs. You have to shed the idea that normalizing is crucial.
One of the compelling reasons I have come across for using Google App Engine is its integration with Google Apps for your domain. Essentially it allows you to create custom, managed web applications that are restricted to the (controlled) logins of your domain.
Most of my experience with this code was building a simple time/task tracking application. The template engine was simple and yet made a multi-page application very approachable. The login/user awareness api is similarly useful. I was able to make a public page/private page paradigm without too much issue. (a user would log in to see the private pages. An anonymous user was only shown the public page.)
I was just getting into the datastore portion of the project when I got pulled away for “real work”.
I was able to accomplish a lot (it still is not done yet) in a very little amount of time. Since I had never used Python before, this was particularly pleasant (both because it was a new language for me, and also because the development was still fast despite the new language). I ran into very little that led me to believe that I wouldn’t be able to accomplish my task. Instead I have a fairly positive impression of the functionality and features.
That is my experience with it. Perhaps it doesn’t represent more than an unfinished toy project, but it does represent an informed trial of the platform, and I hope that helps.
The “App Engine running Django” idea is a bit misleading. App Engine replaces the entire Django model layer so be prepared to spend some time getting acclimated with App Engine’s datastore which requires a different way of modeling and thinking about data.
It’s a bit more than a toy project but not overly complex either. I still depend on a few issues to be addressed by Google, but overall developing the website was an enjoyable experience.
If you don’t want to deal with hosting issues, server administration, etc, I can definitely recommend it. Especially if you already know Python and Django.
I think App Engine is pretty cool for small projects at this point. There’s a lot to be said for never having to worry about hosting. The API also pushes you in the direction of building scalable apps, which is good practice.
app-engine-patch is a good layer between Django and App Engine, enabling the use of the auth app and more.
Google have promised an SLA and pricing model by the end of 2008.
Requests must complete in 10 seconds, sub-requests to web services required to complete in 5 seconds. This forces you to design a fast, lightweight application, off-loading serious processing to other platforms (e.g. a hosted service or an EC2 instance).
More languages are coming soon! Google won’t say which though :-). My money’s on Java next.
This question has been fully answered. Which is good.
But one thing perhaps is worth mentioning.
The google app engine has a plugin for the eclipse ide which is a joy to work with.
If you already do your development with eclipse you are going to be so happy about that.
To deploy on the google app engine’s web site all I need to do is click one little button – with the airplane logo – super.
Take a look the the sql game, it is very stable and actually pushed traffic limits at one point so that it was getting throttled by Google. I have seen nothing but good news about App Engine, other than hosting you app on servers someone else controls completely.
I used GAE to build a simple application which accepts some parameters, formats and send email. It was extremely simple and fast. I also made some performance benchmarks on the GAE datastore and memcache services (http://dbaspects.blogspot.com/2010/01/memcache-vs-datastore-on-google-app.html ). It is not that fast. My opinion is that GAE is serious platform which enforce certain methodology. I think it will evolve to the truly scalable platform, where bad practices simply not allowed.
I used GAE for my flash gaming site, Bearded Games. GAE is a great platform. I used Django templates which are so much easier than the old days of PHP. It comes with a great admin panel, and gives you really good logs. The datastore is different than a database like MySQL, but it’s much easier to work with. Building the site was easy and straightforward and they have lots of helpful advice on the site.
I used GAE and Django to build a Facebook application. I used http://code.google.com/p/app-engine-patch as my starting point as it has Django 1.1 support. I didn’t try to use any of the manage.py commands because I assumed they wouldn’t work, but I didn’t even look into it. The application had three models and also used pyfacebook, but that was the extent of the complexity. I’m in the process of building a much more complicated application which I’m starting to blog about on http://brianyamabe.com.
My Django unit tests take a long time to run, so I’m looking for ways to speed that up. I’m considering installing an SSD, but I know that has its downsides too. Of course, there are things I could do with my code, but I’m looking for a structural fix. Even running a single test is slow since the database needs to be rebuilt / south migrated every time. So here’s my idea…
Since I know the test database will always be quite small, why can’t I just configure the system to always keep the entire test database in RAM? Never touch the disk at all. How do I configure this in Django? I’d prefer to keep using MySQL since that’s what I use in production, but if SQLite 3 or something else makes this easy, I’d go that way.
Does SQLite or MySQL have an option to run entirely in memory? It should be possible to configure a RAM disk and then configure the test database to store its data there, but I’m not sure how to tell Django / MySQL to use a different data directory for a certain database, especially since it keeps getting erased and recreated each run. (I’m on a Mac FWIW.)
I can’t answer your main question, but there are a couple of things that you can do to speed things up.
Firstly, make sure that your MySQL database is set up to use InnoDB. Then it can use transactions to rollback the state of the db before each test, which in my experience has led to a massive speed-up. You can pass a database init command in your settings.py (Django 1.2 syntax):
Secondly, you don’t need to run the South migrations each time. Set SOUTH_TESTS_MIGRATE = False in your settings.py and the database will be created with plain syncdb, which will be much quicker than running through all the historic migrations.
回答 4
您可以进行两次调整：
使用事务表：在每个TestCase之后，将使用数据库回滚来设置初始固定装置状态。
将您的数据库数据目录放在ramdisk上：就数据库创建而言，您将获得很多收益，并且运行测试会更快。
我正在使用这两种技巧，我很高兴。
如何在Ubuntu上为MySQL设置它：
$ sudo service mysql stop
$ sudo cp -pRL /var/lib/mysql /dev/shm/mysql
$ vim /etc/mysql/my.cnf
# datadir = /dev/shm/mysql
$ sudo service mysql start
Another approach: have another instance of MySQL running in a tempfs that uses a RAM Disk. Instructions in this blog post:Speeding up MySQL for testing in Django.
Advantages:
You use the exactly same database that your production server uses
no need to change your default mysql configuration
if len(sys.argv)>1and sys.argv[1]=="test":
os.environ.setdefault("DJANGO_SETTINGS_MODULE","mysite.test_settings")else:
os.environ.setdefault("DJANGO_SETTINGS_MODULE","mysite.settings")
First – I know that this question has been asked and answered before (see Django: Overriding AND extending an app template) but as the answer says it isn’t directly applicable if you’re using the app_directories template loader (which is most of the time).
My current workaround is to make copies and extend from them instead of extending directly from the admin templates. This works great but it’s really confusing and adds extra work when the admin templates change.
It could think of some custom extend-tag for the templates but I don’t want to reinvent the wheel if there already exists a solution.
On a side note: Does anybody know if this problem will be addressed by Django itself?
I had the same issue about a year and a half ago and I found a nice template loader on djangosnippets.org that makes this easy. It allows you to extend a template in a specific app, giving you the ability to create your own admin/index.html that extends the admin/index.html template from the admin app. Like this:
As for Django 1.8 being the current release, there is no need to symlink, copy the admin/templates to your project folder, or install middlewares as suggested by the answers above. Here is what to do:
your_project
|-- your_project/
|-- myapp/
|-- templates/
|-- admin/
|-- myapp/
|-- change_form.html <- do not misspell this
Note: The location of this file is not important. You can put it inside your app and it will still work. As long as its location can be discovered by django. What’s more important is the name of the HTML file has to be the same as the original HTML file name provided by django.
Identify the name and block you want to override. This is done by looking into django’s admin/templates directory. I am using virtualenv, so for me, the path is here:
In this example, I want to modify the add new user form. The template responsiblve for this view is change_form.html. Open up the change_form.html and find the {% block %} that you want to extend.
In your change_form.html, write somethings like this:
{% extends "admin/change_form.html" %}
{% block field_sets %}
{# your modification here #}
{% endblock %}
For those templates that cannot be overridden in this way, you may
still override them for your entire project. Just place the new
version in your templates/admin directory. This is particularly useful
to create custom 404 and 500 pages
I had to overwrite the login.html of the admin and therefore had to put the overwritten template in this folder structure:
your_project
|-- your_project/
|-- myapp/
|-- templates/
|-- admin/
|-- login.html <- do not misspell this
(without the myapp subfolder in the admin)
I do not have enough repution for commenting on Cheng’s post this is why I had to write this as new answer.
The best way to do it is to put the Django admin templates inside your project. So your templates would be in templates/admin while the stock Django admin templates would be in say template/django_admin. Then, you can do something like the following:
templates/admin/change_form.html
{% extends 'django_admin/change_form.html' %}
Your stuff here
If you’re worried about keeping the stock templates up to date, you can include them with svn externals or similar.
from django.contrib.admin importAdminSiteclassCustomAdminSite(AdminSite):# set values for `site_header`, `site_title`, `index_title` etc.
site_header ='Custom Admin Site'...# extend / override admin views, such as `index()`def index(self, request, extra_context=None):
extra_context = extra_context or{}# do whatever you want to do and save the values in `extra_context`
extra_context['world']='Earth'return super(CustomAdminSite, self).index(request, extra_context)
custom_admin_site =CustomAdminSite()
I couldn’t find a single answer or a section in the official Django docs that had all the information I needed to override/extend the default admin templates, so I’m writing this answer as a complete guide, hoping that it would be helpful for others in the future.
In mysite/admin.py, create a sub-class of AdminSite:
from django.contrib.admin import AdminSite
class CustomAdminSite(AdminSite):
# set values for `site_header`, `site_title`, `index_title` etc.
site_header = 'Custom Admin Site'
...
# extend / override admin views, such as `index()`
def index(self, request, extra_context=None):
extra_context = extra_context or {}
# do whatever you want to do and save the values in `extra_context`
extra_context['world'] = 'Earth'
return super(CustomAdminSite, self).index(request, extra_context)
custom_admin_site = CustomAdminSite()
Make sure to import custom_admin_site in the admin.py of your apps and register your models on it to display them on your customized admin site (if you want to).
In mysite/apps.py, create a sub-class of AdminConfig and set default_site to admin.CustomAdminSite from the previous step:
from django.contrib.admin.apps import AdminConfig
class CustomAdminConfig(AdminConfig):
default_site = 'admin.CustomAdminSite'
In mysite/settings.py, replace django.admin.site in INSTALLED_APPS with apps.CustomAdminConfig (your custom admin app config from the previous step).
In mysite/urls.py, replace admin.site.urls from the admin URL to custom_admin_site.urls
from .admin import custom_admin_site
urlpatterns = [
...
path('admin/', custom_admin_site.urls),
# for Django 1.x versions: url(r'^admin/', include(custom_admin_site.urls)),
...
]
Create the template you want to modify in your templates directory, maintaining the default Django admin templates directory structure as specified in the docs. For example, if you were modifying admin/index.html, create the file templates/admin/index.html.
All of the existing templates can be modified this way, and their names and structures can be found in Django’s source code.
Now you can either override the template by writing it from scratch or extend it and then override/extend specific blocks.
For example, if you wanted to keep everything as-is but wanted to override the content block (which on the index page lists the apps and their models that you registered), add the following to templates/admin/index.html:
and as you can see it depends on python version and the folder where the Django installed. So in future or on a production server you might need to change the path.
This site had a simple solution that worked with my Django 1.7 configuration.
FIRST: Make a symlink named admin_src in your project’s template/ directory to your installed Django templates. For me on Dreamhost using a virtualenv, my “source” Django admin templates were in:
You can use django-overextends, which provides circular template inheritance for Django.
It comes from the Mezzanine CMS, from where Stephen extracted it into a standalone Django extension.
More infos you find in “Overriding vs Extending Templates” (http:/mezzanine.jupo.org/docs/content-architecture.html#overriding-vs-extending-templates) inside the Mezzanine docs.
For deeper insides look at Stephens Blog “Circular Template Inheritance for Django” (http:/blog.jupo.org/2012/05/17/circular-template-inheritance-for-django).
And in Google Groups the discussion (https:/groups.google.com/forum/#!topic/mezzanine-users/sUydcf_IZkQ) which started the development of this feature.
Note:
I don’t have the reputation to add more than 2 links. But I think the links provide interesting background information. So I just left out a slash after “http(s):”. Maybe someone with better reputation can repair the links and remove this note.
Consider simple Django models Event and Participant:
class Event(models.Model):
title = models.CharField(max_length=100)
class Participant(models.Model):
event = models.ForeignKey(Event, db_index=True)
is_paid = models.BooleanField(default=False, db_index=True)
It’s easy to annotate events query with total number of participants:
How to annotate with count of participants filtered by is_paid=True?
I need to query all events regardless of number of participants, e.g. I don’t need to filter by annotated result. If there are 0 participants, that’s ok, I just need 0 in annotated value.
The example from documentation doesn’t work here, because it excludes objects from query instead of annotating them with 0.
Conditional aggregation in Django 2.0 allows you to further reduce the amount of faff this has been in the past. This will also use Postgres’ filter logic, which is somewhat faster than a sum-case (I’ve seen numbers like 20-30% bandied around).
Anyway, in your case, we’re looking at something as simple as:
from django.db.models import Q, Count
events = Event.objects.annotate(
paid_participants=Count('participants', filter=Q(participants__is_paid=True))
)
There’s a separate section in the docs about filtering on annotations. It’s the same stuff as conditional aggregation but more like my example above. Either which way, this is a lot healthier than the gnarly subqueries I was doing before.
part1l =[Participant.objects.create(event=event1, is_paid=((_%2)==0))\for _ in range(10)]
part2l =[Participant.objects.create(event=event2, is_paid=((_%2)==0))\for _ in range(50)]
将所有Participants按其event字段分组：
Participant.objects.values('event')><QuerySet[{'event':1},{'event':1},{'event':1},{'event':1},{'event':1},{'event':1},{'event':1},{'event':1},{'event':1},{'event':1},{'event':2},{'event':2},{'event':2},{'event':2},{'event':2},{'event':2},{'event':2},{'event':2},{'event':2},{'event':2},'...(remaining elements truncated)...']>
part1l = [Participant.objects.create(event=event1, is_paid=((_%2) == 0))\
for _ in range(10)]
part2l = [Participant.objects.create(event=event2, is_paid=((_%2) == 0))\
for _ in range(50)]
What .values and .distinct are doing here is that they are creating two buckets of Participants grouped by their element event. Note that those buckets contain Participant.
You can then annotate those buckets as they contain the set of original Participant. Here we want to count the number of Participant, this is simply done by counting the ids of the elements in those buckets (since those are Participant):
Finally you want only Participant with a is_paid being True, you may just add a filter in front of the previous expression, and this yield the expression shown above:
Once you add http_method_names, you will not be able to do put and patch anymore.
If you want put but don’t want patch, you can keep http_method_names = ['get', 'post', 'head', 'put']
Internally, DRF Views extend from Django CBV. Django CBV has an attribute called http_method_names. So you can use http_method_names with DRF views too.
from rest_framework import viewsets, statusfrom rest_framework.response importResponseclassNameWhateverYouWantViewSet(viewsets.ModelViewSet):def create(self, request):
response ={'message':'Create function is not offered in this path.'}returnResponse(response, status=status.HTTP_403_FORBIDDEN)def update(self, request, pk=None):
response ={'message':'Update function is not offered in this path.'}returnResponse(response, status=status.HTTP_403_FORBIDDEN)def partial_update(self, request, pk=None):
response ={'message':'Update function is not offered in this path.'}returnResponse(response, status=status.HTTP_403_FORBIDDEN)def destroy(self, request, pk=None):
response ={'message':'Delete function is not offered in this path.'}returnResponse(response, status=status.HTTP_403_FORBIDDEN)
Although it’s been a while for this post, I suddenly found out that actually there is a way to disable those functions, you can edit it in the views.py directly.
from rest_framework import viewsets, status
from rest_framework.response import Response
class NameThisClassWhateverYouWantViewSet(viewsets.ModelViewSet):
def create(self, request):
response = {'message': 'Create function is not offered in this path.'}
return Response(response, status=status.HTTP_403_FORBIDDEN)
def update(self, request, pk=None):
response = {'message': 'Update function is not offered in this path.'}
return Response(response, status=status.HTTP_403_FORBIDDEN)
def partial_update(self, request, pk=None):
response = {'message': 'Update function is not offered in this path.'}
return Response(response, status=status.HTTP_403_FORBIDDEN)
def destroy(self, request, pk=None):
response = {'message': 'Delete function is not offered in this path.'}
return Response(response, status=status.HTTP_403_FORBIDDEN)
回答 3
如果尝试从DRF视图集中禁用PUT方法，则可以创建一个自定义路由器：
from rest_framework.routers importDefaultRouterclassNoPutRouter(DefaultRouter):"""
Router class that disables the PUT method.
"""def get_method_map(self, viewset, method_map):
bound_methods = super().get_method_map(viewset, method_map)if'put'in bound_methods.keys():del bound_methods['put']return bound_methods
If you are trying to disable the PUT method from a DRF viewset, you can create a custom router:
from rest_framework.routers import DefaultRouter
class NoPutRouter(DefaultRouter):
"""
Router class that disables the PUT method.
"""
def get_method_map(self, viewset, method_map):
bound_methods = super().get_method_map(viewset, method_map)
if 'put' in bound_methods.keys():
del bound_methods['put']
return bound_methods
By disabling the method at the router, your api schema documentation will be correct.
回答 4
如何在DRF中为ViewSet禁用“删除”方法
classYourViewSet(viewsets.ModelViewSet):def _allowed_methods(self):return[m for m in super(YourViewSet, self)._allowed_methods()if m notin['DELETE']]
class YourViewSet(viewsets.ModelViewSet):
def _allowed_methods(self):
return [m for m in super(YourViewSet, self)._allowed_methods() if m not in ['DELETE']]
P.S. This is more reliable than explicitly specifying all the necessary methods, so there is less chance of forgetting some of important methods OPTIONS, HEAD, etc
P.P.S.
by default DRF has http_method_names = ['get', 'post', 'put', 'patch', 'delete', 'head', 'options', 'trace']
path('sample/',SampleViewSet.as_view({'get':'list','post':'create'})),
path('sample/<pk>/',SampleViewSet.as_view({# for get sample by id.'get':'retrieve'}))
如您所见，上述路由设置中没有no delete和putrequest，因此例如，如果您将put请求发送到url，它将以405响应您Method Not Allowed：
In Django Rest Framework 3.x.x you can simply enable every each method you want to be enabled for ModelViewSet, by passing a dictionary to as_view method. In this dictionary, the key must contain request type (GET, POST, DELETE, etc) and the value must contain corresponding method name (list, retrieve, update, etc). For example let say you want Sample model to be created or read but you don’t want it to be modified. So it means you want list, retrieve and create method to be enable (and you want others to be disabled.)
All you need to do is to add paths to urlpatterns like these:
path('sample/', SampleViewSet.as_view({
'get': 'list',
'post': 'create'
})),
path('sample/<pk>/', SampleViewSet.as_view({ # for get sample by id.
'get': 'retrieve'
}))
As you can see there’s no delete and put request in above routing settings, so for example if you send a put request to the url, it response you with 405 Method Not Allowed:
How can I convert a Django QuerySet into a list of dicts? I haven’t found an answer to this so I’m wondering if I’m missing some sort of common helper function that everyone uses.
Note: the result is a QuerySet which mostly behaves like a list, but isn’t actually an instance of list. Use list(Blog.objects.values(…)) if you really need an instance of list.
result =Blog.objects.values()# return ValuesQuerySet object
list_result =[entry for entry in result]# converts ValuesQuerySet into Python listreturn list_result
The .values() method will return you a result of type ValuesQuerySet which is typically what you need in most cases.
But if you wish, you could turn ValuesQuerySet into a native Python list using Python list comprehension as illustrated in the example below.
result = Blog.objects.values() # return ValuesQuerySet object
list_result = [entry for entry in result] # converts ValuesQuerySet into Python list
return list_result
I find the above helps if you are writing unit tests and need to assert that the expected return value of a function matches the actual return value, in which case both expected_result and actual_result must be of the same type (e.g. dictionary).
You do not exactly define what the dictionaries should look like, but most likely you are referring to QuerySet.values(). From the official django documentation:
Returns a ValuesQuerySet — a QuerySet subclass that returns
dictionaries when used as an iterable, rather than model-instance
objects.
Each of those dictionaries represents an object, with the keys
corresponding to the attribute names of model objects.
You can use the values() method on the dict you got from the Django model field you make the queries on and then you can easily access each field by a index value.
Call it like this –
myList = dictOfSomeData.values()
itemNumberThree = myList[2] #If there's a value in that index off course...
def queryset_to_dict(qs,fields=None, exclude=None):
my_array=[]for x in qs:
my_array.append(model_to_dict(x,fields=fields,exclude=exclude))return my_array
You could define a function using model_to_dict as follows:
def queryset_to_dict(qs,fields=None, exclude=None):
my_array=[]
for x in qs:
my_array.append(model_to_dict(x,fields=fields,exclude=exclude))
return my_array
I realize this is an old post, but I ran into the same problem and don’t see the correct answer so I will give it a try
Python Error:
_csv.Error: new-line character seen in unquoted field
Caused by trying to read Macintosh (pre OS X formatted) CSV files. These are text files that use CR for end of line. If using MS Office make sure you select either plain CSV format or CSV (MS-DOS). Do not use CSV (Macintosh) as save-as type.
My preferred EOL version would be LF (Unix/Linux/Apple), but I don’t think MS Office provides the option to save in this format.
This is an error that I faced. I had saved .csv file in MAC OSX.
While saving, save it as “Windows Comma Separated Values (.csv)” which resolved the issue.
回答 6
这在OSX上对我有用。
# allow variable to opened as filesfrom io importStringIO# library to map other strange (accented) characters back into UTF-8from unidecode import unidecode# cleanse input file with Windows formating to plain UTF-8 stringwith open(filename,'rb')as fID:
uncleansedBytes = fID.read()# decode the file using the correct encoding scheme# (probably this old windows one)
uncleansedText = uncleansedBytes.decode('Windows-1252')# replace carriage-returns with new-lines
cleansedText = uncleansedText.replace('\r','\n')# map any other non UTF-8 characters into UTF-8
asciiText = unidecode(cleansedText)# read each line of the csv file and store as an array of dicts, # use first line as field names for each dict.
reader = csv.DictReader(StringIO(cleansedText))for line_entry in reader:# do something with your read data
# allow variable to opened as files
from io import StringIO
# library to map other strange (accented) characters back into UTF-8
from unidecode import unidecode
# cleanse input file with Windows formating to plain UTF-8 string
with open(filename, 'rb') as fID:
uncleansedBytes = fID.read()
# decode the file using the correct encoding scheme
# (probably this old windows one)
uncleansedText = uncleansedBytes.decode('Windows-1252')
# replace carriage-returns with new-lines
cleansedText = uncleansedText.replace('\r', '\n')
# map any other non UTF-8 characters into UTF-8
asciiText = unidecode(cleansedText)
# read each line of the csv file and store as an array of dicts,
# use first line as field names for each dict.
reader = csv.DictReader(StringIO(cleansedText))
for line_entry in reader:
# do something with your read data
with urllib.request.urlopen(q)as response:
raw_data = response.read()
encoding = response.info().get_content_charset('utf8')
data = raw_data.decode(encoding)if'\r\n'notin data:# proably a windows delimited thing...try to update it
data = data.replace('\r','\r\n')
I know this has been answered for quite some time but not solve my problem. I am using DictReader and StringIO for my csv reading due to some other complications. I was able to solve problem more simply by replacing delimiters explicitly:
with urllib.request.urlopen(q) as response:
raw_data = response.read()
encoding = response.info().get_content_charset('utf8')
data = raw_data.decode(encoding)
if '\r\n' not in data:
# proably a windows delimited thing...try to update it
data = data.replace('\r', '\r\n')
Might not be reasonable for enormous CSV files, but worked well for my use case.
Alternative and fast solution : I faced the same error. I reopened the “wierd” csv file in GNUMERIC on my lubuntu machine and exported the file as csv file. This corrected the issue.
Have just hit this problem. You can also use db.start_transaction() and db.commit_transaction() in the schema migration to separate data changes from schema changes. Probably not so clean as to have a separate data migration but in my case I would need schema, data, and then another schema migration so I decided to do it all at once.
回答 3
在操作中，我将SET约束：
operations =[
migrations.RunSQL('SET CONSTRAINTS ALL IMMEDIATE;'),
migrations.RunPython(migration_func),
migrations.RunSQL('SET CONSTRAINTS ALL DEFERRED;'),]
You are altering the column schema. That footer column can no longer contain a blank value. There are most likely blank values already stored in the DB for that column. Django is going to update those blank rows in your DB from blank to the now default value with the migrate command. Django tries to update the rows where footer column has a blank value and change the schema at the same time it seems (I’m not sure).
The problem is you can’t alter the same column schema you are trying to update the values for at the same time.
One solution would be to delete the migrations file updating the schema. Then, run a script to update all those values to your default value. Then re-run the migration to update the schema. This way, the update is already done. Django migration is only altering the schema.
