标签归档:ajax

传入的Django请求中的JSON数据在哪里?

问题:传入的Django请求中的JSON数据在哪里?

我正在尝试使用Django / Python处理传入的JSON / Ajax请求。

request.is_ajax()True在请求中,但是我不知道有效负载在哪里以及JSON数据。

request.POST.dir 包含以下内容:

['__class__', '__cmp__', '__contains__', '__copy__', '__deepcopy__', '__delattr__',
 '__delitem__', '__dict__', '__doc__', '__eq__', '__ge__', '__getattribute__',
'__getitem__', '__gt__', '__hash__', '__init__', '__iter__', '__le__', '__len__',
 '__lt__', '__module__', '__ne__', '__new__', '__reduce__', '__reduce_ex__', '__repr__', 
'__setattr__', '__setitem__', '__str__', '__weakref__', '_assert_mutable', '_encoding', 
'_get_encoding', '_mutable', '_set_encoding', 'appendlist', 'clear', 'copy', 'encoding', 
'fromkeys', 'get', 'getlist', 'has_key', 'items', 'iteritems', 'iterkeys', 'itervalues', 
'keys', 'lists', 'pop', 'popitem', 'setdefault', 'setlist', 'setlistdefault', 'update', 
'urlencode', 'values']

在请求发布键中显然没有键。

当我查看Firebug中的POST时,请求中发送了JSON数据。

I’m trying to process incoming JSON/Ajax requests with Django/Python.

request.is_ajax() is True on the request, but I have no idea where the payload is with the JSON data.

request.POST.dir contains this:

['__class__', '__cmp__', '__contains__', '__copy__', '__deepcopy__', '__delattr__',
 '__delitem__', '__dict__', '__doc__', '__eq__', '__ge__', '__getattribute__',
'__getitem__', '__gt__', '__hash__', '__init__', '__iter__', '__le__', '__len__',
 '__lt__', '__module__', '__ne__', '__new__', '__reduce__', '__reduce_ex__', '__repr__', 
'__setattr__', '__setitem__', '__str__', '__weakref__', '_assert_mutable', '_encoding', 
'_get_encoding', '_mutable', '_set_encoding', 'appendlist', 'clear', 'copy', 'encoding', 
'fromkeys', 'get', 'getlist', 'has_key', 'items', 'iteritems', 'iterkeys', 'itervalues', 
'keys', 'lists', 'pop', 'popitem', 'setdefault', 'setlist', 'setlistdefault', 'update', 
'urlencode', 'values']

There are apparently no keys in the request post keys.

When I look at the POST in Firebug, there is JSON data being sent up in the request.


回答 0

如果您要将JSON发布到Django,我想您想要request.bodyrequest.raw_post_data在Django <1.4上)。这将为您提供通过帖子发送的原始JSON数据。从那里您可以进一步处理它。

这是一个使用JavaScript,jQuery,jquery-json和Django 的示例。

JavaScript:

var myEvent = {id: calEvent.id, start: calEvent.start, end: calEvent.end,
               allDay: calEvent.allDay };
$.ajax({
    url: '/event/save-json/',
    type: 'POST',
    contentType: 'application/json; charset=utf-8',
    data: $.toJSON(myEvent),
    dataType: 'text',
    success: function(result) {
        alert(result.Result);
    }
});

Django:

def save_events_json(request):
    if request.is_ajax():
        if request.method == 'POST':
            print 'Raw Data: "%s"' % request.body   
    return HttpResponse("OK")

Django <1.4:

  def save_events_json(request):
    if request.is_ajax():
        if request.method == 'POST':
            print 'Raw Data: "%s"' % request.raw_post_data
    return HttpResponse("OK")

If you are posting JSON to Django, I think you want request.body (request.raw_post_data on Django < 1.4). This will give you the raw JSON data sent via the post. From there you can process it further.

Here is an example using JavaScript, jQuery, jquery-json and Django.

JavaScript:

var myEvent = {id: calEvent.id, start: calEvent.start, end: calEvent.end,
               allDay: calEvent.allDay };
$.ajax({
    url: '/event/save-json/',
    type: 'POST',
    contentType: 'application/json; charset=utf-8',
    data: $.toJSON(myEvent),
    dataType: 'text',
    success: function(result) {
        alert(result.Result);
    }
});

Django:

def save_events_json(request):
    if request.is_ajax():
        if request.method == 'POST':
            print 'Raw Data: "%s"' % request.body   
    return HttpResponse("OK")

Django < 1.4:

  def save_events_json(request):
    if request.is_ajax():
        if request.method == 'POST':
            print 'Raw Data: "%s"' % request.raw_post_data
    return HttpResponse("OK")

回答 1

我有同样的问题。我一直在发布复杂的JSON响应,但无法使用request.POST字典读取数据。

我的JSON POST数据为:

//JavaScript code:
//Requires json2.js and jQuery.
var response = {data:[{"a":1, "b":2},{"a":2, "b":2}]}
json_response = JSON.stringify(response); // proper serialization method, read 
                                          // http://ejohn.org/blog/ecmascript-5-strict-mode-json-and-more/
$.post('url',json_response);

在这种情况下,您需要使用金黄色葡萄球菌提供的方法。阅读request.body并使用json stdlib反序列化。

#Django code:
import json
def save_data(request):
  if request.method == 'POST':
    json_data = json.loads(request.body) # request.raw_post_data w/ Django < 1.4
    try:
      data = json_data['data']
    except KeyError:
      HttpResponseServerError("Malformed data!")
    HttpResponse("Got json data")

I had the same problem. I had been posting a complex JSON response, and I couldn’t read my data using the request.POST dictionary.

My JSON POST data was:

//JavaScript code:
//Requires json2.js and jQuery.
var response = {data:[{"a":1, "b":2},{"a":2, "b":2}]}
json_response = JSON.stringify(response); // proper serialization method, read 
                                          // http://ejohn.org/blog/ecmascript-5-strict-mode-json-and-more/
$.post('url',json_response);

In this case you need to use method provided by aurealus. Read the request.body and deserialize it with the json stdlib.

#Django code:
import json
def save_data(request):
  if request.method == 'POST':
    json_data = json.loads(request.body) # request.raw_post_data w/ Django < 1.4
    try:
      data = json_data['data']
    except KeyError:
      HttpResponseServerError("Malformed data!")
    HttpResponse("Got json data")

回答 2

方法一

客户:发送为 JSON

$.ajax({
    url: 'example.com/ajax/',
    type: 'POST',
    contentType: 'application/json; charset=utf-8',
    processData: false,
    data: JSON.stringify({'name':'John', 'age': 42}),
    ...
});

//Sent as a JSON object {'name':'John', 'age': 42}

服务器:

data = json.loads(request.body) # {'name':'John', 'age': 42}

方法二

客户端:发送为x-www-form-urlencoded
(注意:contentTypeprocessData已更改,JSON.stringify不需要)

$.ajax({
    url: 'example.com/ajax/',
    type: 'POST',    
    data: {'name':'John', 'age': 42},
    contentType: 'application/x-www-form-urlencoded; charset=utf-8',  //Default
    processData: true,       
});

//Sent as a query string name=John&age=42

服务器:

data = request.POST # will be <QueryDict: {u'name':u'John', u'age': 42}>

在1.5+版本中进行了更改:https : //docs.djangoproject.com/en/dev/releases/1.5/#non-form-data-in-http-requests

HTTP请求中的非格式数据
request.POST将不再包含通过HTTP请求发布的数据,该数据头中包含非特定于格式的内容类型。在以前的版本中,以multipart / form-data或application / x-www-form-urlencoded以外的内容类型发布的数据仍将最终在request.POST属性中表示。对于这些情况,希望访问原始POST数据的开发人员应改用request.body属性。

可能相关

Method 1

Client : Send as JSON

$.ajax({
    url: 'example.com/ajax/',
    type: 'POST',
    contentType: 'application/json; charset=utf-8',
    processData: false,
    data: JSON.stringify({'name':'John', 'age': 42}),
    ...
});

//Sent as a JSON object {'name':'John', 'age': 42}

Server :

data = json.loads(request.body) # {'name':'John', 'age': 42}

Method 2

Client : Send as x-www-form-urlencoded
(Note: contentType & processData have changed, JSON.stringify is not needed)

$.ajax({
    url: 'example.com/ajax/',
    type: 'POST',    
    data: {'name':'John', 'age': 42},
    contentType: 'application/x-www-form-urlencoded; charset=utf-8',  //Default
    processData: true,       
});

//Sent as a query string name=John&age=42

Server :

data = request.POST # will be <QueryDict: {u'name':u'John', u'age': 42}>

Changed in 1.5+ : https://docs.djangoproject.com/en/dev/releases/1.5/#non-form-data-in-http-requests

Non-form data in HTTP requests :
request.POST will no longer include data posted via HTTP requests with non form-specific content-types in the header. In prior versions, data posted with content-types other than multipart/form-data or application/x-www-form-urlencoded would still end up represented in the request.POST attribute. Developers wishing to access the raw POST data for these cases, should use the request.body attribute instead.

Probably related


回答 3

重要的是要记住,Python 3以不同的方式表示字符串-它们是字节数组。

使用Django 1.9和Python 2.7并在主体(而非标头)中发送JSON数据,您将使用类似以下内容:

mydata = json.loads(request.body)

但是对于Django 1.9和Python 3.4,您可以使用:

mydata = json.loads(request.body.decode("utf-8"))

我刚刚经历了这个学习过程,制作了我的第一个Py3 Django应用!

Its important to remember Python 3 has a different way to represent strings – they are byte arrays.

Using Django 1.9 and Python 2.7 and sending the JSON data in the main body (not a header) you would use something like:

mydata = json.loads(request.body)

But for Django 1.9 and Python 3.4 you would use:

mydata = json.loads(request.body.decode("utf-8"))

I just went through this learning curve making my first Py3 Django app!


回答 4

request.raw_response现在已弃用。request.body而是使用它来处理非常规表单数据,例如XML有效负载,二进制图像等。

有关此问题的Django文档

request.raw_response is now deprecated. Use request.body instead to process non-conventional form data such as XML payloads, binary images, etc.

Django documentation on the issue.


回答 5

在Django 1.6 python 3.3上

客户

$.ajax({
    url: '/urll/',
    type: 'POST',
    contentType: 'application/json; charset=utf-8',
    data: JSON.stringify(json_object),
    dataType: 'json',
    success: function(result) {
        alert(result.Result);
    }
});

服务器

def urll(request):

if request.is_ajax():
    if request.method == 'POST':
        print ('Raw Data:', request.body) 

        print ('type(request.body):', type(request.body)) # this type is bytes

        print(json.loads(request.body.decode("utf-8")))

on django 1.6 python 3.3

client

$.ajax({
    url: '/urll/',
    type: 'POST',
    contentType: 'application/json; charset=utf-8',
    data: JSON.stringify(json_object),
    dataType: 'json',
    success: function(result) {
        alert(result.Result);
    }
});

server

def urll(request):

if request.is_ajax():
    if request.method == 'POST':
        print ('Raw Data:', request.body) 

        print ('type(request.body):', type(request.body)) # this type is bytes

        print(json.loads(request.body.decode("utf-8")))

回答 6

HTTP POST有效负载只是一堆字节。Django(与大多数框架一样)通过URL编码参数或MIME多部分编码将其解码为字典。如果仅将JSON数据转储到POST内容中,则Django将不会对其进行解码。从完整的POST内容(而不是字典)中进行JSON解码;或将JSON数据放入MIME多部分包装器中。

简而言之,请显示JavaScript代码。问题似乎在那里。

The HTTP POST payload is just a flat bunch of bytes. Django (like most frameworks) decodes it into a dictionary from either URL encoded parameters, or MIME-multipart encoding. If you just dump the JSON data in the POST content, Django won’t decode it. Either do the JSON decoding from the full POST content (not the dictionary); or put the JSON data into a MIME-multipart wrapper.

In short, show the JavaScript code. The problem seems to be there.


回答 7

request.raw_post_data已不推荐使用。使用request.body替代

request.raw_post_data has been deprecated. Use request.body instead


回答 8

这样的事情。它的工作原理:从客户端请求数据

registerData = {
{% for field in userFields%}
  {{ field.name }}: {{ field.name }},
{% endfor %}
}


var request = $.ajax({
   url: "{% url 'MainApp:rq-create-account-json' %}",
   method: "POST",
   async: false,
   contentType: "application/json; charset=utf-8",
   data: JSON.stringify(registerData),
   dataType: "json"
});

request.done(function (msg) {
   [alert(msg);]
   alert(msg.name);
});

request.fail(function (jqXHR, status) {
  alert(status);
});

在服务器上处理请求

@csrf_exempt
def rq_create_account_json(request):
   if request.is_ajax():
       if request.method == 'POST':
           json_data = json.loads(request.body)
           print(json_data)
           return JsonResponse(json_data)
   return HttpResponse("Error")

Something like this. It’s worked: Request data from client

registerData = {
{% for field in userFields%}
  {{ field.name }}: {{ field.name }},
{% endfor %}
}


var request = $.ajax({
   url: "{% url 'MainApp:rq-create-account-json' %}",
   method: "POST",
   async: false,
   contentType: "application/json; charset=utf-8",
   data: JSON.stringify(registerData),
   dataType: "json"
});

request.done(function (msg) {
   [alert(msg);]
   alert(msg.name);
});

request.fail(function (jqXHR, status) {
  alert(status);
});

Process request at the server

@csrf_exempt
def rq_create_account_json(request):
   if request.is_ajax():
       if request.method == 'POST':
           json_data = json.loads(request.body)
           print(json_data)
           return JsonResponse(json_data)
   return HttpResponse("Error")

回答 9

html code 

file name  : view.html


    <!DOCTYPE html>
    <html>
    <head>
    <script src="http://ajax.googleapis.com/ajax/libs/jquery/1.11.1/jquery.min.js"></script>
    <script>
    $(document).ready(function(){
        $("#mySelect").change(function(){
            selected = $("#mySelect option:selected").text()
            $.ajax({
                type: 'POST',
                dataType: 'json',
                contentType: 'application/json; charset=utf-8',
                url: '/view/',
                data: {
                       'fruit': selected
                      },
                success: function(result) {
                        document.write(result)
                        }
        });
      });
    });
    </script>
    </head>
    <body>

    <form>
        <br>
    Select your favorite fruit:
    <select id="mySelect">
      <option value="apple" selected >Select fruit</option>
      <option value="apple">Apple</option>
      <option value="orange">Orange</option>
      <option value="pineapple">Pineapple</option>
      <option value="banana">Banana</option>
    </select>
    </form>
    </body>
    </html>

Django code:


Inside views.py


def view(request):

    if request.method == 'POST':
        print request.body
        data = request.body
        return HttpResponse(json.dumps(data))
html code 

file name  : view.html


    <!DOCTYPE html>
    <html>
    <head>
    <script src="http://ajax.googleapis.com/ajax/libs/jquery/1.11.1/jquery.min.js"></script>
    <script>
    $(document).ready(function(){
        $("#mySelect").change(function(){
            selected = $("#mySelect option:selected").text()
            $.ajax({
                type: 'POST',
                dataType: 'json',
                contentType: 'application/json; charset=utf-8',
                url: '/view/',
                data: {
                       'fruit': selected
                      },
                success: function(result) {
                        document.write(result)
                        }
        });
      });
    });
    </script>
    </head>
    <body>

    <form>
        <br>
    Select your favorite fruit:
    <select id="mySelect">
      <option value="apple" selected >Select fruit</option>
      <option value="apple">Apple</option>
      <option value="orange">Orange</option>
      <option value="pineapple">Pineapple</option>
      <option value="banana">Banana</option>
    </select>
    </form>
    </body>
    </html>

Django code:


Inside views.py


def view(request):

    if request.method == 'POST':
        print request.body
        data = request.body
        return HttpResponse(json.dumps(data))

回答 10

使用Angular您应该添加标头以请求或将其添加到模块配置标头: {'Content-Type': 'application/x-www-form-urlencoded'}

$http({
    url: url,
    method: method,
    timeout: timeout,
    data: data,
    headers: {'Content-Type': 'application/x-www-form-urlencoded'}
})

Using Angular you should add header to request or add it to module config headers: {'Content-Type': 'application/x-www-form-urlencoded'}

$http({
    url: url,
    method: method,
    timeout: timeout,
    data: data,
    headers: {'Content-Type': 'application/x-www-form-urlencoded'}
})

回答 11

request.POST只是一个类似于字典的对象,因此只需使用dict语法对其进行索引。

假设您的表单字段为fred,则可以执行以下操作:

if 'fred' in request.POST:
    mydata = request.POST['fred']

或者,使用表单对象处理POST数据。

request.POST is just a dictionary-like object, so just index into it with dict syntax.

Assuming your form field is fred, you could do something like this:

if 'fred' in request.POST:
    mydata = request.POST['fred']

Alternately, use a form object to deal with the POST data.


可以使用scrapy从使用AJAX的网站中抓取动态内容吗?

问题:可以使用scrapy从使用AJAX的网站中抓取动态内容吗?

我最近一直在学习Python,并全力以赴来构建网络抓取工具。一点都不花哨。其唯一目的是从博彩网站上获取数据,并将此数据放入Excel。

大多数问题都是可以解决的,我周围有些混乱。但是,我在一个问题上遇到了巨大的障碍。如果站点加载一张马表并列出当前的投注价格,则此信息不在任何源文件中。线索是这些数据有时是实时的,而数字显然是从某个远程服务器上更新的。我PC上的HTML只是有一个漏洞,他们的服务器正在推送我需要的所有有趣数据。

现在我对动态Web内容的经验很低,所以这件事使我难以理解。

我认为Java或Javascript是关键,这经常弹出。

刮板只是赔率比较引擎。有些网站有API,但对于那些没有的API则需要。我正在使用python 2.7的scrapy库

如果这个问题过于开放,我深表歉意。简而言之,我的问题是:如何使用scrapy来抓取此动态数据,以便可以使用它?这样我就可以实时抓取该赔率赔率数据?

I have recently been learning Python and am dipping my hand into building a web-scraper. It’s nothing fancy at all; its only purpose is to get the data off of a betting website and have this data put into Excel.

Most of the issues are solvable and I’m having a good little mess around. However I’m hitting a massive hurdle over one issue. If a site loads a table of horses and lists current betting prices this information is not in any source file. The clue is that this data is live sometimes, with the numbers being updated obviously from some remote server. The HTML on my PC simply has a hole where their servers are pushing through all the interesting data that I need.

Now my experience with dynamic web content is low, so this thing is something I’m having trouble getting my head around.

I think Java or Javascript is a key, this pops up often.

The scraper is simply a odds comparison engine. Some sites have APIs but I need this for those that don’t. I’m using the scrapy library with Python 2.7

I do apologize if this question is too open-ended. In short, my question is: how can scrapy be used to scrape this dynamic data so that I can use it? So that I can scrape this betting odds data in real-time?


回答 0

基于Webkit的浏览器(例如Google Chrome或Safari)具有内置的开发人员工具。在Chrome中,您可以将其打开Menu->Tools->Developer Tools。该Network选项卡使您可以查看有关每个请求和响应的所有信息:

在图片的底部,您可以看到我已将请求过滤为XHR-这些是由javascript代码发出的请求。

提示:每次加载页面时都会清除日志,在图片底部,黑点按钮将保留日志。

分析请求和响应后,您可以模拟来自网络爬虫的请求并提取有价值的数据。在许多情况下,获取数据比解析HTML更容易,因为该数据不包含表示逻辑,并且其格式设置为可以由javascript代码访问。

Firefox具有类似的扩展名,称为firebug。有人会说萤火虫功能更强大,但我喜欢webkit的简单性。

Webkit based browsers (like Google Chrome or Safari) has built-in developer tools. In Chrome you can open it Menu->Tools->Developer Tools. The Network tab allows you to see all information about every request and response:

In the bottom of the picture you can see that I’ve filtered request down to XHR – these are requests made by javascript code.

Tip: log is cleared every time you load a page, at the bottom of the picture, the black dot button will preserve log.

After analyzing requests and responses you can simulate these requests from your web-crawler and extract valuable data. In many cases it will be easier to get your data than parsing HTML, because that data does not contain presentation logic and is formatted to be accessed by javascript code.

Firefox has similar extension, it is called firebug. Some will argue that firebug is even more powerful but I like the simplicity of webkit.


回答 1

这是一个scrapy带有AJAX请求的简单示例 。让我们看看网站rubin-kazan.ru

所有消息都加载了AJAX请求。我的目标是获取这些消息及其所有属性(作者,日期等):

当我分析页面的源代码时,因为网页使用AJAX技术,所以看不到所有这些消息。但是我可以使用Mozilla Firefox中的Firebug(或其他浏览器中的等效工具)来分析HTTP请求,该请求会在网页上生成消息:

它不会重新加载整个页面,而是仅重新加载页面中包含消息的部分。为此,我单击底部任意数量的页面:

我观察到负责邮件正文的HTTP请求:

完成后,我分析请求的标头(我必须引用将从var部分的源页面中提取的该URL,请参见下面的代码):

和请求的表单数据内容(HTTP方法为“ Post”):

以及响应的内容,它是一个JSON文件:

其中显示了我正在寻找的所有信息。

从现在开始,我必须抓紧实施所有这些知识。为此,我们定义蜘蛛:

class spider(BaseSpider):
    name = 'RubiGuesst'
    start_urls = ['http://www.rubin-kazan.ru/guestbook.html']

    def parse(self, response):
        url_list_gb_messages = re.search(r'url_list_gb_messages="(.*)"', response.body).group(1)
        yield FormRequest('http://www.rubin-kazan.ru' + url_list_gb_messages, callback=self.RubiGuessItem,
                          formdata={'page': str(page + 1), 'uid': ''})

    def RubiGuessItem(self, response):
        json_file = response.body

parse函数中,我有第一个请求的响应。在RubiGuessItem我有所有信息的JSON文件。

Here is a simple example of scrapy with an AJAX request. Let see the site rubin-kazan.ru.

All messages are loaded with an AJAX request. My goal is to fetch these messages with all their attributes (author, date, …):

When I analyze the source code of the page I can’t see all these messages because the web page uses AJAX technology. But I can with Firebug from Mozilla Firefox (or an equivalent tool in other browsers) to analyze the HTTP request that generate the messages on the web page:

It doesn’t reload the whole page but only the parts of the page that contain messages. For this purpose I click an arbitrary number of page on the bottom:

And I observe the HTTP request that is responsible for message body:

After finish, I analyze the headers of the request (I must quote that this URL I’ll extract from source page from var section, see the code below):

And the form data content of the request (the HTTP method is “Post”):

And the content of response, which is a JSON file:

Which presents all the information I’m looking for.

From now, I must implement all this knowledge in scrapy. Let’s define the spider for this purpose:

class spider(BaseSpider):
    name = 'RubiGuesst'
    start_urls = ['http://www.rubin-kazan.ru/guestbook.html']

    def parse(self, response):
        url_list_gb_messages = re.search(r'url_list_gb_messages="(.*)"', response.body).group(1)
        yield FormRequest('http://www.rubin-kazan.ru' + url_list_gb_messages, callback=self.RubiGuessItem,
                          formdata={'page': str(page + 1), 'uid': ''})

    def RubiGuessItem(self, response):
        json_file = response.body

In parse function I have the response for first request. In RubiGuessItem I have the JSON file with all information.


回答 2

进行爬网时,很多时候我们会遇到这样的问题:页面上呈现的内容是使用Javascript生成的,因此scrapy无法为其进行爬网(例如ajax请求,jQuery疯狂)。

但是,如果您将Scrapy与Web测试框架Selenium一起使用,则我们能够对普通Web浏览器中显示的内容进行爬网。

注意事项:

  • 您必须安装Python版本的Selenium RC才能正常工作,并且必须正确设置Selenium。这也只是模板搜寻器。您可能会更疯狂,更先进,但是我只是想展示基本思想。如代码所示,您将对任何给定的URL进行两个请求。Scrapy提出了一个请求,Selenium提出了另一个请求。我确信有办法解决这个问题,这样您就可以让Selenium做一个唯一的请求,但是我没有理会实现这一点,通过执行两个请求,您也可以使用Scrapy抓取页面。

  • 这非常强大,因为现在您可以抓取整个呈现的DOM,并且仍然可以使用Scrapy中所有不错的抓取功能。当然,这将使爬网速度变慢,但是取决于您需要呈现的DOM的多少,可能值得等待。

    from scrapy.contrib.spiders import CrawlSpider, Rule
    from scrapy.contrib.linkextractors.sgml import SgmlLinkExtractor
    from scrapy.selector import HtmlXPathSelector
    from scrapy.http import Request
    
    from selenium import selenium
    
    class SeleniumSpider(CrawlSpider):
        name = "SeleniumSpider"
        start_urls = ["http://www.domain.com"]
    
        rules = (
            Rule(SgmlLinkExtractor(allow=('\.html', )), callback='parse_page',follow=True),
        )
    
        def __init__(self):
            CrawlSpider.__init__(self)
            self.verificationErrors = []
            self.selenium = selenium("localhost", 4444, "*chrome", "http://www.domain.com")
            self.selenium.start()
    
        def __del__(self):
            self.selenium.stop()
            print self.verificationErrors
            CrawlSpider.__del__(self)
    
        def parse_page(self, response):
            item = Item()
    
            hxs = HtmlXPathSelector(response)
            #Do some XPath selection with Scrapy
            hxs.select('//div').extract()
    
            sel = self.selenium
            sel.open(response.url)
    
            #Wait for javscript to load in Selenium
            time.sleep(2.5)
    
            #Do some crawling of javascript created content with Selenium
            sel.get_text("//div")
            yield item
    
    # Snippet imported from snippets.scrapy.org (which no longer works)
    # author: wynbennett
    # date  : Jun 21, 2011

参考:http//snipplr.com/view/66998/

Many times when crawling we run into problems where content that is rendered on the page is generated with Javascript and therefore scrapy is unable to crawl for it (eg. ajax requests, jQuery craziness).

However, if you use Scrapy along with the web testing framework Selenium then we are able to crawl anything displayed in a normal web browser.

Some things to note:

  • You must have the Python version of Selenium RC installed for this to work, and you must have set up Selenium properly. Also this is just a template crawler. You could get much crazier and more advanced with things but I just wanted to show the basic idea. As the code stands now you will be doing two requests for any given url. One request is made by Scrapy and the other is made by Selenium. I am sure there are ways around this so that you could possibly just make Selenium do the one and only request but I did not bother to implement that and by doing two requests you get to crawl the page with Scrapy too.

  • This is quite powerful because now you have the entire rendered DOM available for you to crawl and you can still use all the nice crawling features in Scrapy. This will make for slower crawling of course but depending on how much you need the rendered DOM it might be worth the wait.

    from scrapy.contrib.spiders import CrawlSpider, Rule
    from scrapy.contrib.linkextractors.sgml import SgmlLinkExtractor
    from scrapy.selector import HtmlXPathSelector
    from scrapy.http import Request
    
    from selenium import selenium
    
    class SeleniumSpider(CrawlSpider):
        name = "SeleniumSpider"
        start_urls = ["http://www.domain.com"]
    
        rules = (
            Rule(SgmlLinkExtractor(allow=('\.html', )), callback='parse_page',follow=True),
        )
    
        def __init__(self):
            CrawlSpider.__init__(self)
            self.verificationErrors = []
            self.selenium = selenium("localhost", 4444, "*chrome", "http://www.domain.com")
            self.selenium.start()
    
        def __del__(self):
            self.selenium.stop()
            print self.verificationErrors
            CrawlSpider.__del__(self)
    
        def parse_page(self, response):
            item = Item()
    
            hxs = HtmlXPathSelector(response)
            #Do some XPath selection with Scrapy
            hxs.select('//div').extract()
    
            sel = self.selenium
            sel.open(response.url)
    
            #Wait for javscript to load in Selenium
            time.sleep(2.5)
    
            #Do some crawling of javascript created content with Selenium
            sel.get_text("//div")
            yield item
    
    # Snippet imported from snippets.scrapy.org (which no longer works)
    # author: wynbennett
    # date  : Jun 21, 2011
    

Reference: http://snipplr.com/view/66998/


回答 3

另一个解决方案是实现下载处理程序或下载处理程序中间件。(有关下载器中间件的更多信息,请参见scrapy docs)以下是将硒与无头phantomjs webdriver一起使用的示例类:

1)middlewares.py脚本中定义类。

from selenium import webdriver
from scrapy.http import HtmlResponse

class JsDownload(object):

    @check_spider_middleware
    def process_request(self, request, spider):
        driver = webdriver.PhantomJS(executable_path='D:\phantomjs.exe')
        driver.get(request.url)
        return HtmlResponse(request.url, encoding='utf-8', body=driver.page_source.encode('utf-8'))

2)JsDownload()在变量DOWNLOADER_MIDDLEWARE内添加类settings.py

DOWNLOADER_MIDDLEWARES = {'MyProj.middleware.MiddleWareModule.MiddleWareClass': 500}

3)整合HTMLResponse内部your_spider.py。解码响应主体将为您提供所需的输出。

class Spider(CrawlSpider):
    # define unique name of spider
    name = "spider"

    start_urls = ["https://www.url.de"] 

    def parse(self, response):
        # initialize items
        item = CrawlerItem()

        # store data as items
        item["js_enabled"] = response.body.decode("utf-8") 

可选的插件:
我希望能够告诉不同的Spider使用哪个中间件,因此我实现了此包装器:

def check_spider_middleware(method):
@functools.wraps(method)
def wrapper(self, request, spider):
    msg = '%%s %s middleware step' % (self.__class__.__name__,)
    if self.__class__ in spider.middleware:
        spider.log(msg % 'executing', level=log.DEBUG)
        return method(self, request, spider)
    else:
        spider.log(msg % 'skipping', level=log.DEBUG)
        return None

return wrapper

为了使包装工作,所有蜘蛛必须至少具有:

middleware = set([])

包括中间件:

middleware = set([MyProj.middleware.ModuleName.ClassName])

优点:
以这种方式而不是在蜘蛛网中实现它的主要优点是,您最终只会提出一个请求。例如,在AT解决方案中:下载处理程序处理请求,然后将响应传递给蜘蛛。然后,蜘蛛程序将在其parse_page函数中提出一个全新的请求-这是对相同内容的两个请求。

Another solution would be to implement a download handler or download handler middleware. (see scrapy docs for more information on downloader middleware) The following is an example class using selenium with headless phantomjs webdriver:

1) Define class within the middlewares.py script.

from selenium import webdriver
from scrapy.http import HtmlResponse

class JsDownload(object):

    @check_spider_middleware
    def process_request(self, request, spider):
        driver = webdriver.PhantomJS(executable_path='D:\phantomjs.exe')
        driver.get(request.url)
        return HtmlResponse(request.url, encoding='utf-8', body=driver.page_source.encode('utf-8'))

2) Add JsDownload() class to variable DOWNLOADER_MIDDLEWARE within settings.py:

DOWNLOADER_MIDDLEWARES = {'MyProj.middleware.MiddleWareModule.MiddleWareClass': 500}

3) Integrate the HTMLResponse within your_spider.py. Decoding the response body will get you the desired output.

class Spider(CrawlSpider):
    # define unique name of spider
    name = "spider"

    start_urls = ["https://www.url.de"] 

    def parse(self, response):
        # initialize items
        item = CrawlerItem()

        # store data as items
        item["js_enabled"] = response.body.decode("utf-8") 

Optional Addon:
I wanted the ability to tell different spiders which middleware to use so I implemented this wrapper:

def check_spider_middleware(method):
@functools.wraps(method)
def wrapper(self, request, spider):
    msg = '%%s %s middleware step' % (self.__class__.__name__,)
    if self.__class__ in spider.middleware:
        spider.log(msg % 'executing', level=log.DEBUG)
        return method(self, request, spider)
    else:
        spider.log(msg % 'skipping', level=log.DEBUG)
        return None

return wrapper

for wrapper to work all spiders must have at minimum:

middleware = set([])

to include a middleware:

middleware = set([MyProj.middleware.ModuleName.ClassName])

Advantage:
The main advantage to implementing it this way rather than in the spider is that you only end up making one request. In A T’s solution for example: The download handler processes the request and then hands off the response to the spider. The spider then makes a brand new request in it’s parse_page function — That’s two requests for the same content.


回答 4

我当时使用的是自定义下载器中间件,但对此并不满意,因为我没有设法使缓存与之配合使用。

更好的方法是实现自定义下载处理程序。

有一个工作示例这里。看起来像这样:

# encoding: utf-8
from __future__ import unicode_literals

from scrapy import signals
from scrapy.signalmanager import SignalManager
from scrapy.responsetypes import responsetypes
from scrapy.xlib.pydispatch import dispatcher
from selenium import webdriver
from six.moves import queue
from twisted.internet import defer, threads
from twisted.python.failure import Failure


class PhantomJSDownloadHandler(object):

    def __init__(self, settings):
        self.options = settings.get('PHANTOMJS_OPTIONS', {})

        max_run = settings.get('PHANTOMJS_MAXRUN', 10)
        self.sem = defer.DeferredSemaphore(max_run)
        self.queue = queue.LifoQueue(max_run)

        SignalManager(dispatcher.Any).connect(self._close, signal=signals.spider_closed)

    def download_request(self, request, spider):
        """use semaphore to guard a phantomjs pool"""
        return self.sem.run(self._wait_request, request, spider)

    def _wait_request(self, request, spider):
        try:
            driver = self.queue.get_nowait()
        except queue.Empty:
            driver = webdriver.PhantomJS(**self.options)

        driver.get(request.url)
        # ghostdriver won't response when switch window until page is loaded
        dfd = threads.deferToThread(lambda: driver.switch_to.window(driver.current_window_handle))
        dfd.addCallback(self._response, driver, spider)
        return dfd

    def _response(self, _, driver, spider):
        body = driver.execute_script("return document.documentElement.innerHTML")
        if body.startswith("<head></head>"):  # cannot access response header in Selenium
            body = driver.execute_script("return document.documentElement.textContent")
        url = driver.current_url
        respcls = responsetypes.from_args(url=url, body=body[:100].encode('utf8'))
        resp = respcls(url=url, body=body, encoding="utf-8")

        response_failed = getattr(spider, "response_failed", None)
        if response_failed and callable(response_failed) and response_failed(resp, driver):
            driver.close()
            return defer.fail(Failure())
        else:
            self.queue.put(driver)
            return defer.succeed(resp)

    def _close(self):
        while not self.queue.empty():
            driver = self.queue.get_nowait()
            driver.close()

假设您的刮板称为“刮板”。如果您将提到的代码放在“ scraper”文件夹根目录下的名为handlers.py的文件中,则可以将其添加到settings.py中:

DOWNLOAD_HANDLERS = {
    'http': 'scraper.handlers.PhantomJSDownloadHandler',
    'https': 'scraper.handlers.PhantomJSDownloadHandler',
}

还有JS解析DOM的voilà,其中包含抓取的缓存,重试等。

I was using a custom downloader middleware, but wasn’t very happy with it, as I didn’t manage to make the cache work with it.

A better approach was to implement a custom download handler.

There is a working example here. It looks like this:

# encoding: utf-8
from __future__ import unicode_literals

from scrapy import signals
from scrapy.signalmanager import SignalManager
from scrapy.responsetypes import responsetypes
from scrapy.xlib.pydispatch import dispatcher
from selenium import webdriver
from six.moves import queue
from twisted.internet import defer, threads
from twisted.python.failure import Failure


class PhantomJSDownloadHandler(object):

    def __init__(self, settings):
        self.options = settings.get('PHANTOMJS_OPTIONS', {})

        max_run = settings.get('PHANTOMJS_MAXRUN', 10)
        self.sem = defer.DeferredSemaphore(max_run)
        self.queue = queue.LifoQueue(max_run)

        SignalManager(dispatcher.Any).connect(self._close, signal=signals.spider_closed)

    def download_request(self, request, spider):
        """use semaphore to guard a phantomjs pool"""
        return self.sem.run(self._wait_request, request, spider)

    def _wait_request(self, request, spider):
        try:
            driver = self.queue.get_nowait()
        except queue.Empty:
            driver = webdriver.PhantomJS(**self.options)

        driver.get(request.url)
        # ghostdriver won't response when switch window until page is loaded
        dfd = threads.deferToThread(lambda: driver.switch_to.window(driver.current_window_handle))
        dfd.addCallback(self._response, driver, spider)
        return dfd

    def _response(self, _, driver, spider):
        body = driver.execute_script("return document.documentElement.innerHTML")
        if body.startswith("<head></head>"):  # cannot access response header in Selenium
            body = driver.execute_script("return document.documentElement.textContent")
        url = driver.current_url
        respcls = responsetypes.from_args(url=url, body=body[:100].encode('utf8'))
        resp = respcls(url=url, body=body, encoding="utf-8")

        response_failed = getattr(spider, "response_failed", None)
        if response_failed and callable(response_failed) and response_failed(resp, driver):
            driver.close()
            return defer.fail(Failure())
        else:
            self.queue.put(driver)
            return defer.succeed(resp)

    def _close(self):
        while not self.queue.empty():
            driver = self.queue.get_nowait()
            driver.close()

Suppose your scraper is called “scraper”. If you put the mentioned code inside a file called handlers.py on the root of the “scraper” folder, then you could add to your settings.py:

DOWNLOAD_HANDLERS = {
    'http': 'scraper.handlers.PhantomJSDownloadHandler',
    'https': 'scraper.handlers.PhantomJSDownloadHandler',
}

And voilà, the JS parsed DOM, with scrapy cache, retries, etc.


回答 5

如何使用scrapy来抓取此动态数据,以便可以使用它?

我想知道为什么没有人仅使用Scrapy发布解决方案。

查阅Scrapy小组SCRAPING INFINITE SCROLLING PAGES 的博客文章。该示例抓取了使用无限滚动的http://spidyquotes.herokuapp.com/scroll网站。

这个想法是使用浏览器的开发人员工具并注意AJAX请求,然后根据该信息创建对Scrapy的请求

import json
import scrapy


class SpidyQuotesSpider(scrapy.Spider):
    name = 'spidyquotes'
    quotes_base_url = 'http://spidyquotes.herokuapp.com/api/quotes?page=%s'
    start_urls = [quotes_base_url % 1]
    download_delay = 1.5

    def parse(self, response):
        data = json.loads(response.body)
        for item in data.get('quotes', []):
            yield {
                'text': item.get('text'),
                'author': item.get('author', {}).get('name'),
                'tags': item.get('tags'),
            }
        if data['has_next']:
            next_page = data['page'] + 1
            yield scrapy.Request(self.quotes_base_url % next_page)

how can scrapy be used to scrape this dynamic data so that I can use it?

I wonder why no one has posted the solution using Scrapy only.

Check out the blog post from Scrapy team SCRAPING INFINITE SCROLLING PAGES . The example scraps http://spidyquotes.herokuapp.com/scroll website which uses infinite scrolling.

The idea is to use Developer Tools of your browser and notice the AJAX requests, then based on that information create the requests for Scrapy.

import json
import scrapy


class SpidyQuotesSpider(scrapy.Spider):
    name = 'spidyquotes'
    quotes_base_url = 'http://spidyquotes.herokuapp.com/api/quotes?page=%s'
    start_urls = [quotes_base_url % 1]
    download_delay = 1.5

    def parse(self, response):
        data = json.loads(response.body)
        for item in data.get('quotes', []):
            yield {
                'text': item.get('text'),
                'author': item.get('author', {}).get('name'),
                'tags': item.get('tags'),
            }
        if data['has_next']:
            next_page = data['page'] + 1
            yield scrapy.Request(self.quotes_base_url % next_page)

回答 6

是的,Scrapy可以废弃动态网站,即通过javaScript呈现的网站。

抓取此类网站有两种方法。

第一,

您可以splash用来呈现Javascript代码,然后解析呈现的HTML。你可以在这里找到文档和项目Scrapy splash,git

第二,

就像每个人都在说的那样,通过监视network calls,可以的,您可以找到获取数据的api调用,并在您的scrapy spider中模拟该调用可能会帮助您获取所需的数据。

yes, Scrapy can scrap dynamic websites, website that are rendered through javaScript.

There are Two approaches to scrapy these kind of websites.

First,

you can use splash to render Javascript code and then parse the rendered HTML. you can find the doc and project here Scrapy splash, git

Second,

As everyone is stating, by monitoring the network calls, yes, you can find the api call that fetch the data and mock that call in your scrapy spider might help you to get desired data.


回答 7

我通过使用Selenium和Firefox Web驱动程序来处理ajax请求。如果需要使用爬虫作为守护程序,速度不是很快,但是比任何手动解决方案要好得多。我在这里写了一个简短的教程供参考

I handle the ajax request by using Selenium and the Firefox web driver. It is not that fast if you need the crawler as a daemon, but much better than any manual solution. I wrote a short tutorial here for reference


Django CSRF检查失败,并带有Ajax POST请求

问题:Django CSRF检查失败,并带有Ajax POST请求

我可以通过我的AJAX帖子向遵循Django CSRF保护机制的人员提供帮助。我按照这里的指示进行:

http://docs.djangoproject.com/en/dev/ref/contrib/csrf/

我已经完全复制了该页面上的AJAX示例代码:

http://docs.djangoproject.com/en/dev/ref/contrib/csrf/#ajax

getCookie('csrftoken')xhr.setRequestHeader呼叫之前打印了警报内容,并确实在其中填充了一些数据。我不确定如何验证令牌是否正确,但是我鼓励它正在查找并发送一些消息。

但是Django仍然拒绝我的AJAX帖子。

这是我的JavaScript:

$.post("/memorize/", data, function (result) {
    if (result != "failure") {
        get_random_card();
    }
    else {
        alert("Failed to save card data.");
    }
});

这是我在Django中看到的错误:

[23 / Feb / 2011 22:08:29]“ POST / memorize / HTTP / 1.1” 403 2332

我确定我缺少某些东西,也许很简单,但是我不知道它是什么。我在SO周围搜索,并看到了一些有关通过csrf_exempt装饰器关闭CSRF检查以获取我的视图的信息,但是我发现这没有吸引力。我已经尝试过了,并且可以工作,但是我宁愿让POST按照Django期望的方式工作。

以防万一,这是我的观点正在做的要点:

def myview(request):

    profile = request.user.profile

    if request.method == 'POST':
        """
        Process the post...
        """
        return HttpResponseRedirect('/memorize/')
    else: # request.method == 'GET'

        ajax = request.GET.has_key('ajax')

        """
        Some irrelevent code...
        """

        if ajax:
            response = HttpResponse()
            profile.get_stack_json(response)
            return response
        else:
            """
            Get data to send along with the content of the page.
            """

        return render_to_response('memorize/memorize.html',
                """ My data """
                context_instance=RequestContext(request))

感谢您的回复!

I could use some help complying with Django’s CSRF protection mechanism via my AJAX post. I’ve followed the directions here:

http://docs.djangoproject.com/en/dev/ref/contrib/csrf/

I’ve copied the AJAX sample code they have on that page exactly:

http://docs.djangoproject.com/en/dev/ref/contrib/csrf/#ajax

I put an alert printing the contents of getCookie('csrftoken') before the xhr.setRequestHeader call and it is indeed populated with some data. I’m not sure how to verify that the token is correct, but I’m encouraged that it’s finding and sending something.

But Django is still rejecting my AJAX post.

Here’s my JavaScript:

$.post("/memorize/", data, function (result) {
    if (result != "failure") {
        get_random_card();
    }
    else {
        alert("Failed to save card data.");
    }
});

Here’s the error I’m seeing from Django:

[23/Feb/2011 22:08:29] “POST /memorize/ HTTP/1.1” 403 2332

I’m sure I’m missing something, and maybe it’s simple, but I don’t know what it is. I’ve searched around SO and saw some information about turning off the CSRF check for my view via the csrf_exempt decorator, but I find that unappealing. I’ve tried that out and it works, but I’d rather get my POST to work the way Django was designed to expect it, if possible.

Just in case it’s helpful, here’s the gist of what my view is doing:

def myview(request):

    profile = request.user.profile

    if request.method == 'POST':
        """
        Process the post...
        """
        return HttpResponseRedirect('/memorize/')
    else: # request.method == 'GET'

        ajax = request.GET.has_key('ajax')

        """
        Some irrelevent code...
        """

        if ajax:
            response = HttpResponse()
            profile.get_stack_json(response)
            return response
        else:
            """
            Get data to send along with the content of the page.
            """

        return render_to_response('memorize/memorize.html',
                """ My data """
                context_instance=RequestContext(request))

Thanks for your replies!


回答 0

真正的解决方案

好的,我设法找出问题所在。它位于Javascript(正如我在下面建议的)代码中。

您需要的是:

$.ajaxSetup({ 
     beforeSend: function(xhr, settings) {
         function getCookie(name) {
             var cookieValue = null;
             if (document.cookie && document.cookie != '') {
                 var cookies = document.cookie.split(';');
                 for (var i = 0; i < cookies.length; i++) {
                     var cookie = jQuery.trim(cookies[i]);
                     // Does this cookie string begin with the name we want?
                     if (cookie.substring(0, name.length + 1) == (name + '=')) {
                         cookieValue = decodeURIComponent(cookie.substring(name.length + 1));
                         break;
                     }
                 }
             }
             return cookieValue;
         }
         if (!(/^http:.*/.test(settings.url) || /^https:.*/.test(settings.url))) {
             // Only send the token to relative URLs i.e. locally.
             xhr.setRequestHeader("X-CSRFToken", getCookie('csrftoken'));
         }
     } 
});

而不是官方文档中发布的代码:https//docs.djangoproject.com/en/2.2/ref/csrf/

工作代码来自以下Django条目:http : //www.djangoproject.com/weblog/2011/feb/08/security/

因此,一般的解决方案是:“使用ajaxSetup处理程序而不是ajaxSend处理程序”。我不知道为什么会这样。但这对我有用:)

以前的帖子(无答案)

我实际上遇到了同样的问题。

它在更新到Django 1.2.5之后发生-在Django 1.2.4中AJAX POST请求没有错误(AJAX不受任何保护,但效果很好)。

就像OP一样,我尝试了Django文档中发布的JavaScript代码段。我正在使用jQuery 1.5。我也在使用“ django.middleware.csrf.CsrfViewMiddleware”中间件。

我尝试遵循中间件代码,但我知道它在此方面失败:

request_csrf_token = request.META.get('HTTP_X_CSRFTOKEN', '')

然后

if request_csrf_token != csrf_token:
    return self._reject(request, REASON_BAD_TOKEN)

此“如果”为true,因为“ request_csrf_token”为空。

基本上,这意味着未设置标头。那么此JS行有什么问题吗:

xhr.setRequestHeader("X-CSRFToken", getCookie('csrftoken'));

我希望提供的详细信息将有助于我们解决问题:)

Real solution

Ok, I managed to trace the problem down. It lies in the Javascript (as I suggested below) code.

What you need is this:

$.ajaxSetup({ 
     beforeSend: function(xhr, settings) {
         function getCookie(name) {
             var cookieValue = null;
             if (document.cookie && document.cookie != '') {
                 var cookies = document.cookie.split(';');
                 for (var i = 0; i < cookies.length; i++) {
                     var cookie = jQuery.trim(cookies[i]);
                     // Does this cookie string begin with the name we want?
                     if (cookie.substring(0, name.length + 1) == (name + '=')) {
                         cookieValue = decodeURIComponent(cookie.substring(name.length + 1));
                         break;
                     }
                 }
             }
             return cookieValue;
         }
         if (!(/^http:.*/.test(settings.url) || /^https:.*/.test(settings.url))) {
             // Only send the token to relative URLs i.e. locally.
             xhr.setRequestHeader("X-CSRFToken", getCookie('csrftoken'));
         }
     } 
});

instead of the code posted in the official docs: https://docs.djangoproject.com/en/2.2/ref/csrf/

The working code, comes from this Django entry: http://www.djangoproject.com/weblog/2011/feb/08/security/

So the general solution is: “use ajaxSetup handler instead of ajaxSend handler”. I don’t know why it works. But it works for me :)

Previous post (without answer)

I’m experiencing the same problem actually.

It occurs after updating to Django 1.2.5 – there were no errors with AJAX POST requests in Django 1.2.4 (AJAX wasn’t protected in any way, but it worked just fine).

Just like OP, I have tried the JavaScript snippet posted in Django documentation. I’m using jQuery 1.5. I’m also using the “django.middleware.csrf.CsrfViewMiddleware” middleware.

I tried to follow the the middleware code and I know that it fails on this:

request_csrf_token = request.META.get('HTTP_X_CSRFTOKEN', '')

and then

if request_csrf_token != csrf_token:
    return self._reject(request, REASON_BAD_TOKEN)

this “if” is true, because “request_csrf_token” is empty.

Basically it means that the header is NOT set. So is there anything wrong with this JS line:

xhr.setRequestHeader("X-CSRFToken", getCookie('csrftoken'));

?

I hope that provided details will help us in resolving the issue :)


回答 1

如果使用该$.ajax函数,则可以简单地将csrf令牌添加到数据主体中:

$.ajax({
    data: {
        somedata: 'somedata',
        moredata: 'moredata',
        csrfmiddlewaretoken: '{{ csrf_token }}'
    },

If you use the $.ajax function, you can simply add the csrf token in the data body:

$.ajax({
    data: {
        somedata: 'somedata',
        moredata: 'moredata',
        csrfmiddlewaretoken: '{{ csrf_token }}'
    },

回答 2

将此行添加到您的jQuery代码中:

$.ajaxSetup({
  data: {csrfmiddlewaretoken: '{{ csrf_token }}' },
});

并做了。

Add this line to your jQuery code:

$.ajaxSetup({
  data: {csrfmiddlewaretoken: '{{ csrf_token }}' },
});

and done.


回答 3

问题是因为django希望将cookie中的值作为表单数据的一部分传递回去。上一个答案中的代码使javascript能够找出cookie值并将其放入表单数据中。从技术的角度来看,这是一种不错的方法,但是它确实有些冗长。

过去,我更简单地通过获取JavaScript将令牌值放入发布数据中来完成此操作。

如果您在模板中使用{%csrf_token%},则会发出带有该值的隐藏表单字段。但是,如果您使用{{csrf_token}},则只会获得令牌的裸值,因此可以在javascript中使用此代码…。

csrf_token = "{{ csrf_token }}";

然后,您可以在哈希中添加所需的键名称,然后将其作为数据提交给ajax调用。

The issue is because django is expecting the value from the cookie to be passed back as part of the form data. The code from the previous answer is getting javascript to hunt out the cookie value and put it into the form data. Thats a lovely way of doing it from a technical point of view, but it does look a bit verbose.

In the past, I have done it more simply by getting the javascript to put the token value into the post data.

If you use {% csrf_token %} in your template, you will get a hidden form field emitted that carries the value. But, if you use {{ csrf_token }} you will just get the bare value of the token, so you can use this in javascript like this….

csrf_token = "{{ csrf_token }}";

Then you can include that, with the required key name in the hash you then submit as the data to the ajax call.


回答 4

{% csrf_token %}在HTML模板放里面<form></form>

转换为:

<input type='hidden' name='csrfmiddlewaretoken' value='Sdgrw2HfynbFgPcZ5sjaoAI5zsMZ4wZR' />

所以为什么不像这样在您的JS中grep它:

token = $("#change_password-form").find('input[name=csrfmiddlewaretoken]').val()

然后通过它,例如进行一些POST,例如:

$.post( "/panel/change_password/", {foo: bar, csrfmiddlewaretoken: token}, function(data){
    console.log(data);
});

The {% csrf_token %} put in html templates inside <form></form>

translates to something like:

<input type='hidden' name='csrfmiddlewaretoken' value='Sdgrw2HfynbFgPcZ5sjaoAI5zsMZ4wZR' />

so why not just grep it in your JS like this:

token = $("#change_password-form").find('input[name=csrfmiddlewaretoken]').val()

and then pass it e.g doing some POST, like:

$.post( "/panel/change_password/", {foo: bar, csrfmiddlewaretoken: token}, function(data){
    console.log(data);
});

回答 5

非jQuery答案:

var csrfcookie = function() {
    var cookieValue = null,
        name = 'csrftoken';
    if (document.cookie && document.cookie !== '') {
        var cookies = document.cookie.split(';');
        for (var i = 0; i < cookies.length; i++) {
            var cookie = cookies[i].trim();
            if (cookie.substring(0, name.length + 1) == (name + '=')) {
                cookieValue = decodeURIComponent(cookie.substring(name.length + 1));
                break;
            }
        }
    }
    return cookieValue;
};

用法:

var request = new XMLHttpRequest();
request.open('POST', url, true);
request.setRequestHeader('Content-Type', 'application/x-www-form-urlencoded; charset=UTF-8');
request.setRequestHeader('X-CSRFToken', csrfcookie());
request.onload = callback;
request.send(data);

Non-jquery answer:

var csrfcookie = function() {
    var cookieValue = null,
        name = 'csrftoken';
    if (document.cookie && document.cookie !== '') {
        var cookies = document.cookie.split(';');
        for (var i = 0; i < cookies.length; i++) {
            var cookie = cookies[i].trim();
            if (cookie.substring(0, name.length + 1) == (name + '=')) {
                cookieValue = decodeURIComponent(cookie.substring(name.length + 1));
                break;
            }
        }
    }
    return cookieValue;
};

usage:

var request = new XMLHttpRequest();
request.open('POST', url, true);
request.setRequestHeader('Content-Type', 'application/x-www-form-urlencoded; charset=UTF-8');
request.setRequestHeader('X-CSRFToken', csrfcookie());
request.onload = callback;
request.send(data);

回答 6

如果您的表单在没有JS的Django中正确发布,那么您应该能够使用Ajax逐步增强它,而不会造成任何黑客攻击或csrf令牌的混乱传递。只需序列化整个表单,它将自动选择所有表单字段,包括隐藏的csrf字段:

$('#myForm').submit(function(){
    var action = $(this).attr('action');
    var that = $(this);
    $.ajax({
        url: action,
        type: 'POST',
        data: that.serialize()
        ,success: function(data){
            console.log('Success!');
        }
    });
    return false;
});

我已经使用Django 1.3+和jQuery 1.5+进行了测试。显然,这将适用于任何HTML表单,而不仅仅是Django应用。

If your form posts correctly in Django without JS, you should be able to progressively enhance it with ajax without any hacking or messy passing of the csrf token. Just serialize the whole form and that will automatically pick up all your form fields including the hidden csrf field:

$('#myForm').submit(function(){
    var action = $(this).attr('action');
    var that = $(this);
    $.ajax({
        url: action,
        type: 'POST',
        data: that.serialize()
        ,success: function(data){
            console.log('Success!');
        }
    });
    return false;
});

I’ve tested this with Django 1.3+ and jQuery 1.5+. Obviously this will work for any HTML form, not just Django apps.


回答 7

将Firefox与Firebug结合使用。触发ajax请求时,打开“控制台”选项卡。这样一来,DEBUG=True您将得到漂亮的django错误页面作为响应,甚至可以在控制台选项卡中看到ajax响应的呈现的html。

然后,您将知道错误是什么。

Use Firefox with Firebug. Open the ‘Console’ tab while firing the ajax request. With DEBUG=True you get the nice django error page as response and you can even see the rendered html of the ajax response in the console tab.

Then you will know what the error is.


回答 8

可接受的答案很可能是鲱鱼。Django 1.2.4和1.2.5之间的区别是AJAX请求需要CSRF令牌。

我在Django 1.3上遇到了这个问题,这是由于未设置CSRF cookie引起的首先。除非必须,否则Django不会设置cookie。因此,运行在Django 1.2.4上的排他或大量ajax站点可能永远不会向客户端发送令牌,然后需要令牌的升级会导致403错误。

理想的解决方法是在这里:http : //docs.djangoproject.com/en/dev/ref/contrib/csrf/#page-uses-ajax-without-any-html-form
但您必须等待1.4,除非这只是跟上代码的文档

编辑

还要注意,以后的Django文档注意到jQuery 1.5中的一个错误,因此请确保您将Django建议的代码与1.5.1或更高版本一起使用:http : //docs.djangoproject.com/en/1.3/ref/contrib/csrf/#阿贾克斯

The accepted answer is most likely a red herring. The difference between Django 1.2.4 and 1.2.5 was the requirement for a CSRF token for AJAX requests.

I came across this problem on Django 1.3 and it was caused by the CSRF cookie not being set in the first place. Django will not set the cookie unless it has to. So an exclusively or heavily ajax site running on Django 1.2.4 would potentially never have sent a token to the client and then the upgrade requiring the token would cause the 403 errors.

The ideal fix is here: http://docs.djangoproject.com/en/dev/ref/contrib/csrf/#page-uses-ajax-without-any-html-form
but you’d have to wait for 1.4 unless this is just documentation catching up with the code

Edit

Note also that the later Django docs note a bug in jQuery 1.5 so ensure you are using 1.5.1 or later with the Django suggested code: https://docs.djangoproject.com/en/dev/ref/csrf/#ajax


回答 9

似乎没有人提到过使用X-CSRFTokenheader和来在纯JS中执行此操作的方法{{ csrf_token }},因此这是一个简单的解决方案,您无需搜索cookie或DOM:

var xhttp = new XMLHttpRequest();
xhttp.open("POST", url, true);
xhttp.setRequestHeader("X-CSRFToken", "{{ csrf_token }}");
xhttp.send();

It seems nobody has mentioned how to do this in pure JS using the X-CSRFToken header and {{ csrf_token }}, so here’s a simple solution where you don’t need to search through the cookies or the DOM:

var xhttp = new XMLHttpRequest();
xhttp.open("POST", url, true);
xhttp.setRequestHeader("X-CSRFToken", "{{ csrf_token }}");
xhttp.send();

回答 10

由于当前答案中没有任何地方说明,因此如果您不将 js 嵌入模板中,最快的解决方案是:

<script type="text/javascript"> window.CSRF_TOKEN = "{{ csrf_token }}"; </script>你的模板里面引用之前的script.js文件,然后添加csrfmiddlewaretoken到您的data字典在你的js文件:

$.ajax({
            type: 'POST',
            url: somepathname + "do_it/",
            data: {csrfmiddlewaretoken: window.CSRF_TOKEN},
            success: function() {
                console.log("Success!");
            }
        })

As it is not stated anywhere in the current answers, the fastest solution if you are not embedding js into your template is:

Put <script type="text/javascript"> window.CSRF_TOKEN = "{{ csrf_token }}"; </script> before your reference to script.js file in your template, then add csrfmiddlewaretoken into your data dictionary in your js file:

$.ajax({
            type: 'POST',
            url: somepathname + "do_it/",
            data: {csrfmiddlewaretoken: window.CSRF_TOKEN},
            success: function() {
                console.log("Success!");
            }
        })

回答 11

我刚刚遇到了一些不同但相似的情况。不能100%确定是否可以解决您的问题,但是我通过设置POST参数’csrfmiddlewaretoken’并使用适当的cookie值字符串解决了Django 1.3的问题,该参数通常由Django的home HTML形式返回。带有'{%csrf_token%}’标签的模板系统。我没有尝试使用较旧的Django,只是在Django1.3上发生并解决了。我的问题是,通过Ajax从表单提交的第一个请求已成功完成,但从完全相同的第二次请求失败,导致了403状态,即使标头’X-CSRFToken’也已正确放置CSRF令牌值就像第一次尝试一样。希望这可以帮助。

问候,

I’ve just encountered a bit different but similar situation. Not 100% sure if it’d be a resolution to your case, but I resolved the issue for Django 1.3 by setting a POST parameter ‘csrfmiddlewaretoken’ with the proper cookie value string which is usually returned within the form of your home HTML by Django’s template system with ‘{% csrf_token %}’ tag. I did not try on the older Django, just happened and resolved on Django1.3. My problem was that the first request submitted via Ajax from a form was successfully done but the second attempt from the exact same from failed, resulted in 403 state even though the header ‘X-CSRFToken’ is correctly placed with the CSRF token value as well as in the case of the first attempt. Hope this helps.

Regards,

Hiro


回答 12

您可以将此js粘贴到您的html文件中,请记住将其放在其他js函数之前

<script>
  // using jQuery
  function getCookie(name) {
    var cookieValue = null;
    if (document.cookie && document.cookie != '') {
      var cookies = document.cookie.split(';');
      for (var i = 0; i < cookies.length; i++) {
        var cookie = jQuery.trim(cookies[i]);
        // Does this cookie string begin with the name we want?
        if (cookie.substring(0, name.length + 1) == (name + '=')) {
          cookieValue = decodeURIComponent(cookie.substring(name.length + 1));
          break;
        }
      }
    }
    return cookieValue;
  }

  function csrfSafeMethod(method) {
    // these HTTP methods do not require CSRF protection
    return (/^(GET|HEAD|OPTIONS|TRACE)$/.test(method));
  }

  $(document).ready(function() {
    var csrftoken = getCookie('csrftoken');
    $.ajaxSetup({
      beforeSend: function(xhr, settings) {
        if (!csrfSafeMethod(settings.type) && !this.crossDomain) {
          xhr.setRequestHeader("X-CSRFToken", csrftoken);
        }
      }
    });
  });
</script>

You can paste this js into your html file, remember put it before other js function

<script>
  // using jQuery
  function getCookie(name) {
    var cookieValue = null;
    if (document.cookie && document.cookie != '') {
      var cookies = document.cookie.split(';');
      for (var i = 0; i < cookies.length; i++) {
        var cookie = jQuery.trim(cookies[i]);
        // Does this cookie string begin with the name we want?
        if (cookie.substring(0, name.length + 1) == (name + '=')) {
          cookieValue = decodeURIComponent(cookie.substring(name.length + 1));
          break;
        }
      }
    }
    return cookieValue;
  }

  function csrfSafeMethod(method) {
    // these HTTP methods do not require CSRF protection
    return (/^(GET|HEAD|OPTIONS|TRACE)$/.test(method));
  }

  $(document).ready(function() {
    var csrftoken = getCookie('csrftoken');
    $.ajaxSetup({
      beforeSend: function(xhr, settings) {
        if (!csrfSafeMethod(settings.type) && !this.crossDomain) {
          xhr.setRequestHeader("X-CSRFToken", csrftoken);
        }
      }
    });
  });
</script>

回答 13

每个会话(即每次登录)都分配一个CSRF令牌。因此,在希望获取用户输入的某些数据并将其作为ajax调用发送给受csrf_protect装饰器保护的某个函数之前,请先尝试查找正在被调用的函数,然后再从用户获取此数据。例如,必须渲染一些模板,以便用户在其上输入数据。该模板由某些功能渲染。在此函数中,您可以按以下方式获取csrf令牌:csrf = request.COOKIES [‘csrftoken’]现在在上下文字典中传递此csrf值,针对该字典呈现相关模板。现在在该模板中编写以下行:现在,在您的javascript函数中,在发出ajax请求之前,请编写:var csrf = $(’#csrf’)。val()这将选择传递给模板的令牌的值并将其存储在变量csrf中。现在,在进行ajax调用时,在您的帖子数据中,还要传递此值:“ csrfmiddlewaretoken”:csrf

即使您未实现Django表单,这也将起作用。

实际上,这里的逻辑是:您需要可以从请求中获取的令牌。因此,您只需要确定登录后立即调用的函数即可。一旦有了此令牌,就可以再次调用ajax来获取它,或者将其传递给可被ajax访问的模板。

One CSRF token is assigned to every session ( i.e. every time you log in). So before you wish to get some data entered by user and send that as ajax call to some function which is protected by csrf_protect decorator, try to find the functions that are being called before you are getting this data from user. E.g. some template must be being rendered on which your user is entering data. That template is being rendered by some function. In this function you can get csrf token as follows: csrf = request.COOKIES[‘csrftoken’] Now pass this csrf value in context dictionary against which template in question is being rendered. Now in that template write this line: Now in your javascript function, before making ajax request, write this: var csrf = $(‘#csrf’).val() this will pick value of token passed to template and store it in variable csrf. Now while making ajax call, in your post data, pass this value as well : “csrfmiddlewaretoken”: csrf

This will work even if you are not implementing django forms.

In fact, logic over here is : You need token which you can get from request. So you just need to figure out the function being called immediately after log in. Once you have this token, either make another ajax call to get it or pass it to some template which is accessible by your ajax.


回答 14

对于遇到此问题并尝试调试的人:

1)django csrf检查(假设您要发送一个)是 这里

2)在我的情况下,将settings.CSRF_HEADER_NAME其设置为“ HTTP_X_CSRFTOKEN”,而我的AJAX调用正在发送名为“ HTTP_X_CSRF_TOKEN”的标头,因此无法正常工作。我可以在AJAX调用或Django设置中更改它。

3)如果您选择在服务器端进行更改,请找到django的安装位置,并csrf middleware在使用的.f中放置一个断点virtualenv,它将类似于:~/.envs/my-project/lib/python2.7/site-packages/django/middleware/csrf.py

import ipdb; ipdb.set_trace() # breakpoint!!
if request_csrf_token == "":
    # Fall back to X-CSRFToken, to make things easier for AJAX,
    # and possible for PUT/DELETE.
    request_csrf_token = request.META.get(settings.CSRF_HEADER_NAME, '')

然后,确保csrf令牌是从request.META正确获取的。

4)如果您需要更改标题等,请在设置文件中更改该变量

for someone who comes across this and is trying to debug:

1) the django csrf check (assuming you’re sending one) is here

2) In my case, settings.CSRF_HEADER_NAME was set to ‘HTTP_X_CSRFTOKEN’ and my AJAX call was sending a header named ‘HTTP_X_CSRF_TOKEN’ so stuff wasn’t working. I could either change it in the AJAX call, or django setting.

3) If you opt to change it server-side, find your install location of django and throw a breakpoint in the csrf middleware.f you’re using virtualenv, it’ll be something like: ~/.envs/my-project/lib/python2.7/site-packages/django/middleware/csrf.py

import ipdb; ipdb.set_trace() # breakpoint!!
if request_csrf_token == "":
    # Fall back to X-CSRFToken, to make things easier for AJAX,
    # and possible for PUT/DELETE.
    request_csrf_token = request.META.get(settings.CSRF_HEADER_NAME, '')

Then, make sure the csrf token is correctly sourced from request.META

4) If you need to change your header, etc – change that variable in your settings file


回答 15

如果有人正在努力使用axios进行这项工作,那么这对我有帮助:

import axios from 'axios';

axios.defaults.xsrfCookieName = 'csrftoken'
axios.defaults.xsrfHeaderName = 'X-CSRFToken'

资料来源:https : //cbuelter.wordpress.com/2017/04/10/django-csrf-with-axios/

If someone is strugling with axios to make this work this helped me:

import axios from 'axios';

axios.defaults.xsrfCookieName = 'csrftoken'
axios.defaults.xsrfHeaderName = 'X-CSRFToken'

Source: https://cbuelter.wordpress.com/2017/04/10/django-csrf-with-axios/


回答 16

在我的情况下,问题出在我将nginx配置从主服务器复制到一个临时的服务器,并禁用了第二个服务器上不需要的https。

我必须在配置中注释掉这两行以使其再次起作用:

# uwsgi_param             UWSGI_SCHEME    https;
# uwsgi_pass_header       X_FORWARDED_PROTO;

In my case the problem was with the nginx config that I’ve copied from main server to a temporary one with disabling https that is not needed on the second one in the process.

I had to comment out these two lines in the config to make it work again:

# uwsgi_param             UWSGI_SCHEME    https;
# uwsgi_pass_header       X_FORWARDED_PROTO;

回答 17

这是Django提供的较为简单的解决方案:

<script type="text/javascript">
// using jQuery
var csrftoken = jQuery("[name=csrfmiddlewaretoken]").val();

function csrfSafeMethod(method) {
    // these HTTP methods do not require CSRF protection
    return (/^(GET|HEAD|OPTIONS|TRACE)$/.test(method));
}
// set csrf header
$.ajaxSetup({
    beforeSend: function(xhr, settings) {
        if (!csrfSafeMethod(settings.type) && !this.crossDomain) {
            xhr.setRequestHeader("X-CSRFToken", csrftoken);
        }
    }
});

// Ajax call here
$.ajax({
    url:"{% url 'members:saveAccount' %}",
    data: fd,
    processData: false,
    contentType: false,
    type: 'POST',
    success: function(data) {
        alert(data);
        }
    });
</script>

来源:https : //docs.djangoproject.com/en/1.11/ref/csrf/

Here’s a less verbose solution provided by Django:

<script type="text/javascript">
// using jQuery
var csrftoken = jQuery("[name=csrfmiddlewaretoken]").val();

function csrfSafeMethod(method) {
    // these HTTP methods do not require CSRF protection
    return (/^(GET|HEAD|OPTIONS|TRACE)$/.test(method));
}
// set csrf header
$.ajaxSetup({
    beforeSend: function(xhr, settings) {
        if (!csrfSafeMethod(settings.type) && !this.crossDomain) {
            xhr.setRequestHeader("X-CSRFToken", csrftoken);
        }
    }
});

// Ajax call here
$.ajax({
    url:"{% url 'members:saveAccount' %}",
    data: fd,
    processData: false,
    contentType: false,
    type: 'POST',
    success: function(data) {
        alert(data);
        }
    });
</script>

Source: https://docs.djangoproject.com/en/1.11/ref/csrf/


如何将Ajax与Django应用程序集成?

问题:如何将Ajax与Django应用程序集成?

我是Django的新手,而Ajax的新手。我正在一个需要将两者整合的项目中。我相信我理解它们背后的原理,但是并没有找到对两者的良好解释。

有人可以给我一个简短的解释,说明如何将两者集成在一起才能更改代码库?

例如,我仍然可以将HttpResponseAjax与一起使用,还是必须随着Ajax的使用而改变我的回答?如果是这样,请您举例说明如何更改请求的响应?如果有什么不同,我返回的数据是JSON。

I am new to Django and pretty new to Ajax. I am working on a project where I need to integrate the two. I believe that I understand the principles behind them both, but have not found a good explanation of the two together.

Could someone give me a quick explanation of how the codebase must change with the two of them integrating together?

For example, can I still use the HttpResponse with Ajax, or do my responses have to change with the use of Ajax? If so, could you please provide an example of how the responses to the requests must change? If it makes any difference, the data I am returning is JSON.


回答 0

即使这不完全符合SO精神,但我还是很喜欢这个问题,因为刚开始时我遇到了同样的麻烦,因此我将为您提供快速指南。显然,您不了解其背后的原理(不要将其视为冒犯,但如果您这样做,您将不会提出疑问)。

Django是服务器端的。这就意味着,比如说客户访问一个URL,您内部就有一个函数views,该函数可以呈现他所看到的内容并以HTML返回响应。让我们将其分解为示例:

views.py:

def hello(request):
    return HttpResponse('Hello World!')

def home(request):
    return render_to_response('index.html', {'variable': 'world'})

index.html:

<h1>Hello {{ variable }}, welcome to my awesome site</h1>

urls.py:

url(r'^hello/', 'myapp.views.hello'),
url(r'^home/', 'myapp.views.home'),

这是最简单的用法示例。转到127.0.0.1:8000/hello表示对hello()函数的请求,转到127.0.0.1:8000/home将返回index.html并按要求替换所有变量(您现在可能已经知道了这一切)。

现在让我们谈谈AJAX。AJAX调用是执行异步请求的客户端代码。这听起来很复杂,但这仅意味着它会在后台为您提出请求,然后处理响应。因此,当您对某个URL进行AJAX调用时,您将获得与访问该位置的用户相同的数据。

例如,对的AJAX调用127.0.0.1:8000/hello将返回与您访问它相同的东西。只有这一次,您才将其包含在JavaScript函数中,并且可以根据需要进行处理。让我们看一个简单的用例:

$.ajax({
    url: '127.0.0.1:8000/hello',
    type: 'get', // This is the default though, you don't actually need to always mention it
    success: function(data) {
        alert(data);
    },
    failure: function(data) { 
        alert('Got an error dude');
    }
}); 

一般过程是这样的:

  1. 127.0.0.1:8000/hello就像您打开一个新标签页并自己完成一样,呼叫将转到URL 。
  2. 如果成功(状态码200),请执行成功功能,这将提醒收到的数据。
  3. 如果失败,请执行其他功能。

现在在这里会发生什么?您会收到有关“ hello world”的警报。如果您拨打AJAX到家里怎么办?同样,您会收到警告,说明<h1>Hello world, welcome to my awesome site</h1>

换句话说-AJAX调用没有新内容。它们只是让您在不离开页面的情况下让用户获取数据和信息的一种方法,它使您的网站设计流畅而整洁。您应该注意一些准则:

  1. 学习jQuery。我不能太强调这一点。您将需要稍微了解一下才能知道如何处理收到的数据。您还需要了解一些基本的JavaScript语法(与python相距不远,您将习惯它)。我强烈推荐Envato的jQuery视频教程,它们很棒,将使您走上正确的道路。
  2. 何时使用JSON?。您将看到很多示例,其中Django视图发送的数据使用JSON。我没有进入细节上,因为它不是重要的是如何做到这一点(有很多的解释,比比皆是)和很多更重要的时候。答案是-JSON数据是序列化数据。也就是说,您可以操纵的数据。就像我提到的,AJAX调用将获取响应,就好像用户自己做了一样。现在说您不想弄乱所有html,而是想发送数据(也许是对象列表)。JSON对此很有好处,因为它作为对象发送(JSON数据看起来像python字典),然后您可以对其进行迭代或执行其他操作,从而无需筛选无用的html。
  3. 最后添加。当您构建Web应用程序并想要实现AJAX时-帮个忙。首先,构建完全没有任何AJAX的应用程序。看到一切正常。然后,直到那时,才开始编写AJAX调用。这是一个很好的过程,可以帮助您学习很多东西。
  4. 使用chrome的开发人员工具。由于AJAX调用是在后台完成的,因此有时很难调试它们。您应该使用chrome开发人员工具(或类似的工具,例如firebug)和console.log要调试的东西。我不会详细解释,只是在Google周围寻找它。这对您会非常有帮助。
  5. CSRF意识。最后,请记住Django中的发布请求需要csrf_token。通过AJAX调用,很多时候您想发送数据而不刷新页面。您可能会遇到一些麻烦,最后才想起那个-等待,您忘记了发送csrf_token。这是AJAX-Django集成中的一个已知的初学者障碍,但是当您学习了如何使其变得更好时,它就像馅饼一样容易。

这就是我想到的一切。这是一个广泛的主题,但是,可能没有足够的例子。只要按照自己的方式做,就慢慢地,最终会得到它。

Even though this isn’t entirely in the SO spirit, I love this question, because I had the same trouble when I started, so I’ll give you a quick guide. Obviously you don’t understand the principles behind them (don’t take it as an offense, but if you did you wouldn’t be asking).

Django is server-side. It means, say a client goes to a URL, you have a function inside views that renders what he sees and returns a response in HTML. Let’s break it up into examples:

views.py:

def hello(request):
    return HttpResponse('Hello World!')

def home(request):
    return render_to_response('index.html', {'variable': 'world'})

index.html:

<h1>Hello {{ variable }}, welcome to my awesome site</h1>

urls.py:

url(r'^hello/', 'myapp.views.hello'),
url(r'^home/', 'myapp.views.home'),

That’s an example of the simplest of usages. Going to 127.0.0.1:8000/hello means a request to the hello() function, going to 127.0.0.1:8000/home will return the index.html and replace all the variables as asked (you probably know all this by now).

Now let’s talk about AJAX. AJAX calls are client-side code that does asynchronous requests. That sounds complicated, but it simply means it does a request for you in the background and then handles the response. So when you do an AJAX call for some URL, you get the same data you would get as a user going to that place.

For example, an AJAX call to 127.0.0.1:8000/hello will return the same thing it would as if you visited it. Only this time, you have it inside a JavaScript function and you can deal with it however you’d like. Let’s look at a simple use case:

$.ajax({
    url: '127.0.0.1:8000/hello',
    type: 'get', // This is the default though, you don't actually need to always mention it
    success: function(data) {
        alert(data);
    },
    failure: function(data) { 
        alert('Got an error dude');
    }
}); 

The general process is this:

  1. The call goes to the URL 127.0.0.1:8000/hello as if you opened a new tab and did it yourself.
  2. If it succeeds (status code 200), do the function for success, which will alert the data received.
  3. If fails, do a different function.

Now what would happen here? You would get an alert with ‘hello world’ in it. What happens if you do an AJAX call to home? Same thing, you’ll get an alert stating <h1>Hello world, welcome to my awesome site</h1>.

In other words – there’s nothing new about AJAX calls. They are just a way for you to let the user get data and information without leaving the page, and it makes for a smooth and very neat design of your website. A few guidelines you should take note of:

  1. Learn jQuery. I cannot stress this enough. You’re gonna have to understand it a little to know how to handle the data you receive. You’ll also need to understand some basic JavaScript syntax (not far from python, you’ll get used to it). I strongly recommend Envato’s video tutorials for jQuery, they are great and will put you on the right path.
  2. When to use JSON?. You’re going to see a lot of examples where the data sent by the Django views is in JSON. I didn’t go into detail on that, because it isn’t important how to do it (there are plenty of explanations abound) and a lot more important when. And the answer to that is – JSON data is serialized data. That is, data you can manipulate. Like I mentioned, an AJAX call will fetch the response as if the user did it himself. Now say you don’t want to mess with all the html, and instead want to send data (a list of objects perhaps). JSON is good for this, because it sends it as an object (JSON data looks like a python dictionary), and then you can iterate over it or do something else that removes the need to sift through useless html.
  3. Add it last. When you build a web app and want to implement AJAX – do yourself a favor. First, build the entire app completely devoid of any AJAX. See that everything is working. Then, and only then, start writing the AJAX calls. That’s a good process that helps you learn a lot as well.
  4. Use chrome’s developer tools. Since AJAX calls are done in the background it’s sometimes very hard to debug them. You should use the chrome developer tools (or similar tools such as firebug) and console.log things to debug. I won’t explain in detail, just google around and find out about it. It would be very helpful to you.
  5. CSRF awareness. Finally, remember that post requests in Django require the csrf_token. With AJAX calls, a lot of times you’d like to send data without refreshing the page. You’ll probably face some trouble before you’d finally remember that – wait, you forgot to send the csrf_token. This is a known beginner roadblock in AJAX-Django integration, but after you learn how to make it play nice, it’s easy as pie.

That’s everything that comes to my head. It’s a vast subject, but yeah, there’s probably not enough examples out there. Just work your way there, slowly, you’ll get it eventually.


回答 1

除了yuvi的出色答案之外,我想添加一个有关如何在Django中处理此问题的小示例(除了将要使用的所有js)。该示例使用AjaxableResponseMixin并假设一个Author模型。

import json

from django.http import HttpResponse
from django.views.generic.edit import CreateView
from myapp.models import Author

class AjaxableResponseMixin(object):
    """
    Mixin to add AJAX support to a form.
    Must be used with an object-based FormView (e.g. CreateView)
    """
    def render_to_json_response(self, context, **response_kwargs):
        data = json.dumps(context)
        response_kwargs['content_type'] = 'application/json'
        return HttpResponse(data, **response_kwargs)

    def form_invalid(self, form):
        response = super(AjaxableResponseMixin, self).form_invalid(form)
        if self.request.is_ajax():
            return self.render_to_json_response(form.errors, status=400)
        else:
            return response

    def form_valid(self, form):
        # We make sure to call the parent's form_valid() method because
        # it might do some processing (in the case of CreateView, it will
        # call form.save() for example).
        response = super(AjaxableResponseMixin, self).form_valid(form)
        if self.request.is_ajax():
            data = {
                'pk': self.object.pk,
            }
            return self.render_to_json_response(data)
        else:
            return response

class AuthorCreate(AjaxableResponseMixin, CreateView):
    model = Author
    fields = ['name']

来源:Django文档,具有基于类的视图的表单处理

Django 1.6版的链接不再可用,已更新至1.11版

Further from yuvi’s excellent answer, I would like to add a small specific example on how to deal with this within Django (beyond any js that will be used). The example uses AjaxableResponseMixin and assumes an Author model.

import json

from django.http import HttpResponse
from django.views.generic.edit import CreateView
from myapp.models import Author

class AjaxableResponseMixin(object):
    """
    Mixin to add AJAX support to a form.
    Must be used with an object-based FormView (e.g. CreateView)
    """
    def render_to_json_response(self, context, **response_kwargs):
        data = json.dumps(context)
        response_kwargs['content_type'] = 'application/json'
        return HttpResponse(data, **response_kwargs)

    def form_invalid(self, form):
        response = super(AjaxableResponseMixin, self).form_invalid(form)
        if self.request.is_ajax():
            return self.render_to_json_response(form.errors, status=400)
        else:
            return response

    def form_valid(self, form):
        # We make sure to call the parent's form_valid() method because
        # it might do some processing (in the case of CreateView, it will
        # call form.save() for example).
        response = super(AjaxableResponseMixin, self).form_valid(form)
        if self.request.is_ajax():
            data = {
                'pk': self.object.pk,
            }
            return self.render_to_json_response(data)
        else:
            return response

class AuthorCreate(AjaxableResponseMixin, CreateView):
    model = Author
    fields = ['name']

Source: Django documentation, Form handling with class-based views

The link to version 1.6 of Django is no longer available updated to version 1.11


回答 2

我写这封信是因为可接受的答案很旧,需要复习。

所以这就是我在2019年将Ajax与Django集成的方式:)让我们举一个实际的例子说明何时需要Ajax:-

可以说我有一个带有注册用户名的模型,并借助于Ajax我想知道给定的用户名是否存在。

的HTML:

<p id="response_msg"></p> 
<form id="username_exists_form" method='GET'>
      Name: <input type="username" name="username" />
      <button type='submit'> Check </button>           
</form>   

阿贾克斯:

$('#username_exists_form').on('submit',function(e){
    e.preventDefault();
    var username = $(this).find('input').val();
    $.get('/exists/',
          {'username': username},   
          function(response){ $('#response_msg').text(response.msg); }
    );
}); 

urls.py:

from django.contrib import admin
from django.urls import path
from . import views

urlpatterns = [
    path('admin/', admin.site.urls),
    path('exists/', views.username_exists, name='exists'),
]

views.py:

def username_exists(request):
    data = {'msg':''}   
    if request.method == 'GET':
        username = request.GET.get('username').lower()
        exists = Usernames.objects.filter(name=username).exists()
        if exists:
            data['msg'] = username + ' already exists.'
        else:
            data['msg'] = username + ' does not exists.'
    return JsonResponse(data)

同样不推荐使用的render_to_response已被render取代,从Django 1.7起,而不是HttpResponse,我们使用JsonResponse进行Ajax响应。因为它带有JSON编码器,所以您不需要在返回响应对象之前对数据进行序列化,但HttpResponse不建议使用。

I am writing this because the accepted answer is pretty old, it needs a refresher.

So this is how I would integrate Ajax with Django in 2019 :) And lets take a real example of when we would need Ajax :-

Lets say I have a model with registered usernames and with the help of Ajax I wanna know if a given username exists.

html:

<p id="response_msg"></p> 
<form id="username_exists_form" method='GET'>
      Name: <input type="username" name="username" />
      <button type='submit'> Check </button>           
</form>   

ajax:

$('#username_exists_form').on('submit',function(e){
    e.preventDefault();
    var username = $(this).find('input').val();
    $.get('/exists/',
          {'username': username},   
          function(response){ $('#response_msg').text(response.msg); }
    );
}); 

urls.py:

from django.contrib import admin
from django.urls import path
from . import views

urlpatterns = [
    path('admin/', admin.site.urls),
    path('exists/', views.username_exists, name='exists'),
]

views.py:

def username_exists(request):
    data = {'msg':''}   
    if request.method == 'GET':
        username = request.GET.get('username').lower()
        exists = Usernames.objects.filter(name=username).exists()
        if exists:
            data['msg'] = username + ' already exists.'
        else:
            data['msg'] = username + ' does not exists.'
    return JsonResponse(data)

Also render_to_response which is deprecated and has been replaced by render and from Django 1.7 onwards instead of HttpResponse we use JsonResponse for ajax response. Because it comes with a JSON encoder, so you don’t need to serialize the data before returning the response object but HttpResponse is not deprecated.


回答 3

简单而漂亮。您无需更改视图。Bjax处理所有链接。看看这个: Bjax

用法:

<script src="bjax.min.js" type="text/javascript"></script>
<link href="bjax.min.css" rel="stylesheet" type="text/css" />

最后,将其包含在html的HEAD中:

$('a').bjax();

有关更多设置,请在此处签出演示: Bjax演示

Simple and Nice. You don’t have to change your views. Bjax handles all your links. Check this out: Bjax

Usage:

<script src="bjax.min.js" type="text/javascript"></script>
<link href="bjax.min.css" rel="stylesheet" type="text/css" />

Finally, include this in the HEAD of your html:

$('a').bjax();

For more settings, checkout demo here: Bjax Demo


回答 4

AJAX是执行异步任务的最佳方法。在任何网站建设中,进行异步调用都是很常见的事情。我们将以一个简短的例子来学习如何在Django中实现AJAX。我们需要使用jQuery,以便减少JavaScript。

这是Contact示例,这是最简单的示例,我用它来解释AJAX的基本知识及其在Django中的实现。在此示例中,我们将发出POST请求。我正在关注本文的示例之一:https : //djangopy.org/learn/step-up-guide-to-implement-ajax-in-django

models.py

首先,创建具有基本细节的Contact模型。

from django.db import models

class Contact(models.Model):
    name = models.CharField(max_length = 100)
    email = models.EmailField()
    message = models.TextField()
    timestamp = models.DateTimeField(auto_now_add = True)

    def __str__(self):
        return self.name

表格

创建上述模型的表单。

from django import forms
from .models import Contact

class ContactForm(forms.ModelForm):
    class Meta:
        model = Contact
        exclude = ["timestamp", ]

views.py

这些视图看起来类似于基于功能的基本创建视图,但是我们不使用render来返回,而是使用JsonResponse响应。

from django.http import JsonResponse
from .forms import ContactForm

def postContact(request):
    if request.method == "POST" and request.is_ajax():
        form = ContactForm(request.POST)
        form.save()
        return JsonResponse({"success":True}, status=200)
    return JsonResponse({"success":False}, status=400)

urls.py

让我们创建以上视图的路线。

from django.contrib import admin
from django.urls import path
from app_1 import views as app1

urlpatterns = [
    path('ajax/contact', app1.postContact, name ='contact_submit'),
]

模板

移至前端部分,呈现在封闭表单标签上方创建的表单以及csrf_token和Submit按钮。请注意,我们已经包含了jquery库。

<form id = "contactForm" method= "POST">{% csrf_token %}
   {{ contactForm.as_p }}
  <input type="submit" name="contact-submit" class="btn btn-primary" />
</form>

<script src="https://cdnjs.cloudflare.com/ajax/libs/jquery/3.3.1/jquery.min.js"></script>

Java脚本

现在让我们谈谈javascript部分,在表单提交中,我们发出POST类型的ajax请求,获取表单数据并发送到服务器端。

$("#contactForm").submit(function(e){
    // prevent from normal form behaviour
        e.preventDefault();
        // serialize the form data  
        var serializedData = $(this).serialize();
        $.ajax({
            type : 'POST',
            url :  "{% url 'contact_submit' %}",
            data : serializedData,
            success : function(response){
            //reset the form after successful submit
                $("#contactForm")[0].reset(); 
            },
            error : function(response){
                console.log(response)
            }
        });
   });

这只是使用Django进行AJAX入门的基本示例,如果您想了解更多示例,可以阅读本文:https : //djangopy.org/learn/step-up-guide-to-在Django中实现ajax

AJAX is the best way to do asynchronous tasks. Making asynchronous calls is something common in use in any website building. We will take a short example to learn how we can implement AJAX in Django. We need to use jQuery so as to write less javascript.

This is Contact example, which is the simplest example, I am using to explain the basics of AJAX and its implementation in Django. We will be making POST request in this example. I am following one of the example of this post: https://djangopy.org/learn/step-up-guide-to-implement-ajax-in-django

models.py

Let’s first create the model of Contact, having basic details.

from django.db import models

class Contact(models.Model):
    name = models.CharField(max_length = 100)
    email = models.EmailField()
    message = models.TextField()
    timestamp = models.DateTimeField(auto_now_add = True)

    def __str__(self):
        return self.name

forms.py

Create the form for the above model.

from django import forms
from .models import Contact

class ContactForm(forms.ModelForm):
    class Meta:
        model = Contact
        exclude = ["timestamp", ]

views.py

The views look similar to the basic function-based create view, but instead of returning with render, we are using JsonResponse response.

from django.http import JsonResponse
from .forms import ContactForm

def postContact(request):
    if request.method == "POST" and request.is_ajax():
        form = ContactForm(request.POST)
        form.save()
        return JsonResponse({"success":True}, status=200)
    return JsonResponse({"success":False}, status=400)

urls.py

Let’s create the route of the above view.

from django.contrib import admin
from django.urls import path
from app_1 import views as app1

urlpatterns = [
    path('ajax/contact', app1.postContact, name ='contact_submit'),
]

template

Moving to frontend section, render the form which was created above enclosing form tag along with csrf_token and submit button. Note that we have included the jquery library.

<form id = "contactForm" method= "POST">{% csrf_token %}
   {{ contactForm.as_p }}
  <input type="submit" name="contact-submit" class="btn btn-primary" />
</form>

<script src="https://cdnjs.cloudflare.com/ajax/libs/jquery/3.3.1/jquery.min.js"></script>

Javascript

Let’s now talk about javascript part, on the form submit we are making ajax request of type POST, taking the form data and sending to the server side.

$("#contactForm").submit(function(e){
    // prevent from normal form behaviour
        e.preventDefault();
        // serialize the form data  
        var serializedData = $(this).serialize();
        $.ajax({
            type : 'POST',
            url :  "{% url 'contact_submit' %}",
            data : serializedData,
            success : function(response){
            //reset the form after successful submit
                $("#contactForm")[0].reset(); 
            },
            error : function(response){
                console.log(response)
            }
        });
   });

This is just a basic example to get started with AJAX with django, if you want to get dive with several more examples, you can go through this article: https://djangopy.org/learn/step-up-guide-to-implement-ajax-in-django


回答 5

我尝试在项目中使用AjaxableResponseMixin,但最终出现以下错误消息:

配置不正确:没有要重定向到的URL。在模型上提供一个URL或定义一个get_absolute_url方法。

这是因为当您向浏览器发送JSON请求时,CreateView将返回重定向响应,而不是返回HttpResponse。因此,我对进行了一些更改AjaxableResponseMixin。如果该请求是ajax请求,则不会调用该super.form_valid方法,而只是form.save()直接调用。

from django.http import JsonResponse
from django import forms
from django.db import models

class AjaxableResponseMixin(object):
    success_return_code = 1
    error_return_code = 0
    """
    Mixin to add AJAX support to a form.
    Must be used with an object-based FormView (e.g. CreateView)
    """
    def form_invalid(self, form):
        response = super(AjaxableResponseMixin, self).form_invalid(form)
        if self.request.is_ajax():
            form.errors.update({'result': self.error_return_code})
            return JsonResponse(form.errors, status=400)
        else:
            return response

    def form_valid(self, form):
        # We make sure to call the parent's form_valid() method because
        # it might do some processing (in the case of CreateView, it will
        # call form.save() for example).
        if self.request.is_ajax():
            self.object = form.save()
            data = {
                'result': self.success_return_code
            }
            return JsonResponse(data)
        else:
            response = super(AjaxableResponseMixin, self).form_valid(form)
            return response

class Product(models.Model):
    name = models.CharField('product name', max_length=255)

class ProductAddForm(forms.ModelForm):
    '''
    Product add form
    '''
    class Meta:
        model = Product
        exclude = ['id']


class PriceUnitAddView(AjaxableResponseMixin, CreateView):
    '''
    Product add view
    '''
    model = Product
    form_class = ProductAddForm

I have tried to use AjaxableResponseMixin in my project, but had ended up with the following error message:

ImproperlyConfigured: No URL to redirect to. Either provide a url or define a get_absolute_url method on the Model.

That is because the CreateView will return a redirect response instead of returning a HttpResponse when you to send JSON request to the browser. So I have made some changes to the AjaxableResponseMixin. If the request is an ajax request, it will not call the super.form_valid method, just call the form.save() directly.

from django.http import JsonResponse
from django import forms
from django.db import models

class AjaxableResponseMixin(object):
    success_return_code = 1
    error_return_code = 0
    """
    Mixin to add AJAX support to a form.
    Must be used with an object-based FormView (e.g. CreateView)
    """
    def form_invalid(self, form):
        response = super(AjaxableResponseMixin, self).form_invalid(form)
        if self.request.is_ajax():
            form.errors.update({'result': self.error_return_code})
            return JsonResponse(form.errors, status=400)
        else:
            return response

    def form_valid(self, form):
        # We make sure to call the parent's form_valid() method because
        # it might do some processing (in the case of CreateView, it will
        # call form.save() for example).
        if self.request.is_ajax():
            self.object = form.save()
            data = {
                'result': self.success_return_code
            }
            return JsonResponse(data)
        else:
            response = super(AjaxableResponseMixin, self).form_valid(form)
            return response

class Product(models.Model):
    name = models.CharField('product name', max_length=255)

class ProductAddForm(forms.ModelForm):
    '''
    Product add form
    '''
    class Meta:
        model = Product
        exclude = ['id']


class PriceUnitAddView(AjaxableResponseMixin, CreateView):
    '''
    Product add view
    '''
    model = Product
    form_class = ProductAddForm

回答 6

当我们使用Django时:

Server ===> Client(Browser)   
      Send a page

When you click button and send the form,
----------------------------
Server <=== Client(Browser)  
      Give data back. (data in form will be lost)
Server ===> Client(Browser)  
      Send a page after doing sth with these data
----------------------------

如果要保留旧数据,则可以不使用Ajax。(页面将刷新)

Server ===> Client(Browser)   
      Send a page
Server <=== Client(Browser)  
      Give data back. (data in form will be lost)
Server ===> Client(Browser)  
      1. Send a page after doing sth with data
      2. Insert data into form and make it like before. 
      After these thing, server will send a html page to client. It means that server do more work, however, the way to work is same.

或者您可以使用Ajax(页面不会刷新)

--------------------------
<Initialization> 
Server ===> Client(Browser) [from URL1]    
      Give a page                      
--------------------------  
<Communication>
Server <=== Client(Browser)     
      Give data struct back but not to refresh the page.
Server ===> Client(Browser) [from URL2] 
      Give a data struct(such as JSON)
---------------------------------

如果使用Ajax,则必须执行以下操作:

  1. 使用URL1初始化HTML页面(我们通常使用Django模板初始化页面)。然后服务器向客户端发送一个html页面。
  2. 使用Ajax通过URL2与服务器通信。然后服务器向客户端发送数据结构。

Django与Ajax不同。原因如下:

  • 返回给客户的东西是不同的。Django的情况是HTML页面。Ajax的情况是数据结构。 
  • Django擅长创建某些东西,但是只能创建一次,不能更改任何东西。Django就像动漫,由许多图片组成。相比之下,Ajax并不擅长于在现有的html页面中创建某事,但擅长于更改某事。

我认为,如果您想在任何地方使用ajax。当您首先需要使用数据初始化页面时,可以将Django与Ajax结合使用。但是在某些情况下,您只需要一个静态页面,而服务器上没有任何内容,则不需要使用Django模板。

如果您不认为Ajax是最佳实践。您可以使用Django模板来完成所有操作,例如动漫。

(我英文不太好)

When we use Django:

Server ===> Client(Browser)   
      Send a page

When you click button and send the form,
----------------------------
Server <=== Client(Browser)  
      Give data back. (data in form will be lost)
Server ===> Client(Browser)  
      Send a page after doing sth with these data
----------------------------

If you want to keep old data, you can do it without Ajax. (Page will be refreshed)

Server ===> Client(Browser)   
      Send a page
Server <=== Client(Browser)  
      Give data back. (data in form will be lost)
Server ===> Client(Browser)  
      1. Send a page after doing sth with data
      2. Insert data into form and make it like before. 
      After these thing, server will send a html page to client. It means that server do more work, however, the way to work is same.

Or you can do with Ajax (Page will be not refreshed)

--------------------------
<Initialization> 
Server ===> Client(Browser) [from URL1]    
      Give a page                      
--------------------------  
<Communication>
Server <=== Client(Browser)     
      Give data struct back but not to refresh the page.
Server ===> Client(Browser) [from URL2] 
      Give a data struct(such as JSON)
---------------------------------

If you use Ajax, you must do these:

  1. Initial a HTML page using URL1 (we usually initial page by Django template). And then server send client a html page.
  2. Use Ajax to communicate with server using URL2. And then server send client a data struct.

Django is different from Ajax. The reason for this is as follows:

  • The thing return to client is different. The case of Django is HTML page. The case of Ajax is data struct. 
  • Django is good at creating something, but it only can create once, it cannot change anything. Django is like anime, consist of many picture. By contrast, Ajax is not good at creating sth but good at change sth in exist html page.

In my opinion, if you would like to use ajax everywhere. when you need to initial a page with data at first, you can use Django with Ajax. But in some case, you just need a static page without anything from server, you need not use Django template.

If you don’t think Ajax is the best practice. you can use Django template to do everything, like anime.

(My English is not good)