I have a RESTful API that I have exposed using an implementation of Elasticsearch on an EC2 instance to index a corpus of content. I can query the search by running the following from my terminal (MacOSX):
How do I turn above into a API request using python/requests or python/urllib2 (not sure which one to go for – have been using urllib2, but hear that requests is better…)? Do I pass as a header or otherwise?
Depending on what kind of response your API returns, you will then probably want to look at response.text or response.json() (or possibly inspect response.status_code first). See the quickstart docs here, especially this section.
#Python 2.7.6#RestfulClient.pyimport requests
from requests.auth importHTTPDigestAuthimport json
# Replace with the correct URL
url ="http://api_url"# It is a good practice not to hardcode the credentials. So ask the user to enter credentials at runtime
myResponse = requests.get(url,auth=HTTPDigestAuth(raw_input("username: "), raw_input("Password: ")), verify=True)#print (myResponse.status_code)# For successful API call, response code will be 200 (OK)if(myResponse.ok):# Loading the response data into a dict variable# json.loads takes in only binary or string variables so using content to fetch binary content# Loads (Load String) takes a Json file and converts into python data structure (dict or list, depending on JSON)
jData = json.loads(myResponse.content)print("The response contains {0} properties".format(len(jData)))print("\n")for key in jData:print key +" : "+ jData[key]else:# If response code is not ok (200), print the resulting http error code with description
myResponse.raise_for_status()
Assuming the API returns a JSON, parse the JSON object into a
Python dict using json.loads function
Loop through the dict to extract information.
Requests module provides you useful function to loop for success and failure.
if(Response.ok): will help help you determine if your API call is successful (Response code – 200)
Response.raise_for_status() will help you fetch the http code that is returned from the API.
Below is a sample code for making such API calls. Also can be found in github. The code assumes that the API makes use of digest authentication. You can either skip this or use other appropriate authentication modules to authenticate the client invoking the API.
#Python 2.7.6
#RestfulClient.py
import requests
from requests.auth import HTTPDigestAuth
import json
# Replace with the correct URL
url = "http://api_url"
# It is a good practice not to hardcode the credentials. So ask the user to enter credentials at runtime
myResponse = requests.get(url,auth=HTTPDigestAuth(raw_input("username: "), raw_input("Password: ")), verify=True)
#print (myResponse.status_code)
# For successful API call, response code will be 200 (OK)
if(myResponse.ok):
# Loading the response data into a dict variable
# json.loads takes in only binary or string variables so using content to fetch binary content
# Loads (Load String) takes a Json file and converts into python data structure (dict or list, depending on JSON)
jData = json.loads(myResponse.content)
print("The response contains {0} properties".format(len(jData)))
print("\n")
for key in jData:
print key + " : " + jData[key]
else:
# If response code is not ok (200), print the resulting http error code with description
myResponse.raise_for_status()
I’m using the python requests module to send a RESTful GET to a server, for which I get a response in JSON. The JSON response is basically just a list of lists.
What’s the best way to coerce the response to a native Python object so I can either iterate or print it out using pprint?
Is there a list somewhere of recommendations of different Python-based REST frameworks for use on the serverside to write your own RESTful APIs? Preferably with pros and cons.
Something to be careful about when designing a RESTful API is the conflation of GET and POST, as if they were the same thing. It’s easy to make this mistake with Django‘s function-based views and CherryPy‘s default dispatcher, although both frameworks now provide a way around this problem (class-based views and MethodDispatcher, respectively).
Some frameworks that get it right are web.py, Flask and Bottle. When combined with the mimerender library (full disclosure: I wrote it), they allow you to write nice RESTful webservices:
import web
import json
from mimerender import mimerender
render_xml = lambda message: '<message>%s</message>'%message
render_json = lambda **args: json.dumps(args)
render_html = lambda message: '<html><body>%s</body></html>'%message
render_txt = lambda message: message
urls = (
'/(.*)', 'greet'
)
app = web.application(urls, globals())
class greet:
@mimerender(
default = 'html',
html = render_html,
xml = render_xml,
json = render_json,
txt = render_txt
)
def GET(self, name):
if not name:
name = 'world'
return {'message': 'Hello, ' + name + '!'}
if __name__ == "__main__":
app.run()
The service’s logic is implemented only once, and the correct representation selection (Accept header) + dispatch to the proper render function (or template) is done in a tidy, transparent way.
Update (April 2012): added information about Django’s class-based views, CherryPy’s MethodDispatcher and Flask and Bottle frameworks. Neither existed back when the question was asked.
Note that — out of the box — Django did not have fine-grained enough authentication for our needs. We used the Django-REST interface, which helped a lot. [We’ve since rolled our own because we’d made so many extensions that it had become a maintenance nightmare.]
We have two kinds of URL’s: “html” URL’s which implement the human-oriented HTML pages, and “json” URL’s which implement the web-services oriented processing. Our view functions often look like this.
def someUsefulThing( request, object_id ):
# do some processing
return { a dictionary with results }
def htmlView( request, object_id ):
d = someUsefulThing( request, object_id )
render_to_response( 'template.html', d, ... )
def jsonView( request, object_id ):
d = someUsefulThing( request, object_id )
data = serializers.serialize( 'json', d['object'], fields=EXPOSED_FIELDS )
response = HttpResponse( data, status=200, content_type='application/json' )
response['Location']= reverse( 'some.path.to.this.view', kwargs={...} )
return response
The point being that the useful functionality is factored out of the two presentations. The JSON presentation is usually just one object that was requested. The HTML presentation often includes all kinds of navigation aids and other contextual clues that help people be productive.
The jsonView functions are all very similar, which can be a bit annoying. But it’s Python, so make them part of a callable class or write decorators if it helps.
This emphasizes what I really like about CherryPy; this is a completely working example that’s very understandable even to someone who doesn’t know the framework. If you run this code, then you can immediately see the results in your web browser; e.g. visiting http://localhost:8080/celc_to_fahr?degrees=50 will display 122.0 in your web browser.
I don’t see any reason to use Django just to expose a REST api, there are lighter and more flexible solutions. Django carries a lot of other things to the table, that are not always needed. For sure not needed if you only want to expose some code as a REST service.
My personal experience, fwiw, is that once you have a one-size-fits-all framework, you’ll start to use its ORM, its plugins, etc. just because it’s easy, and in no time you end up having a dependency that is very hard to get rid of.
Choosing a web framework is a tough decision, and I would avoid picking a full stack solution just to expose a REST api.
Now, if you really need/want to use Django, then Piston is a nice REST framework for django apps.
That being said, CherryPy looks really nice too, but seems more RPC than REST.
Looking at the samples (I never used it), probably web.py is the best and cleanest if you only need REST.
In particular it mentions a built in CherryPy dispatcher called MethodDispatcher, which invokes methods based on their HTTP-verb identifiers (GET, POST, etc…).
In 2010, the Pylons and repoze.bfg communities “joined forces” to create Pyramid, a web framework based most heavily on repoze.bfg. It retains the philosophies of its parent frameworks, and can be used for RESTful services. It’s worth a look.
Seems all kinds of python web frameworks can implement RESTful interfaces now.
For Django, besides tastypie and piston, django-rest-framework is a promising one worth to mention. I’ve already migrated one of my project on it smoothly.
Django REST framework is a lightweight REST framework for Django, that
aims to make it easy to build well-connected, self-describing RESTful
Web APIs.
Quick example:
from django.conf.urls.defaults import patterns, url
from djangorestframework.resources import ModelResource
from djangorestframework.views import ListOrCreateModelView, InstanceModelView
from myapp.models import MyModel
class MyResource(ModelResource):
model = MyModel
urlpatterns = patterns('',
url(r'^$', ListOrCreateModelView.as_view(resource=MyResource)),
url(r'^(?P<pk>[^/]+)/$', InstanceModelView.as_view(resource=MyResource)),
)
Take the example from official site, all above codes provide api, self explained document(like soap based webservice) and even sandbox to test a bit. Very convenience.
web2py includes support for easily building RESTful API’s, described here and here (video). In particular, look at parse_as_rest, which lets you define URL patterns that map request args to database queries; and smart_query, which enables you to pass arbitrary natural language queries in the URL.
I you are using Django then you can consider django-tastypie as an alternative to django-piston. It is easier to tune to non-ORM data sources than piston, and has great documentation.
fromdjango.urlsimportpath, includefromdjango.contrib.auth.modelsimportUserfromrest_frameworkimportserializers, viewsets, routers# Serializers define the API representation.classUserSerializer(serializers.HyperlinkedModelSerializer):
classMeta:
model=Userfields= ['url', 'username', 'email', 'is_staff']
# ViewSets define the view behavior.classUserViewSet(viewsets.ModelViewSet):
queryset=User.objects.all()
serializer_class=UserSerializer# Routers provide a way of automatically determining the URL conf.router=routers.DefaultRouter()
router.register(r'users', UserViewSet)
# Wire up our API using automatic URL routing.# Additionally, we include login URLs for the browsable API.urlpatterns= [
path('', include(router.urls)),
path('api-auth/', include('rest_framework.urls', namespace='rest_framework')),
]
我们还想为我们的API配置几个设置
将以下内容添加到您的settings.py模块：
INSTALLED_APPS= [
... # Make sure to include the default installed apps here.
'rest_framework',
]
REST_FRAMEWORK= {
# Use Django's standard `django.contrib.auth` permissions,# or allow read-only access for unauthenticated users.'DEFAULT_PERMISSION_CLASSES': [
'rest_framework.permissions.DjangoModelPermissionsOrAnonReadOnly',
]
}
$ uvicorn main:app --reloadINFO: Uvicorn running on http://127.0.0.1:8000 (Press CTRL+C to quit)INFO: Started reloader process [28720]INFO: Started server process [28722]INFO: Waiting for application startup.INFO: Application startup complete.
因此http协议创始人警告我们这些凡人们正在错误的使用http协议，除了警告，他还发表了一篇博客，大概意思就是教大家如何正确使用http协议，如何正确定义url，这就是REST（Representational State Transfer），不需要管这几个英文单词代表什么意思，只需要记住下面一句话：
用url唯一定位资源，用Http请求方式（GET, POST, DELETE, PUT)描述用户行为
按照之前的方式，我们使用request.POST, 如果打印该值，会发现是一个空对象：request post <QueryDict: {}>，该现象证明Django并不能处理请求协议为application/json编码协议的数据，我们可以去看看request源码，可以看到下面这一段：
if self.content_type == 'multipart/form-data':
if hasattr(self, '_body'):
# Use already read data
data = BytesIO(self._body)
else:
data = self
try:
self._post, self._files = self.parse_file_upload(self.META, data)
except MultiPartParserError:
# An error occurred while parsing POST data. Since when
# formatting the error the request handler might access
# self.POST, set self._post and self._file to prevent
# attempts to parse POST data again.
# Mark that an error occurred. This allows self.__repr__ to
# be explicit about it instead of simply representing an
# empty POST
self._mark_post_parse_error()
raise
elif self.content_type == 'application/x-www-form-urlencoded':
self._post, self._files = QueryDict(self.body, encoding=self._encoding), MultiValueDict()
else:
self._post, self._files = QueryDict(encoding=self._encoding), MultiValueDict()
@classonlymethod
def as_view(cls, **initkwargs):
"""Main entry point for a request-response process."""
for key in initkwargs:
if key in cls.http_method_names:
raise TypeError("You tried to pass in the %s method name as a "
"keyword argument to %s(). Don't do that."
% (key, cls.__name__))
if not hasattr(cls, key):
raise TypeError("%s() received an invalid keyword %r. as_view "
"only accepts arguments that are already "
"attributes of the class." % (cls.__name__, key))
def view(request, *args, **kwargs):
self = cls(**initkwargs)
if hasattr(self, 'get') and not hasattr(self, 'head'):
self.head = self.get
self.request = request
self.args = args
self.kwargs = kwargs
return self.dispatch(request, *args, **kwargs)
view.view_class = cls
view.view_initkwargs = initkwargs
# take name and docstring from class
update_wrapper(view, cls, updated=())
# and possible attributes set by decorators
# like csrf_exempt from dispatch
update_wrapper(view, cls.dispatch, assigned=())
return view
@classonlymethod
def as_view(cls, **initkwargs):
"""Main entry point for a request-response process."""
for key in initkwargs:
if key in cls.http_method_names:
raise TypeError("You tried to pass in the %s method name as a "
"keyword argument to %s(). Don't do that."
% (key, cls.__name__))
if not hasattr(cls, key):
raise TypeError("%s() received an invalid keyword %r. as_view "
"only accepts arguments that are already "
"attributes of the class." % (cls.__name__, key))
def view(request, *args, **kwargs):
if request.content_type == "application/json":
import json
return HttpResponse(json.dumps({"error": "Unsupport content type!"}))
self = cls(**initkwargs)
if hasattr(self, 'get') and not hasattr(self, 'head'):
self.head = self.get
self.request = request
self.args = args
self.kwargs = kwargs
return self.dispatch(request, *args, **kwargs)
view.view_class = cls
view.view_initkwargs = initkwargs
# take name and docstring from class
update_wrapper(view, cls, updated=())
# and possible attributes set by decorators
# like csrf_exempt from dispatch
update_wrapper(view, cls.dispatch, assigned=())
return view
