I want to execute a curl command in python.

Usually, I just need enter the command in terminal and press return key. However, I don’t know how it works in python.

The command shows below:

curl -d @request.json --header "Content-Type: application/json" https://www.googleapis.com/qpxExpress/v1/trips/search?key=mykeyhere

There is a request.json file to be sent to get response.

I searched a lot and got confused. I tried to write a piece of code, although I could not fully understand. It didn’t work.

import pycurl
import StringIO

response = StringIO.StringIO()
c = pycurl.Curl()
c.setopt(c.URL, 'https://www.googleapis.com/qpxExpress/v1/trips/search?key=mykeyhere')
c.setopt(c.WRITEFUNCTION, response.write)
c.setopt(c.HTTPHEADER, ['Content-Type: application/json','Accept-Charset: UTF-8'])
c.setopt(c.POSTFIELDS, '@request.json')
c.perform()
c.close()
print response.getvalue()
response.close()

The error message is ‘Parse Error’.Can anyone tell me how to fix it? or how to get response from the sever correctly?

For sake of simplicity, maybe you should consider using the Requests library.

An example with json response content would be something like:

import requests
r = requests.get('https://github.com/timeline.json')
r.json()

If you look for further information, in the Quickstart section, they have lots of working examples.

EDIT:

For your specific curl translation:

import requests
url = 'https://www.googleapis.com/qpxExpress/v1/trips/search?key=mykeyhere'
payload = open("request.json")
headers = {'content-type': 'application/json', 'Accept-Charset': 'UTF-8'}
r = requests.post(url, data=payload, headers=headers)

Just use this website. It’ll convert any curl command into Python, Node.js, PHP, R, or Go.

Example:

curl -X POST -H 'Content-type: application/json' --data '{"text":"Hello, World!"}' https://hooks.slack.com/services/asdfasdfasdf

Becomes this in Python,

import requests

headers = {
    'Content-type': 'application/json',
}

data = '{"text":"Hello, World!"}'

response = requests.post('https://hooks.slack.com/services/asdfasdfasdf', headers=headers, data=data)

import requests
url = "https://www.googleapis.com/qpxExpress/v1/trips/search?key=mykeyhere"
data = requests.get(url).json

maybe?

if you are trying to send a file

files = {'request_file': open('request.json', 'rb')}
r = requests.post(url, files=files)
print r.text, print r.json

ahh thanks @LukasGraf now i better understand what his original code is doing

import requests,json
url = "https://www.googleapis.com/qpxExpress/v1/trips/search?key=mykeyhere"
my_json_data = json.load(open("request.json"))
req = requests.post(url,data=my_json_data)
print req.text
print 
print req.json # maybe? 

curl -d @request.json --header "Content-Type: application/json" https://www.googleapis.com/qpxExpress/v1/trips/search?key=mykeyhere

its python implementation be like

import requests

headers = {
    'Content-Type': 'application/json',
}

params = (
    ('key', 'mykeyhere'),
)

data = open('request.json')
response = requests.post('https://www.googleapis.com/qpxExpress/v1/trips/search', headers=headers, params=params, data=data)

#NB. Original query string below. It seems impossible to parse and
#reproduce query strings 100% accurately so the one below is given
#in case the reproduced version is not "correct".
# response = requests.post('https://www.googleapis.com/qpxExpress/v1/trips/search?key=mykeyhere', headers=headers, data=data)

check this link, it will help convert cURl command to python,php and nodejs

My answer is WRT python 2.6.2.

import commands

status, output = commands.getstatusoutput("curl -H \"Content-Type:application/json\" -k -u (few other parameters required) -X GET https://example.org -s")

print output

I apologize for not providing the required parameters ‘coz it’s confidential.

Some background: I went looking for exactly this question because I had to do something to retrieve content, but all I had available was an old version of python with inadequate SSL support. If you’re on an older MacBook, you know what I’m talking about. In any case, curl runs fine from a shell (I suspect it has modern SSL support linked in) so sometimes you want to do this without using requests or urllib2.

You can use the subprocess module to execute curl and get at the retrieved content:

import subprocess

// 'response' contains a []byte with the retrieved content.
// use '-s' to keep curl quiet while it does its job, but
// it's useful to omit that while you're still writing code
// so you know if curl is working
response = subprocess.check_output(['curl', '-s', baseURL % page_num])

Python 3’s subprocess module also contains .run() with a number of useful options. I’ll leave it to someone who is actually running python 3 to provide that answer.

This could be achieve with the below mentioned psuedo code approach

Import os import requests Data = os.execute(curl URL) R= Data.json()

I have a cURL call that I use in PHP:

curl -i -H ‘Accept: application/xml’ -u login:key “https://app.streamsend.com/emails“

I need a way to do the same thing in Python. Is there an alternative to cURL in Python. I know of urllib but I’m a Python noob and have no idea how to use it.

import urllib2

manager = urllib2.HTTPPasswordMgrWithDefaultRealm()
manager.add_password(None, 'https://app.streamsend.com/emails', 'login', 'key')
handler = urllib2.HTTPBasicAuthHandler(manager)

director = urllib2.OpenerDirector()
director.add_handler(handler)

req = urllib2.Request('https://app.streamsend.com/emails', headers = {'Accept' : 'application/xml'})

result = director.open(req)
# result.read() will contain the data
# result.info() will contain the HTTP headers

# To get say the content-length header
length = result.info()['Content-Length']

Your cURL call using urllib2 instead. Completely untested.

You can use HTTP Requests that are described in the Requests: HTTP for Humans user guide.

Here’s a simple example using urllib2 that does a basic authentication against GitHub’s API.

import urllib2

u='username'
p='userpass'
url='https://api.github.com/users/username'

# simple wrapper function to encode the username & pass
def encodeUserData(user, password):
    return "Basic " + (user + ":" + password).encode("base64").rstrip()

# create the request object and set some headers
req = urllib2.Request(url)
req.add_header('Accept', 'application/json')
req.add_header("Content-type", "application/x-www-form-urlencoded")
req.add_header('Authorization', encodeUserData(u, p))
# make the request and print the results
res = urllib2.urlopen(req)
print res.read()

Furthermore if you wrap this in a script and run it from a terminal you can pipe the response string to ‘mjson.tool’ to enable pretty printing.

>> basicAuth.py | python -mjson.tool

One last thing to note, urllib2 only supports GET & POST requests.
If you need to use other HTTP verbs like DELETE, PUT, etc you’ll probably want to take a look at PYCURL

If you are using a command to just call curl like that, you can do the same thing in Python with subprocess. Example:

subprocess.call(['curl', '-i', '-H', '"Accept: application/xml"', '-u', 'login:key', '"https://app.streamsend.com/emails"'])

Or you could try PycURL if you want to have it as a more structured api like what PHP has.

import requests

url = 'https://example.tld/'
auth = ('username', 'password')

r = requests.get(url, auth=auth)
print r.content

This is the simplest I’ve been able to get it.

Some example, how to use urllib for that things, with some sugar syntax. I know about requests and other libraries, but urllib is standard lib for python and doesn’t require anything to be installed separately.

Python 2/3 compatible.

import sys
if sys.version_info.major == 3:
  from urllib.request import HTTPPasswordMgrWithDefaultRealm, HTTPBasicAuthHandler, Request, build_opener
  from urllib.parse import urlencode
else:
  from urllib2 import HTTPPasswordMgrWithDefaultRealm, HTTPBasicAuthHandler, Request, build_opener
  from urllib import urlencode


def curl(url, params=None, auth=None, req_type="GET", data=None, headers=None):
  post_req = ["POST", "PUT"]
  get_req = ["GET", "DELETE"]

  if params is not None:
    url += "?" + urlencode(params)

  if req_type not in post_req + get_req:
    raise IOError("Wrong request type \"%s\" passed" % req_type)

  _headers = {}
  handler_chain = []

  if auth is not None:
    manager = HTTPPasswordMgrWithDefaultRealm()
    manager.add_password(None, url, auth["user"], auth["pass"])
    handler_chain.append(HTTPBasicAuthHandler(manager))

  if req_type in post_req and data is not None:
    _headers["Content-Length"] = len(data)

  if headers is not None:
    _headers.update(headers)

  director = build_opener(*handler_chain)

  if req_type in post_req:
    if sys.version_info.major == 3:
      _data = bytes(data, encoding='utf8')
    else:
      _data = bytes(data)

    req = Request(url, headers=_headers, data=_data)
  else:
    req = Request(url, headers=_headers)

  req.get_method = lambda: req_type
  result = director.open(req)

  return {
    "httpcode": result.code,
    "headers": result.info(),
    "content": result.read()
  }


"""
Usage example:
"""

Post data:
  curl("http://127.0.0.1/", req_type="POST", data='cascac')

Pass arguments (http://127.0.0.1/?q=show):
  curl("http://127.0.0.1/", params={'q': 'show'}, req_type="POST", data='cascac')

HTTP Authorization:
  curl("http://127.0.0.1/secure_data.txt", auth={"user": "username", "pass": "password"})

Function is not complete and possibly is not ideal, but shows a basic representation and concept to use. Additional things could be added or changed by taste.

12/08 update

Here is a GitHub link to live updated source. Currently supporting:

• authorization

• CRUD compatible

• automatic charset detection

• automatic encoding(compression) detection

If it’s running all of the above from the command line that you’re looking for, then I’d recommend HTTPie. It is a fantastic cURL alternative and is super easy and convenient to use (and customize).

Here’s is its (succinct and precise) description from GitHub;

HTTPie (pronounced aych-tee-tee-pie) is a command line HTTP client. Its goal is to make CLI interaction with web services as human-friendly as possible.

It provides a simple http command that allows for sending arbitrary HTTP requests using a simple and natural syntax, and displays colorized output. HTTPie can be used for testing, debugging, and generally interacting with HTTP servers.

The documentation around authentication should give you enough pointers to solve your problem(s). Of course, all of the answers above are accurate as well, and provide different ways of accomplishing the same task.

Just so you do NOT have to move away from Stack Overflow, here’s what it offers in a nutshell.

Basic auth:

$ http -a username:password example.org
Digest auth:

$ http --auth-type=digest -a username:password example.org
With password prompt:

$ http -a username example.org