I am having trouble using json.loads to convert to a dict object and I can’t figure out what I’m doing wrong.The exact error I get running this is

ValueError: Expecting property name: line 1 column 2 (char 1)

Here is my code:

from kafka.client import KafkaClient
from kafka.consumer import SimpleConsumer
from kafka.producer import SimpleProducer, KeyedProducer
import pymongo
from pymongo import MongoClient
import json

c = MongoClient("54.210.157.57")
db = c.test_database3
collection = db.tweet_col

kafka = KafkaClient("54.210.157.57:9092")

consumer = SimpleConsumer(kafka,"myconsumer","test")
for tweet in consumer:
    print tweet.message.value
    jsonTweet=json.loads(({u'favorited': False, u'contributors': None})
    collection.insert(jsonTweet)

I’m pretty sure that the error is occuring at the 2nd to last line

jsonTweet=json.loads({u'favorited': False, u'contributors': None})

but I do not know what to do to fix it. Any advice would be appreciated.

json.loads will load a json string into a python dict, json.dumps will dump a python dict to a json string, for example:

>>> json_string = '{"favorited": false, "contributors": null}'
'{"favorited": false, "contributors": null}'
>>> value = json.loads(json_string)
{u'favorited': False, u'contributors': None}
>>> json_dump = json.dumps(value)
'{"favorited": false, "contributors": null}'

So that line is incorrect since you are trying to load a python dict, and json.loads is expecting a valid json string which should have <type 'str'>.

So if you are trying to load the json, you should change what you are loading to look like the json_string above, or you should be dumping it. This is just my best guess from the given information. What is it that you are trying to accomplish?

Also you don’t need to specify the u before your strings, as @Cld mentioned in the comments.

I encountered another problem that returns the same error.

Single quote issue

I used a json string with single quotes :

{
    'property': 1
}

But json.loads accepts only double quotes for json properties :

{
    "property": 1
}

Final comma issue

json.loads doesn’t accept a final comma:

{
  "property": "text", 
  "property2": "text2",
}

Solution: ast to solve single quote and final comma issues

You can use ast (part of standard library for both Python 2 and 3) for this processing. Here is an example :

import ast
# ast.literal_eval() return a dict object, we must use json.dumps to get JSON string
import json

# Single quote to double with ast.literal_eval()
json_data = "{'property': 'text'}"
json_data = ast.literal_eval(json_data)
print(json.dumps(json_data))
# Displays : {"property": "text"}

# ast.literal_eval() with double quotes
json_data = '{"property": "text"}'
json_data = ast.literal_eval(json_data)
print(json.dumps(json_data))
# Displays : {"property": "text"}

# ast.literal_eval() with final coma
json_data = "{'property': 'text', 'property2': 'text2',}"
json_data = ast.literal_eval(json_data)
print(json.dumps(json_data))
# Displays : {"property2": "text2", "property": "text"}

Using ast will prevent you from single quote and final comma issues by interpet the JSON like Python dictionnary (so you must follow the Python dictionnary syntax). It’s a pretty good and safely alternative of eval() function for literal structures.

Python documentation warned us of using large/complex string :

Warning It is possible to crash the Python interpreter with a sufficiently large/complex string due to stack depth limitations in Python’s AST compiler.

json.dumps with single quotes

To use json.dumps with single quotes easily you can use this code:

import ast
import json

data = json.dumps(ast.literal_eval(json_data_single_quote))

ast documentation

ast Python 3 doc

ast Python 2 doc

Tool

If you frequently edit JSON, you may use CodeBeautify. It helps you to fix syntax error and minify/beautify JSON.

I hope it helps.

1. replace all single quotes with double quotes
2. replace ‘u”‘ from your strings to ‘”‘ … so basically convert internal unicodes to strings before loading the string into json

>> strs = "{u'key':u'val'}"
>> strs = strs.replace("'",'"')
>> json.loads(strs.replace('u"','"'))

All other answers may answer your query, but I faced same issue which was due to stray , which I added at the end of my json string like this:

{
 "key":"123sdf",
 "bus_number":"asd234sdf",
}

I finally got it working when I removed extra , like this:

{
 "key":"123sdf",
 "bus_number":"asd234sdf"
}

Hope this help! cheers.

used ast, example

In [15]: a = "[{'start_city': '1', 'end_city': 'aaa', 'number': 1},\
...:      {'start_city': '2', 'end_city': 'bbb', 'number': 1},\
...:      {'start_city': '3', 'end_city': 'ccc', 'number': 1}]"
In [16]: import ast
In [17]: ast.literal_eval(a)
Out[17]:
[{'end_city': 'aaa', 'number': 1, 'start_city': '1'},
 {'end_city': 'bbb', 'number': 1, 'start_city': '2'},
 {'end_city': 'ccc', 'number': 1, 'start_city': '3'}]

A different case in which I encountered this was when I was using echo to pipe the JSON into my python script and carelessly wrapped the JSON string in double quotes:

echo "{"thumbnailWidth": 640}" | myscript.py

Note that the JSON string itself has quotes and I should have done:

echo '{"thumbnailWidth": 640}' | myscript.py

As it was, this is what the python script received: {thumbnailWidth: 640}; the double quotes were effectively stripped.