So I was wondering how to best create a list of blank lists:

[[],[],[]...]

Because of how Python works with lists in memory, this doesn’t work:

[[]]*n

This does create [[],[],...] but each element is the same list:

d = [[]]*n
d[0].append(1)
#[[1],[1],...]

Something like a list comprehension works:

d = [[] for x in xrange(0,n)]

But this uses the Python VM for looping. Is there any way to use an implied loop (taking advantage of it being written in C)?

d = []
map(lambda n: d.append([]),xrange(0,10))

This is actually slower. :(

The probably only way which is marginally faster than

d = [[] for x in xrange(n)]

is

from itertools import repeat
d = [[] for i in repeat(None, n)]

It does not have to create a new int object in every iteration and is about 15 % faster on my machine.

Edit: Using NumPy, you can avoid the Python loop using

d = numpy.empty((n, 0)).tolist()

but this is actually 2.5 times slower than the list comprehension.

The list comprehensions actually are implemented more efficiently than explicit looping (see the dis output for example functions) and the map way has to invoke an ophaque callable object on every iteration, which incurs considerable overhead overhead.

Regardless, [[] for _dummy in xrange(n)] is the right way to do it and none of the tiny (if existent at all) speed differences between various other ways should matter. Unless of course you spend most of your time doing this – but in that case, you should work on your algorithms instead. How often do you create these lists?

Here are two methods, one sweet and simple(and conceptual), the other more formal and can be extended in a variety of situations, after having read a dataset.

Method 1: Conceptual

X2=[]
X1=[1,2,3]
X2.append(X1)
X3=[4,5,6]
X2.append(X3)
X2 thus has [[1,2,3],[4,5,6]] ie a list of lists. 

Method 2 : Formal and extensible

Another elegant way to store a list as a list of lists of different numbers – which it reads from a file. (The file here has the dataset train) Train is a data-set with say 50 rows and 20 columns. ie. Train[0] gives me the 1st row of a csv file, train[1] gives me the 2nd row and so on. I am interested in separating the dataset with 50 rows as one list, except the column 0 , which is my explained variable here, so must be removed from the orignal train dataset, and then scaling up list after list- ie a list of a list. Here’s the code that does that.

Note that I am reading from “1” in the inner loop since I am interested in explanatory variables only. And I re-initialize X1=[] in the other loop, else the X2.append([0:(len(train[0])-1)]) will rewrite X1 over and over again – besides it more memory efficient.

X2=[]
for j in range(0,len(train)):
    X1=[]
    for k in range(1,len(train[0])):
        txt2=train[j][k]
        X1.append(txt2)
    X2.append(X1[0:(len(train[0])-1)])

To create list and list of lists use below syntax

     x = [[] for i in range(10)]

this will create 1-d list and to initialize it put number in [[number] and set length of list put length in range(length)

• To create list of lists use below syntax.

    x = [[[0] for i in range(3)] for i in range(10)]

this will initialize list of lists with 10*3 dimension and with value 0

• To access/manipulate element

    x[1][5]=value

So I did some speed comparisons to get the fastest way. List comprehensions are indeed very fast. The only way to get close is to avoid bytecode getting exectuded during construction of the list. My first attempt was the following method, which would appear to be faster in principle:

l = [[]]
for _ in range(n): l.extend(map(list,l))

(produces a list of length 2**n, of course) This construction is twice as slow as the list comprehension, according to timeit, for both short and long (a million) lists.

My second attempt was to use starmap to call the list constructor for me, There is one construction, which appears to run the list constructor at top speed, but still is slower, but only by a tiny amount:

from itertools import starmap
l = list(starmap(list,[()]*(1<<n)))

Interesting enough the execution time suggests that it is the final list call that is makes the starmap solution slow, since its execution time is almost exactly equal to the speed of:

l = list([] for _ in range(1<<n))

My third attempt came when I realized that list(()) also produces a list, so I tried the apperently simple:

l = list(map(list, [()]*(1<<n)))

but this was slower than the starmap call.

Conclusion: for the speed maniacs: Do use the list comprehension. Only call functions, if you have to. Use builtins.