I can’t figure out how to use an array or matrix in the way that I would normally use a list. I want to create an empty array (or matrix) and then add one column (or row) to it at a time.

At the moment the only way I can find to do this is like:

mat = None
for col in columns:
    if mat is None:
        mat = col
    else:
        mat = hstack((mat, col))

Whereas if it were a list, I’d do something like this:

list = []
for item in data:
    list.append(item)

Is there a way to use that kind of notation for NumPy arrays or matrices?

You have the wrong mental model for using NumPy efficiently. NumPy arrays are stored in contiguous blocks of memory. If you want to add rows or columns to an existing array, the entire array needs to be copied to a new block of memory, creating gaps for the new elements to be stored. This is very inefficient if done repeatedly to build an array.

In the case of adding rows, your best bet is to create an array that is as big as your data set will eventually be, and then assign data to it row-by-row:

>>> import numpy
>>> a = numpy.zeros(shape=(5,2))
>>> a
array([[ 0.,  0.],
   [ 0.,  0.],
   [ 0.,  0.],
   [ 0.,  0.],
   [ 0.,  0.]])
>>> a[0] = [1,2]
>>> a[1] = [2,3]
>>> a
array([[ 1.,  2.],
   [ 2.,  3.],
   [ 0.,  0.],
   [ 0.,  0.],
   [ 0.,  0.]])

A NumPy array is a very different data structure from a list and is designed to be used in different ways. Your use of hstack is potentially very inefficient… every time you call it, all the data in the existing array is copied into a new one. (The append function will have the same issue.) If you want to build up your matrix one column at a time, you might be best off to keep it in a list until it is finished, and only then convert it into an array.

e.g.

mylist = []
for item in data:
    mylist.append(item)
mat = numpy.array(mylist)

item can be a list, an array or any iterable, as long as each item has the same number of elements.
In this particular case (data is some iterable holding the matrix columns) you can simply use

mat = numpy.array(data)

(Also note that using list as a variable name is probably not good practice since it masks the built-in type by that name, which can lead to bugs.)

EDIT:

If for some reason you really do want to create an empty array, you can just use numpy.array([]), but this is rarely useful!

To create an empty multidimensional array in NumPy (e.g. a 2D array m*n to store your matrix), in case you don’t know m how many rows you will append and don’t care about the computational cost Stephen Simmons mentioned (namely re-buildinging the array at each append), you can squeeze to 0 the dimension to which you want to append to: X = np.empty(shape=[0, n]).

This way you can use for example (here m = 5 which we assume we didn’t know when creating the empty matrix, and n = 2):

import numpy as np

n = 2
X = np.empty(shape=[0, n])

for i in range(5):
    for j  in range(2):
        X = np.append(X, [[i, j]], axis=0)

print X

which will give you:

[[ 0.  0.]
 [ 0.  1.]
 [ 1.  0.]
 [ 1.  1.]
 [ 2.  0.]
 [ 2.  1.]
 [ 3.  0.]
 [ 3.  1.]
 [ 4.  0.]
 [ 4.  1.]]

I looked into this a lot because I needed to use a numpy.array as a set in one of my school projects and I needed to be initialized empty… I didn’t found any relevant answer here on Stack Overflow, so I started doodling something.

# Initialize your variable as an empty list first
In [32]: x=[]
# and now cast it as a numpy ndarray
In [33]: x=np.array(x)

The result will be:

In [34]: x
Out[34]: array([], dtype=float64)

Therefore you can directly initialize an np array as follows:

In [36]: x= np.array([], dtype=np.float64)

I hope this helps.

You can use the append function. For rows:

>>> from numpy import *
>>> a = array([10,20,30])
>>> append(a, [[1,2,3]], axis=0)
array([[10, 20, 30],      
       [1, 2, 3]])

For columns:

>>> append(a, [[15],[15]], axis=1)
array([[10, 20, 30, 15],      
       [1, 2, 3, 15]])

EDIT
Of course, as mentioned in other answers, unless you’re doing some processing (ex. inversion) on the matrix/array EVERY time you append something to it, I would just create a list, append to it then convert it to an array.

If you absolutely don’t know the final size of the array, you can increment the size of the array like this:

my_arr = numpy.zeros((0,5))
for i in range(3):
    my_arr=numpy.concatenate( ( my_arr, numpy.ones((1,5)) ) )
print(my_arr)

[[ 1.  1.  1.  1.  1.]  [ 1.  1.  1.  1.  1.]  [ 1.  1.  1.  1.  1.]]

• Notice the 0 in the first line.
• numpy.append is another option. It calls numpy.concatenate.

You can apply it to build any kind of array, like zeros:

a = range(5)
a = [i*0 for i in a]
print a 
[0, 0, 0, 0, 0]

Here is some workaround to make numpys look more like Lists

np_arr = np.array([])
np_arr = np.append(np_arr , 2)
np_arr = np.append(np_arr , 24)
print(np_arr)

OUTPUT: array([ 2., 24.])

Depending on what you are using this for, you may need to specify the data type (see ‘dtype’).

For example, to create a 2D array of 8-bit values (suitable for use as a monochrome image):

myarray = numpy.empty(shape=(H,W),dtype='u1')

For an RGB image, include the number of color channels in the shape: shape=(H,W,3)

You may also want to consider zero-initializing with numpy.zeros instead of using numpy.empty. See the note here.

I think you want to handle most of the work with lists then use the result as a matrix. Maybe this is a way ;

ur_list = []
for col in columns:
    ur_list.append(list(col))

mat = np.matrix(ur_list)

I think you can create empty numpy array like:

>>> import numpy as np
>>> empty_array= np.zeros(0)
>>> empty_array
array([], dtype=float64)
>>> empty_array.shape
(0,)

This format is useful when you want to append numpy array in the loop.

For creating an empty NumPy array without defining its shape there is to way:

1.

arr = np.array([]) 

preferred. cause you know you will be using this as numpy.

2.

arr = []
# and use it as numpy. append to it or etc..

NumPy converts this to np.ndarray type afterward, without extra [] dimionsion.

What is the difference between Numpy’s array() and asarray() functions? When should you use one rather than the other? They seem to generate identical output for all the inputs I can think of.

Since other questions are being redirected to this one which ask about asanyarray or other array creation routines, it’s probably worth having a brief summary of what each of them does.

The differences are mainly about when to return the input unchanged, as opposed to making a new array as a copy.

array offers a wide variety of options (most of the other functions are thin wrappers around it), including flags to determine when to copy. A full explanation would take just as long as the docs (see Array Creation, but briefly, here are some examples:

Assume a is an ndarray, and m is a matrix, and they both have a dtype of float32:

• np.array(a) and np.array(m) will copy both, because that’s the default behavior.
• np.array(a, copy=False) and np.array(m, copy=False) will copy m but not a, because m is not an ndarray.
• np.array(a, copy=False, subok=True) and np.array(m, copy=False, subok=True) will copy neither, because m is a matrix, which is a subclass of ndarray.
• np.array(a, dtype=int, copy=False, subok=True) will copy both, because the dtype is not compatible.

Most of the other functions are thin wrappers around array that control when copying happens:

• asarray: The input will be returned uncopied iff it’s a compatible ndarray (copy=False).
• asanyarray: The input will be returned uncopied iff it’s a compatible ndarray or subclass like matrix (copy=False, subok=True).
• ascontiguousarray: The input will be returned uncopied iff it’s a compatible ndarray in contiguous C order (copy=False, order='C').
• asfortranarray: The input will be returned uncopied iff it’s a compatible ndarray in contiguous Fortran order (copy=False, order='F').
• require: The input will be returned uncopied iff it’s compatible with the specified requirements string.
• copy: The input is always copied.
• fromiter: The input is treated as an iterable (so, e.g., you can construct an array from an iterator’s elements, instead of an object array with the iterator); always copied.

There are also convenience functions, like asarray_chkfinite (same copying rules as asarray, but raises ValueError if there are any nan or inf values), and constructors for subclasses like matrix or for special cases like record arrays, and of course the actual ndarray constructor (which lets you create an array directly out of strides over a buffer).

The definition of asarray is:

def asarray(a, dtype=None, order=None):
    return array(a, dtype, copy=False, order=order)

So it is like array, except it has fewer options, and copy=False. array has copy=True by default.

The main difference is that array (by default) will make a copy of the object, while asarray will not unless necessary.

The difference can be demonstrated by this example:

1. generate a matrix

   >>> A = numpy.matrix(numpy.ones((3,3)))
   >>> A
   matrix([[ 1.,  1.,  1.],
           [ 1.,  1.,  1.],
           [ 1.,  1.,  1.]])
   

2. use numpy.array to modify A. Doesn’t work because you are modifying a copy

   >>> numpy.array(A)[2]=2
   >>> A
   matrix([[ 1.,  1.,  1.],
           [ 1.,  1.,  1.],
           [ 1.,  1.,  1.]])
   

3. use numpy.asarray to modify A. It worked because you are modifying A itself

   >>> numpy.asarray(A)[2]=2
   >>> A
   matrix([[ 1.,  1.,  1.],
           [ 1.,  1.,  1.],
           [ 2.,  2.,  2.]])
   

Hope this helps!

The differences are mentioned quite clearly in the documentation of array and asarray. The differences lie in the argument list and hence the action of the function depending on those parameters.

The function definitions are :

numpy.array(object, dtype=None, copy=True, order=None, subok=False, ndmin=0)

and

numpy.asarray(a, dtype=None, order=None)

The following arguments are those that may be passed to array and not asarray as mentioned in the documentation :

copy : bool, optional If true (default), then the object is copied. Otherwise, a copy will only be made if __array__ returns a copy, if obj is a nested sequence, or if a copy is needed to satisfy any of the other requirements (dtype, order, etc.).

subok : bool, optional If True, then sub-classes will be passed-through, otherwise the returned array will be forced to be a base-class array (default).

ndmin : int, optional Specifies the minimum number of dimensions that the resulting array should have. Ones will be pre-pended to the shape as needed to meet this requirement.

Here’s a simple example that can demonstrate the difference.

The main difference is that array will make a copy of the original data and using different object we can modify the data in the original array.

import numpy as np
a = np.arange(0.0, 10.2, 0.12)
int_cvr = np.asarray(a, dtype = np.int64)

The contents in array (a), remain untouched, and still, we can perform any operation on the data using another object without modifying the content in original array.

asarray(x) is like array(x, copy=False)

Use asarray(x) when you want to ensure that x will be an array before any other operations are done. If x is already an array then no copy would be done. It would not cause a redundant performance hit.

Here is an example of a function that ensure x is converted into an array first.

def mysum(x):
    return np.asarray(x).sum()