I am trying to understand if it makes sense to take the content of a list and append it to another list.

I have the first list created through a loop function, that will get specific lines out of a file and will save them in a list.

Then a second list is used to save these lines, and start a new cycle over another file.

My idea was to get the list once that the for cycle is done, dump it into the second list, then start a new cycle, dump the content of the first list again into the second but appending it, so the second list will be the sum of all the smaller list files created in my loop. The list has to be appended only if certain conditions met.

It looks like something similar to this:

# This is done for each log in my directory, i have a loop running
for logs in mydir:

    for line in mylog:
        #...if the conditions are met
        list1.append(line)

    for item in list1:
        if "string" in item: #if somewhere in the list1 i have a match for a string
            list2.append(list1) # append every line in list1 to list2
            del list1 [:] # delete the content of the list1
            break
        else:
            del list1 [:] # delete the list content and start all over

Does this makes sense or should I go for a different route?

I need something efficient that would not take up too many cycles, since the list of logs is long and each text file is pretty big; so I thought that the lists would fit the purpose.

You probably want

list2.extend(list1)

instead of

list2.append(list1)

Here’s the difference:

>>> a = range(5)
>>> b = range(3)
>>> c = range(2)
>>> b.append(a)
>>> b
[0, 1, 2, [0, 1, 2, 3, 4]]
>>> c.extend(a)
>>> c
[0, 1, 0, 1, 2, 3, 4]

Since list.extend() accepts an arbitrary iterable, you can also replace

for line in mylog:
    list1.append(line)

by

list1.extend(mylog)

Take a look at itertools.chain for a fast way to treat many small lists as a single big list (or at least as a single big iterable) without copying the smaller lists:

>>> import itertools
>>> p = ['a', 'b', 'c']
>>> q = ['d', 'e', 'f']
>>> r = ['g', 'h', 'i']
>>> for x in itertools.chain(p, q, r):
        print x.upper()

That seems fairly reasonable for what you’re trying to do.

A slightly shorter version which leans on Python to do more of the heavy lifting might be:

for logs in mydir:

    for line in mylog:
        #...if the conditions are met
        list1.append(line)

    if any(True for line in list1 if "string" in line):
        list2.extend(list1)
    del list1

    ....

The (True for line in list1 if "string" in line) iterates over list and emits True whenever a match is found. any() uses short-circuit evaluation to return True as soon as the first True element is found. list2.extend() appends the contents of list1 to the end.

You can also combine two lists (say a,b) using the ‘+’ operator. For example,

a = [1,2,3,4]
b = [4,5,6,7]
c = a + b

Output:
>>> c
[1, 2, 3, 4, 4, 5, 6, 7]

To recap on the previous answers. If you have a list with [0,1,2] and another one with [3,4,5] and you want to merge them, so it becomes [0,1,2,3,4,5], you can either use chaining or extending and should know the differences to use it wisely for your needs.

Extending a list

Using the list classes extend method, you can do a copy of the elements from one list onto another. However this will cause extra memory usage, which should be fine in most cases, but might cause problems if you want to be memory efficient.

a = [0,1,2]
b = [3,4,5]
a.extend(b)
>>[0,1,2,3,4,5]

Chaining a list

Contrary you can use itertools.chain to wire many lists, which will return a so called iterator that can be used to iterate over the lists. This is more memory efficient as it is not copying elements over but just pointing to the next list.

import itertools
a = [0,1,2]
b = [3,4,5]
c = itertools.chain(a, b)

Make an iterator that returns elements from the first iterable until it is exhausted, then proceeds to the next iterable, until all of the iterables are exhausted. Used for treating consecutive sequences as a single sequence.

Using the map() and reduce() built-in functions

def file_to_list(file):
     #stuff to parse file to a list
     return list

files = [...list of files...]

L = map(file_to_list, files)

flat_L = reduce(lambda x,y:x+y, L)

Minimal “for looping” and elegant coding pattern :)

If we have list like below:

list  = [2,2,3,4]

two ways to copy it into another list.

1.

x = [list]  # x =[] x.append(list) same 
print("length is {}".format(len(x)))
for i in x:
    print(i)

length is 1
[2, 2, 3, 4]

2.

x = [l for l in list]
print("length is {}".format(len(x)))
for i in x:
    print(i)

length is 4
2
2
3
4