Python 实用宝典

Question 1

What is the best way to get a list of all files in a directory, sorted by date [created | modified], using python, on a windows machine?

Question 2

Update: to sort dirpath‘s entries by modification date in Python 3:

import os
from pathlib import Path

paths = sorted(Path(dirpath).iterdir(), key=os.path.getmtime)

_{(put @Pygirl’s answer here for greater visibility)}

If you already have a list of filenames files, then to sort it inplace by creation time on Windows:

files.sort(key=os.path.getctime)

The list of files you could get, for example, using glob as shown in @Jay’s answer.

^{old answer} Here’s a more verbose version of @Greg Hewgill‘s answer. It is the most conforming to the question requirements. It makes a distinction between creation and modification dates (at least on Windows).

#!/usr/bin/env python
from stat import S_ISREG, ST_CTIME, ST_MODE
import os, sys, time

# path to the directory (relative or absolute)
dirpath = sys.argv[1] if len(sys.argv) == 2 else r'.'

# get all entries in the directory w/ stats
entries = (os.path.join(dirpath, fn) for fn in os.listdir(dirpath))
entries = ((os.stat(path), path) for path in entries)

# leave only regular files, insert creation date
entries = ((stat[ST_CTIME], path)
           for stat, path in entries if S_ISREG(stat[ST_MODE]))
#NOTE: on Windows `ST_CTIME` is a creation date 
#  but on Unix it could be something else
#NOTE: use `ST_MTIME` to sort by a modification date

for cdate, path in sorted(entries):
    print time.ctime(cdate), os.path.basename(path)

Example:

$ python stat_creation_date.py
Thu Feb 11 13:31:07 2009 stat_creation_date.py

Question 3

I’ve done this in the past for a Python script to determine the last updated files in a directory:

import glob
import os

search_dir = "/mydir/"
# remove anything from the list that is not a file (directories, symlinks)
# thanks to J.F. Sebastion for pointing out that the requirement was a list 
# of files (presumably not including directories)  
files = list(filter(os.path.isfile, glob.glob(search_dir + "*")))
files.sort(key=lambda x: os.path.getmtime(x))

That should do what you’re looking for based on file mtime.

EDIT: Note that you can also use os.listdir() in place of glob.glob() if desired – the reason I used glob in my original code was that I was wanting to use glob to only search for files with a particular set of file extensions, which glob() was better suited to. To use listdir here’s what it would look like:

import os

search_dir = "/mydir/"
os.chdir(search_dir)
files = filter(os.path.isfile, os.listdir(search_dir))
files = [os.path.join(search_dir, f) for f in files] # add path to each file
files.sort(key=lambda x: os.path.getmtime(x))

Question 4

There is an os.path.getmtime function that gives the number of seconds since the epoch and should be faster than os.stat.

import os 

os.chdir(directory)
sorted(filter(os.path.isfile, os.listdir('.')), key=os.path.getmtime)

Question 5

Here’s my version:

def getfiles(dirpath):
    a = [s for s in os.listdir(dirpath)
         if os.path.isfile(os.path.join(dirpath, s))]
    a.sort(key=lambda s: os.path.getmtime(os.path.join(dirpath, s)))
    return a

First, we build a list of the file names. isfile() is used to skip directories; it can be omitted if directories should be included. Then, we sort the list in-place, using the modify date as the key.

Question 6

Here’s a one-liner:

import os
import time
from pprint import pprint

pprint([(x[0], time.ctime(x[1].st_ctime)) for x in sorted([(fn, os.stat(fn)) for fn in os.listdir(".")], key = lambda x: x[1].st_ctime)])

This calls os.listdir() to get a list of the filenames, then calls os.stat() for each one to get the creation time, then sorts against the creation time.

Note that this method only calls os.stat() once for each file, which will be more efficient than calling it for each comparison in a sort.

Question 7

Without changing directory:

import os    

path = '/path/to/files/'
name_list = os.listdir(path)
full_list = [os.path.join(path,i) for i in name_list]
time_sorted_list = sorted(full_list, key=os.path.getmtime)

print time_sorted_list

# if you want just the filenames sorted, simply remove the dir from each
sorted_filename_list = [ os.path.basename(i) for i in time_sorted_list]
print sorted_filename_list

Question 8

In python 3.5+

from pathlib import Path
sorted(Path('.').iterdir(), key=lambda f: f.stat().st_mtime)

Question 9

Here’s my answer using glob without filter if you want to read files with a certain extension in date order (Python 3).

dataset_path='/mydir/'   
files = glob.glob(dataset_path+"/morepath/*.extension")   
files.sort(key=os.path.getmtime)

Question 10

# *** the shortest and best way ***
# getmtime --> sort by modified time
# getctime --> sort by created time

import glob,os

lst_files = glob.glob("*.txt")
lst_files.sort(key=os.path.getmtime)
print("\n".join(lst_files))

Question 11

sorted(filter(os.path.isfile, os.listdir('.')), 
    key=lambda p: os.stat(p).st_mtime)

You could use os.walk('.').next()[-1] instead of filtering with os.path.isfile, but that leaves dead symlinks in the list, and os.stat will fail on them.

Question 12

from pathlib import Path
import os

sorted(Path('./').iterdir(), key=lambda t: t.stat().st_mtime)

or

sorted(Path('./').iterdir(), key=os.path.getmtime)

or

sorted(os.scandir('./'), key=lambda t: t.stat().st_mtime)

where m time is modified time.

Question 13

this is a basic step for learn:

import os, stat, sys
import time

dirpath = sys.argv[1] if len(sys.argv) == 2 else r'.'

listdir = os.listdir(dirpath)

for i in listdir:
    os.chdir(dirpath)
    data_001 = os.path.realpath(i)
    listdir_stat1 = os.stat(data_001)
    listdir_stat2 = ((os.stat(data_001), data_001))
    print time.ctime(listdir_stat1.st_ctime), data_001

Question 14

Alex Coventry’s answer will produce an exception if the file is a symlink to an unexistent file, the following code corrects that answer:

import time
import datetime
sorted(filter(os.path.isfile, os.listdir('.')), 
    key=lambda p: os.path.exists(p) and os.stat(p).st_mtime or time.mktime(datetime.now().timetuple())

When the file doesn’t exist, now() is used, and the symlink will go at the very end of the list.

Question 15

Here is a simple couple lines that looks for extention as well as provides a sort option

def get_sorted_files(src_dir, regex_ext='*', sort_reverse=False): 
    files_to_evaluate = [os.path.join(src_dir, f) for f in os.listdir(src_dir) if re.search(r'.*\.({})$'.format(regex_ext), f)]
    files_to_evaluate.sort(key=os.path.getmtime, reverse=sort_reverse)
    return files_to_evaluate

Question 16

For completeness with os.scandir (2x faster over pathlib):

import os
sorted(os.scandir('/tmp/test'), key=lambda d: d.stat().st_mtime)

Question 17

This was my version:

import os

folder_path = r'D:\Movies\extra\new\dramas' # your path
os.chdir(folder_path) # make the path active
x = sorted(os.listdir(), key=os.path.getctime)  # sorted using creation time

folder = 0

for folder in range(len(x)):
    print(x[folder]) # print all the foldername inside the folder_path
    folder = +1

Question 18

Maybe you should use shell commands. In Unix/Linux, find piped with sort will probably be able to do what you want.

Question 19

I would like to get just the folder path from the full path to a file.

For example T:\Data\DBDesign\DBDesign_93_v141b.mdb and I would like to get just T:\Data\DBDesign (excluding the \DBDesign_93_v141b.mdb).

I have tried something like this:

existGDBPath = r'T:\Data\DBDesign\DBDesign_93_v141b.mdb'
wkspFldr = str(existGDBPath.split('\\')[0:-1])
print wkspFldr

but it gave me a result like this:

['T:', 'Data', 'DBDesign']

which is not the result that I require (being T:\Data\DBDesign).

Any ideas on how I can get the path to my file?

Question 20

You were almost there with your use of the split function. You just needed to join the strings, like follows.

>>> import os
>>> '\\'.join(existGDBPath.split('\\')[0:-1])
'T:\\Data\\DBDesign'

Although, I would recommend using the os.path.dirname function to do this, you just need to pass the string, and it’ll do the work for you. Since, you seem to be on windows, consider using the abspath function too. An example:

>>> import os
>>> os.path.dirname(os.path.abspath(existGDBPath))
'T:\\Data\\DBDesign'

If you want both the file name and the directory path after being split, you can use the os.path.split function which returns a tuple, as follows.

>>> import os
>>> os.path.split(os.path.abspath(existGDBPath))
('T:\\Data\\DBDesign', 'DBDesign_93_v141b.mdb')

Question 21

WITH PATHLIB MODULE (UPDATED ANSWER)

One should consider using pathlib for new development. It is in the stdlib for Python3.4, but available on PyPI for earlier versions. This library provides a more object-orented method to manipulate paths <opinion> and is much easier read and program with </opinion>.

>>> import pathlib
>>> existGDBPath = pathlib.Path(r'T:\Data\DBDesign\DBDesign_93_v141b.mdb')
>>> wkspFldr = existGDBPath.parent
>>> print wkspFldr
Path('T:\Data\DBDesign')

WITH OS MODULE

Use the os.path module:

>>> import os
>>> existGDBPath = r'T:\Data\DBDesign\DBDesign_93_v141b.mdb'
>>> wkspFldr = os.path.dirname(existGDBPath)
>>> print wkspFldr 
'T:\Data\DBDesign'

You can go ahead and assume that if you need to do some sort of filename manipulation it’s already been implemented in os.path. If not, you’ll still probably need to use this module as the building block.

Question 22

The built-in submodule os.path has a function for that very task.

import os
os.path.dirname('T:\Data\DBDesign\DBDesign_93_v141b.mdb')

Question 23

Here is the code:

import os
existGDBPath = r'T:\Data\DBDesign\DBDesign_93_v141b.mdb'
wkspFldr = os.path.dirname(existGDBPath)
print wkspFldr # T:\Data\DBDesign

Question 24

Here is my little utility helper for splitting paths int file, path tokens:

import os    
# usage: file, path = splitPath(s)
def splitPath(s):
    f = os.path.basename(s)
    p = s[:-(len(f))-1]
    return f, p

Question 25

Anyone trying to do this in the ESRI GIS Table field calculator interface can do this with the Python parser:

PathToContainingFolder =

"\\".join(!FullFilePathWithFileName!.split("\\")[0:-1])

so that

\Users\me\Desktop\New folder\file.txt

becomes

\Users\me\Desktop\New folder

Question 26

What would be the best way in Python to determine whether a directory is writeable for the user executing the script? Since this will likely involve using the os module I should mention I’m running it under a *nix environment.

Question 27

Although what Christophe suggested is a more Pythonic solution, the os module does have the os.access function to check access:

os.access('/path/to/folder', os.W_OK) # W_OK is for writing, R_OK for reading, etc.

Question 28

It may seem strange to suggest this, but a common Python idiom is

It’s easier to ask for forgiveness than for permission

Following that idiom, one might say:

Try writing to the directory in question, and catch the error if you don’t have the permission to do so.

Question 29

My solution using the tempfile module:

import tempfile
import errno

def isWritable(path):
    try:
        testfile = tempfile.TemporaryFile(dir = path)
        testfile.close()
    except OSError as e:
        if e.errno == errno.EACCES:  # 13
            return False
        e.filename = path
        raise
    return True

Update: After testing the code again on Windows I see that there is indeed an issue when using tempfile there, see issue22107: tempfile module misinterprets access denied error on Windows. In the case of a non-writable directory, the code hangs for several seconds and finally throws an IOError: [Errno 17] No usable temporary file name found. Maybe this is what user2171842 was observing? Unfortunately the issue is not resolved for now so to handle this, the error needs to be catched as well:

    except (OSError, IOError) as e:
        if e.errno == errno.EACCES or e.errno == errno.EEXIST:  # 13, 17

The delay is of course still present in these cases then.

Question 30

Stumbled across this thread searching for examples for someone. First result on Google, congrats!

People talk about the Pythonic way of doing it in this thread, but no simple code examples? Here you go, for anyone else who stumbles in:

import sys

filepath = 'C:\\path\\to\\your\\file.txt'

try:
    filehandle = open( filepath, 'w' )
except IOError:
    sys.exit( 'Unable to write to file ' + filepath )

filehandle.write("I am writing this text to the file\n")

This attempts to open a filehandle for writing, and exits with an error if the file specified cannot be written to: This is far easier to read, and is a much better way of doing it rather than doing prechecks on the file path or the directory, as it avoids race conditions; cases where the file becomes unwriteable between the time you run the precheck, and when you actually attempt to write to the file.

Question 31

If you only care about the file perms, os.access(path, os.W_OK) should do what you ask for. If you instead want to know whether you can write to the directory, open() a test file for writing (it shouldn’t exist beforehand), catch and examine any IOError, and clean up the test file afterwards.

More generally, to avoid TOCTOU attacks (only a problem if your script runs with elevated privileges — suid or cgi or so), you shouldn’t really trust these ahead-of-time tests, but drop privs, do the open(), and expect the IOError.

Question 32

Check the mode bits:

def isWritable(name):
  uid = os.geteuid()
  gid = os.getegid()
  s = os.stat(dirname)
  mode = s[stat.ST_MODE]
  return (
     ((s[stat.ST_UID] == uid) and (mode & stat.S_IWUSR)) or
     ((s[stat.ST_GID] == gid) and (mode & stat.S_IWGRP)) or
     (mode & stat.S_IWOTH)
     )

Question 33

Here is something I created based on ChristopheD’s answer:

import os

def isWritable(directory):
    try:
        tmp_prefix = "write_tester";
        count = 0
        filename = os.path.join(directory, tmp_prefix)
        while(os.path.exists(filename)):
            filename = "{}.{}".format(os.path.join(directory, tmp_prefix),count)
            count = count + 1
        f = open(filename,"w")
        f.close()
        os.remove(filename)
        return True
    except Exception as e:
        #print "{}".format(e)
        return False

directory = "c:\\"
if (isWritable(directory)):
    print "directory is writable"
else:
    print "directory is not writable"

Question 34

 if os.access(path_to_folder, os.W_OK) is not True:
            print("Folder not writable")
 else :
            print("Folder writable")

more info about access can be find it here

Question 35

I ran into this same need while adding an argument via argparse. The built in type=FileType('w') wouldn’t work for me as I was looking for a directory. I ended up writing my own method to solve my problem. Here is the result with argparse snippet.

#! /usr/bin/env python
import os
import argparse

def writable_dir(dir):
    if os.access(dir, os.W_OK) and os.path.isdir(dir):
        return os.path.abspath(dir)
    else:
        raise argparse.ArgumentTypeError(dir + " is not writable or does not exist.")

parser = argparse.ArgumentParser()
parser.add_argument("-d","--dir", type=writable_dir(), default='/tmp/',
    help="Directory to use. Default: /tmp")
opts = parser.parse_args()

That results in the following:

$ python dir-test.py -h
usage: dir-test.py [-h] [-d DIR]

optional arguments:
  -h, --help         show this help message and exit
  -d DIR, --dir DIR  Directory to use. Default: /tmp

$ python dir-test.py -d /not/real
usage: dir-test.py [-h] [-d DIR]
dir-test.py: error: argument -d/--dir: /not/real is not writable or does not exist.

$ python dir-test.py -d ~

I went back and added print opts.dir to the end, and everything appears to be functioning as desired.

Question 36

If you need to check the permission of another user (yes, I realize this contradicts the question, but may come in handy for someone), you can do it through the pwd module, and the directory’s mode bits.

Disclaimer – does not work on Windows, as it doesn’t use the POSIX permissions model (and the pwd module is not available there), e.g. – solution only for *nix systems.

Note that a directory has to have all the 3 bits set – Read, Write and eXecute.
Ok, R is not an absolute must, but w/o it you cannot list the entries in the directory (so you have to know their names). Execute on the other hand is absolutely needed – w/o it the user cannot read the file’s inodes; so even having W, without X files cannot be created or modified. More detailed explanation at this link.

Finally, the modes are available in the stat module, their descriptions are in inode(7) man.

Sample code how to check:

import pwd
import stat
import os

def check_user_dir(user, directory):
    dir_stat = os.stat(directory)

    user_id, group_id = pwd.getpwnam(user).pw_uid, pwd.getpwnam(user).pw_gid
    directory_mode = dir_stat[stat.ST_MODE]

    # use directory_mode as mask 
    if user_id == dir_stat[stat.ST_UID] and stat.S_IRWXU & directory_mode == stat.S_IRWXU:     # owner and has RWX
        return True
    elif group_id == dir_stat[stat.ST_GID] and stat.S_IRWXG & directory_mode == stat.S_IRWXG:  # in group & it has RWX
        return True
    elif stat.S_IRWXO & directory_mode == stat.S_IRWXO:                                        # everyone has RWX
        return True

    # no permissions
    return False

Question 37

I need to extract the name of the parent directory of a certain path. This is what it looks like:

c:\stuff\directory_i_need\subdir\file

I am modifying the content of the “file” with something that uses the directory_i_need name in it (not the path). I have created a function that will give me a list of all the files, and then…

for path in file_list:
   #directory_name = os.path.dirname(path)   # this is not what I need, that's why it is commented
   directories, files = path.split('\\')

   line_replace_add_directory = line_replace + directories  
   # this is what I want to add in the text, with the directory name at the end 
   # of the line.

How can I do that?

Question 38

import os
## first file in current dir (with full path)
file = os.path.join(os.getcwd(), os.listdir(os.getcwd())[0])
file
os.path.dirname(file) ## directory of file
os.path.dirname(os.path.dirname(file)) ## directory of directory of file
...

And you can continue doing this as many times as necessary…

Edit: from os.path, you can use either os.path.split or os.path.basename:

dir = os.path.dirname(os.path.dirname(file)) ## dir of dir of file
## once you're at the directory level you want, with the desired directory as the final path node:
dirname1 = os.path.basename(dir) 
dirname2 = os.path.split(dir)[1] ## if you look at the documentation, this is exactly what os.path.basename does.

Question 39

In Python 3.4 you can use the pathlib module:

>>> from pathlib import Path
>>> p = Path('C:\Program Files\Internet Explorer\iexplore.exe')
>>> p.name
'iexplore.exe'
>>> p.suffix
'.exe'
>>> p.root
'\\'
>>> p.parts
('C:\\', 'Program Files', 'Internet Explorer', 'iexplore.exe')
>>> p.relative_to('C:\Program Files')
WindowsPath('Internet Explorer/iexplore.exe')
>>> p.exists()
True

Question 40

All you need is parent part if you use pathlib.

from pathlib import Path
p = Path(r'C:\Program Files\Internet Explorer\iexplore.exe')
print(p.parent)

Will output:

C:\Program Files\Internet Explorer

Case you need all parts (already covered in other answers) use parts:

p = Path(r'C:\Program Files\Internet Explorer\iexplore.exe')
print(p.parts)

Then you will get a list:

('C:\\', 'Program Files', 'Internet Explorer', 'iexplore.exe')

Saves tone of time.

Question 41

First, see if you have splitunc() as an available function within os.path. The first item returned should be what you want… but I am on Linux and I do not have this function when I import os and try to use it.

Otherwise, one semi-ugly way that gets the job done is to use:

>>> pathname = "\\C:\\mystuff\\project\\file.py"
>>> pathname
'\\C:\\mystuff\\project\\file.py'
>>> print pathname
\C:\mystuff\project\file.py
>>> "\\".join(pathname.split('\\')[:-2])
'\\C:\\mystuff'
>>> "\\".join(pathname.split('\\')[:-1])
'\\C:\\mystuff\\project'

which shows retrieving the directory just above the file, and the directory just above that.

Question 42

This is what I did to extract the piece of the directory:

for path in file_list:
  directories = path.rsplit('\\')
  directories.reverse()
  line_replace_add_directory = line_replace+directories[2]

Thank you for your help.

Question 43

import os

directory = os.path.abspath('\\') # root directory
print(directory) # e.g. 'C:\'

directory = os.path.abspath('.') # current directory
print(directory) # e.g. 'C:\Users\User\Desktop'

parent_directory, directory_name = os.path.split(directory)
print(directory_name) # e.g. 'Desktop'
parent_parent_directory, parent_directory_name = os.path.split(parent_directory)
print(parent_directory_name) # e.g. 'User'

This should also do the trick.

Question 44

You have to put the entire path as a parameter to os.path.split. See The docs. It doesn’t work like string split.

Question 45

I need to iterate through the subdirectories of a given directory and search for files. If I get a file I have to open it and change the content and replace it with my own lines.

I tried this:

import os

rootdir ='C:/Users/sid/Desktop/test'

for subdir, dirs, files in os.walk(rootdir):
    for file in files:
        f=open(file,'r')
        lines=f.readlines()
        f.close()
        f=open(file,'w')
        for line in lines:
            newline = "No you are not"
            f.write(newline)
        f.close()

but I am getting an error. What am I doing wrong?

Question 46

The actual walk through the directories works as you have coded it. If you replace the contents of the inner loop with a simple print statement you can see that each file is found:

import os
rootdir = 'C:/Users/sid/Desktop/test'

for subdir, dirs, files in os.walk(rootdir):
    for file in files:
        print os.path.join(subdir, file)

If you still get errors when running the above, please provide the error message.

Updated for Python3

import os
rootdir = 'C:/Users/sid/Desktop/test'

for subdir, dirs, files in os.walk(rootdir):
    for file in files:
        print(os.path.join(subdir, file))

Question 47

Another way of returning all files in subdirectories is to use the pathlib module, introduced in Python 3.4, which provides an object oriented approach to handling filesystem paths (Pathlib is also available on Python 2.7 via the pathlib2 module on PyPi):

from pathlib import Path

rootdir = Path('C:/Users/sid/Desktop/test')
# Return a list of regular files only, not directories
file_list = [f for f in rootdir.glob('**/*') if f.is_file()]

# For absolute paths instead of relative the current dir
file_list = [f for f in rootdir.resolve().glob('**/*') if f.is_file()]

Since Python 3.5, the glob module also supports recursive file finding:

import os
from glob import iglob

rootdir_glob = 'C:/Users/sid/Desktop/test/**/*' # Note the added asterisks
# This will return absolute paths
file_list = [f for f in iglob('**/*', recursive=True) if os.path.isfile(f)]

The file_list from either of the above approaches can be iterated over without the need for a nested loop:

for f in file_list:
    print(f) # Replace with desired operations

Question 48

From python >= 3.5 onward, you can use **, glob.iglob(path/**, recursive=True) and it seems the most pythonic solution, i.e.:

import glob, os

for filename in glob.iglob('/pardadox-music/**', recursive=True):
    if os.path.isfile(filename): # filter dirs
        print(filename)

Output:

/pardadox-music/modules/her1.mod
/pardadox-music/modules/her2.mod
...

Notes:
1 – glob.iglob

glob.iglob(pathname, recursive=False)

Return an iterator which yields the same values as glob() without actually storing them all simultaneously.

2 – If recursive is True, the pattern '**' will match any files and zero or more directories and subdirectories.

3 – If the directory contains files starting with . they won’t be matched by default. For example, consider a directory containing card.gif and .card.gif:

>>> import glob
>>> glob.glob('*.gif') ['card.gif'] 
>>> glob.glob('.c*')['.card.gif']

4 – You can also use rglob(pattern), which is the same as calling glob() with **/ added in front of the given relative pattern.

Question 49

I used to open files that were in the same directory as the currently running Python script by simply using a command like

open("Some file.txt", "r")

However, I discovered that when the script was run in Windows by double-clicking it, it would try to open the file from the wrong directory.

Since then I’ve used a command of the form

open(os.path.join(sys.path[0], "Some file.txt"), "r")

whenever I wanted to open a file. This works for my particular usage, but I’m not sure if sys.path[0] might fail in some other use case.

So my question is: What is the best and most reliable way to open a file that’s in the same directory as the currently running Python script?

Here’s what I’ve been able to figure out so far:

os.getcwd() and os.path.abspath('') return the “current working directory”, not the script directory.
os.path.dirname(sys.argv[0]) and os.path.dirname(__file__) return the path used to call the script, which may be relative or even blank (if the script is in the cwd). Also, __file__ does not exist when the script is run in IDLE or PythonWin.
sys.path[0] and os.path.abspath(os.path.dirname(sys.argv[0])) seem to return the script directory. I’m not sure if there’s any difference between these two.

Edit:

I just realized that what I want to do would be better described as “open a file in the same directory as the containing module”. In other words, if I import a module I wrote that’s in another directory, and that module opens a file, I want it to look for the file in the module’s directory. I don’t think anything I’ve found is able to do that…

Question 50

I always use:

__location__ = os.path.realpath(
    os.path.join(os.getcwd(), os.path.dirname(__file__)))

The join() call prepends the current working directory, but the documentation says that if some path is absolute, all other paths left of it are dropped. Therefore, getcwd() is dropped when dirname(__file__) returns an absolute path.

Also, the realpath call resolves symbolic links if any are found. This avoids troubles when deploying with setuptools on Linux systems (scripts are symlinked to /usr/bin/ — at least on Debian).

You may the use the following to open up files in the same folder:

f = open(os.path.join(__location__, 'bundled-resource.jpg'))
# ...

I use this to bundle resources with several Django application on both Windows and Linux and it works like a charm!

Question 51

To quote from the Python documentation:

As initialized upon program startup, the first item of this list, path[0], is the directory containing the script that was used to invoke the Python interpreter. If the script directory is not available (e.g. if the interpreter is invoked interactively or if the script is read from standard input), path[0] is the empty string, which directs Python to search modules in the current directory first. Notice that the script directory is inserted before the entries inserted as a result of PYTHONPATH.

sys.path[0] is what you are looking for.

Question 52

Ok here is what I do

sys.argv is always what you type into the terminal or use as the file path when executing it with python.exe or pythonw.exe

For example you can run the file text.py several ways, they each give you a different answer they always give you the path that python was typed.

    C:\Documents and Settings\Admin>python test.py
    sys.argv[0]: test.py
    C:\Documents and Settings\Admin>python "C:\Documents and Settings\Admin\test.py"
    sys.argv[0]: C:\Documents and Settings\Admin\test.py

Ok so know you can get the file name, great big deal, now to get the application directory you can know use os.path, specifically abspath and dirname

    import sys, os
    print os.path.dirname(os.path.abspath(sys.argv[0]))

That will output this:

   C:\Documents and Settings\Admin\

it will always output this no matter if you type python test.py or python “C:\Documents and Settings\Admin\test.py”

The problem with using __file__ Consider these two files test.py

import sys
import os

def paths():
        print "__file__: %s" % __file__
        print "sys.argv: %s" % sys.argv[0]

        a_f = os.path.abspath(__file__)
        a_s = os.path.abspath(sys.argv[0])

        print "abs __file__: %s" % a_f
        print "abs sys.argv: %s" % a_s

if __name__ == "__main__":
    paths()

import_test.py

import test
import sys

test.paths()

print "--------"
print __file__
print sys.argv[0]

Output of “python test.py”

C:\Documents and Settings\Admin>python test.py
__file__: test.py
sys.argv: test.py
abs __file__: C:\Documents and Settings\Admin\test.py
abs sys.argv: C:\Documents and Settings\Admin\test.py

Output of “python test_import.py”

C:\Documents and Settings\Admin>python test_import.py
__file__: C:\Documents and Settings\Admin\test.pyc
sys.argv: test_import.py
abs __file__: C:\Documents and Settings\Admin\test.pyc
abs sys.argv: C:\Documents and Settings\Admin\test_import.py
--------
test_import.py
test_import.py

So as you can see file gives you always the python file it is being run from, where as sys.argv[0] gives you the file that you ran from the interpreter always. Depending on your needs you will need to choose which one best fits your needs.

Question 53

I was able to use the code provided by dcolish successfully as I was having a similar issue with reading a specific text file. The file is not in the same cwd as the Python file.

Question 54

I’d do it this way:

from os.path import abspath, exists

f_path = abspath("fooabar.txt")

if exists(f_path):
    with open(f_path) as f:
        print f.read()

The above code builds an absolute path to the file using abspath and is equivalent to using normpath(join(os.getcwd(), path)) [that’s from the pydocs]. It then checks if that file actually exists and then uses a context manager to open it so you don’t have to remember to call close on the file handle. IMHO, doing it this way will save you a lot of pain in the long run.

Question 55

I want to inherit from a class in a file that lies in a directory above the current one.

Is it possible to relatively import that file?

Question 56

from ..subpkg2 import mod

Per the Python docs: When inside a package hierarchy, use two dots, as the import statement doc says:

When specifying what module to import you do not have to specify the absolute name of the module. When a module or package is contained within another package it is possible to make a relative import within the same top package without having to mention the package name. By using leading dots in the specified module or package after from you can specify how high to traverse up the current package hierarchy without specifying exact names. One leading dot means the current package where the module making the import exists. Two dots means up one package level. Three dots is up two levels, etc. So if you execute from . import mod from a module in the pkg package then you will end up importing pkg.mod. If you execute from ..subpkg2 import mod from within pkg.subpkg1 you will import pkg.subpkg2.mod. The specification for relative imports is contained within PEP 328.

PEP 328 deals with absolute/relative imports.

Question 57

import sys
sys.path.append("..") # Adds higher directory to python modules path.

Question 58

@gimel’s answer is correct if you can guarantee the package hierarchy he mentions. If you can’t — if your real need is as you expressed it, exclusively tied to directories and without any necessary relationship to packaging — then you need to work on __file__ to find out the parent directory (a couple of os.path.dirname calls will do;-), then (if that directory is not already on sys.path) prepend temporarily insert said dir at the very start of sys.path, __import__, remove said dir again — messy work indeed, but, “when you must, you must” (and Pyhon strives to never stop the programmer from doing what must be done — just like the ISO C standard says in the “Spirit of C” section in its preface!-).

Here is an example that may work for you:

import sys
import os.path
sys.path.append(
    os.path.abspath(os.path.join(os.path.dirname(__file__), os.path.pardir)))

import module_in_parent_dir

Question 59

Import module from a directory which is exactly one level above the current directory:

from .. import module

Question 60

How to load a module that is a directory up

preface: I did a substantial rewrite of a previous answer with the hopes of helping ease people into python’s ecosystem, and hopefully give everyone the best change of success with python’s import system.

This will cover relative imports within a package, which I think is the most probable case to OP’s question.

Python is a modular system

This is why we write import foo to load a module “foo” from the root namespace, instead of writing:

foo = dict();  # please avoid doing this
with open(os.path.join(os.path.dirname(__file__), '../foo.py') as foo_fh:  # please avoid doing this
    exec(compile(foo_fh.read(), 'foo.py', 'exec'), foo)  # please avoid doing this

Python isn’t coupled to a file-system

This is why we can embed python in environment where there isn’t a defacto filesystem without providing a virtual one, such as Jython.

Being decoupled from a filesystem lets imports be flexible, this design allows for things like imports from archive/zip files, import singletons, bytecode caching, cffi extensions, even remote code definition loading.

So if imports are not coupled to a filesystem what does “one directory up” mean? We have to pick out some heuristics but we can do that, for example when working within a package, some heuristics have already been defined that makes relative imports like .foo and ..foo work within the same package. Cool!

If you sincerely want to couple your source code loading patterns to a filesystem, you can do that. You’ll have to choose your own heuristics, and use some kind of importing machinery, I recommend importlib

Python’s importlib example looks something like so:

import importlib.util
import sys

# For illustrative purposes.
file_path = os.path.join(os.path.dirname(__file__), '../foo.py')
module_name = 'foo'

foo_spec = importlib.util.spec_from_file_location(module_name, file_path)
# foo_spec is a ModuleSpec specifying a SourceFileLoader
foo_module = importlib.util.module_from_spec(foo_spec)
sys.modules[module_name] = foo_module
foo_spec.loader.exec_module(foo_module)

foo = sys.modules[module_name]
# foo is the sys.modules['foo'] singleton

Packaging

There is a great example project available officially here: https://github.com/pypa/sampleproject

A python package is a collection of information about your source code, that can inform other tools how to copy your source code to other computers, and how to integrate your source code into that system’s path so that import foo works for other computers (regardless of interpreter, host operating system, etc)

Directory Structure

Lets have a package name foo, in some directory (preferably an empty directory).

some_directory/
    foo.py  # `if __name__ == "__main__":`  lives here

My preference is to create setup.py as sibling to foo.py, because it makes writing the setup.py file simpler, however you can write configuration to change/redirect everything setuptools does by default if you like; for example putting foo.py under a “src/” directory is somewhat popular, not covered here.

some_directory/
    foo.py
    setup.py

.

#!/usr/bin/env python3
# setup.py

import setuptools

setuptools.setup(
    name="foo",
    ...
    py_modules=['foo'],
)

.

python3 -m pip install --editable ./  # or path/to/some_directory/

“editable” aka -e will yet-again redirect the importing machinery to load the source files in this directory, instead copying the current exact files to the installing-environment’s library. This can also cause behavioral differences on a developer’s machine, be sure to test your code! There are tools other than pip, however I’d recommend pip be the introductory one :)

I also like to make foo a “package” (a directory containing __init__.py) instead of a module (a single “.py” file), both “packages” and “modules” can be loaded into the root namespace, modules allow for nested namespaces, which is helpful if we want to have a “relative one directory up” import.

some_directory/
    foo/
        __init__.py
    setup.py

.

#!/usr/bin/env python3
# setup.py

import setuptools

setuptools.setup(
    name="foo",
    ...
    packages=['foo'],
)

I also like to make a foo/__main__.py, this allows python to execute the package as a module, eg python3 -m foo will execute foo/__main__.py as __main__.

some_directory/
    foo/
        __init__.py
        __main__.py  # `if __name__ == "__main__":`  lives here, `def main():` too!
    setup.py

.

#!/usr/bin/env python3
# setup.py

import setuptools

setuptools.setup(
    name="foo",
    ...
    packages=['foo'],
    ...
    entry_points={
        'console_scripts': [
            # "foo" will be added to the installing-environment's text mode shell, eg `bash -c foo`
            'foo=foo.__main__:main',
        ]
    },
)

Lets flesh this out with some more modules: Basically, you can have a directory structure like so:

some_directory/
    bar.py           # `import bar`
    foo/
        __init__.py  # `import foo`
        __main__.py
        baz.py       # `import foo.baz
        spam/           
            __init__.py  # `import foo.spam`
            eggs.py      # `import foo.spam.eggs`
    setup.py

setup.py conventionally holds metadata information about the source code within, such as:

what dependencies are needed to install named “install_requires”
what name should be used for package management (install/uninstall “name”), I suggest this match your primary python package name in our case foo, though substituting underscores for hyphens is popular
licensing information
maturity tags (alpha/beta/etc),
audience tags (for developers, for machine learning, etc),
single-page documentation content (like a README),
shell names (names you type at user shell like bash, or names you find in a graphical user shell like a start menu),
a list of python modules this package will install (and uninstall)
a defacto “run tests” entry point python ./setup.py test

Its very expansive, it can even compile c extensions on the fly if a source module is being installed on a development machine. For a every-day example I recommend the PYPA Sample Repository’s setup.py

If you are releasing a build artifact, eg a copy of the code that is meant to run nearly identical computers, a requirements.txt file is a popular way to snapshot exact dependency information, where “install_requires” is a good way to capture minimum and maximum compatible versions. However, given that the target machines are nearly identical anyway, I highly recommend creating a tarball of an entire python prefix. This can be tricky, too detailed to get into here. Check out pip install‘s --target option, or virtualenv aka venv for leads.

back to the example

how to import a file one directory up:

From foo/spam/eggs.py, if we wanted code from foo/baz we could ask for it by its absolute namespace:

import foo.baz

If we wanted to reserve capability to move eggs.py into some other directory in the future with some other relative baz implementation, we could use a relative import like:

import ..baz

Question 61

Python is a modular system

Python doesn’t rely on a file system

To load python code reliably, have that code in a module, and that module installed in python’s library.

Installed modules can always be loaded from the top level namespace with import <name>

There is a great sample project available officially here: https://github.com/pypa/sampleproject

Basically, you can have a directory structure like so:

the_foo_project/
    setup.py  

    bar.py           # `import bar`
    foo/
      __init__.py    # `import foo`

      baz.py         # `import foo.baz`

      faz/           # `import foo.faz`
        __init__.py
        daz.py       # `import foo.faz.daz` ... etc.

.

Be sure to declare your setuptools.setup() in setup.py,

official example: https://github.com/pypa/sampleproject/blob/master/setup.py

In our case we probably want to export bar.py and foo/__init__.py, my brief example:

setup.py

#!/usr/bin/env python3

import setuptools

setuptools.setup(
    ...
    py_modules=['bar'],
    packages=['foo'],
    ...
    entry_points={}, 
        # Note, any changes to your setup.py, like adding to `packages`, or
        # changing `entry_points` will require the module to be reinstalled;
        # `python3 -m pip install --upgrade --editable ./the_foo_project
)

.

Now we can install our module into the python library; with pip, you can install the_foo_project into your python library in edit mode, so we can work on it in real time

python3 -m pip install --editable=./the_foo_project

# if you get a permission error, you can always use 
# `pip ... --user` to install in your user python library

.

Now from any python context, we can load our shared py_modules and packages

foo_script.py

#!/usr/bin/env python3

import bar
import foo

print(dir(bar))
print(dir(foo))

Question 62

I’m having a problem with deleting empty directories. Here is my code:

for dirpath, dirnames, filenames in os.walk(dir_to_search):
    //other codes

    try:
        os.rmdir(dirpath)
    except OSError as ex:
        print(ex)

The argument dir_to_search is where I’m passing the directory where the work needs to be done. That directory looks like this:

test/20/...
test/22/...
test/25/...
test/26/...

Note that all the above folders are empty. When I run this script the folders 20,25 alone gets deleted! But the folders 25 and 26 aren’t deleted, even though they are empty folders.

Edit:

The exception that I’m getting are:

[Errno 39] Directory not empty: '/home/python-user/shell-scripts/s3logs/test'
[Errno 39] Directory not empty: '/home/python-user/shell-scripts/s3logs/test/2012'
[Errno 39] Directory not empty: '/home/python-user/shell-scripts/s3logs/test/2012/10'
[Errno 39] Directory not empty: '/home/python-user/shell-scripts/s3logs/test/2012/10/29'
[Errno 39] Directory not empty: '/home/python-user/shell-scripts/s3logs/test/2012/10/29/tmp'
[Errno 39] Directory not empty: '/home/python-user/shell-scripts/s3logs/test/2012/10/28'
[Errno 39] Directory not empty: '/home/python-user/shell-scripts/s3logs/test/2012/10/28/tmp'
[Errno 39] Directory not empty: '/home/python-user/shell-scripts/s3logs/test/2012/10/26'
[Errno 39] Directory not empty: '/home/python-user/shell-scripts/s3logs/test/2012/10/25'
[Errno 39] Directory not empty: '/home/python-user/shell-scripts/s3logs/test/2012/10/27'
[Errno 39] Directory not empty: '/home/python-user/shell-scripts/s3logs/test/2012/10/27/tmp'

Where am I making a mistake?

Question 63

Try shutil.rmtree:

import shutil
shutil.rmtree('/path/to/your/dir/')

Question 64

The default behavior of os.walk() is to walk from root to leaf. Set topdown=False in os.walk() to walk from leaf to root.

问题：如何获取按python创建日期排序的目录列表？

回答 0

回答 1

回答 2

回答 3

回答 4

回答 5

回答 6

回答 7

回答 8

回答 9

回答 10

回答 11

回答 12

回答 13

回答 14

回答 15

回答 16

问题：如何从Python的文件路径中提取文件夹路径？

回答 0

回答 1

回答 2

回答 3

回答 4

回答 5

问题：确定目录是否可写

回答 0

回答 1

回答 2

回答 3

回答 4

回答 5

回答 6

回答 7

回答 8

回答 9

问题：在Python中提取一部分文件路径（目录）

回答 0

回答 1

回答 2

回答 3

回答 4

回答 5

回答 6

问题：使用Python遍历目录

回答 0

回答 1

回答 2

问题：如何可靠地在与Python脚本相同的目录中打开文件

回答 0

回答 1

回答 2

回答 3

回答 4

问题：如何导入上面目录中的Python类？

回答 0

回答 1

回答 2

回答 3

回答 4

如何加载目录中的模块

Python是一个模块化系统

Python未与文件系统耦合

打包

目录结构

如何在一个目录下导入文件：

How to load a module that is a directory up

Python is a modular system

Python isn’t coupled to a file-system

Packaging

Directory Structure

how to import a file one directory up:

回答 5

Python是一个模块化系统

Python不依赖文件系统

setup.py

foo_script.py

Python is a modular system

Python doesn’t rely on a file system

setup.py

文件

file

没有`sys.path`黑客的解决方案

1）将A添加`setup.py`到根文件夹

4）通过`mainfolder`在每次导入之前进行导入