Python 实用宝典

Question 1

How can I import variables from one file to another?

example: file1 has the variables x1 and x2 how to pass them to file2?

How can I import all of the variables from one to another?

Question 2

from file1 import *

will import all objects and methods in file1

Question 3

Import file1 inside file2:

To import all variables from file1 without flooding file2’s namespace, use:

import file1

#now use file1.x1, file2.x2, ... to access those variables

To import all variables from file1 to file2’s namespace( not recommended):

from file1 import *
#now use x1, x2..

From the docs:

While it is valid to use from module import * at module level it is usually a bad idea. For one, this loses an important property Python otherwise has — you can know where each toplevel name is defined by a simple “search” function in your favourite editor. You also open yourself to trouble in the future, if some module grows additional functions or classes.

Question 4

Best to import x1 and x2 explicitly:

from file1 import x1, x2

This allows you to avoid unnecessary namespace conflicts with variables and functions from file1 while working in file2.

But if you really want, you can import all the variables:

from file1 import *

Question 5

Actually this is not really the same to import a variable with:

from file1 import x1
print(x1)

and

import file1
print(file1.x1)

Altough at import time x1 and file1.x1 have the same value, they are not the same variables. For instance, call a function in file1 that modifies x1 and then try to print the variable from the main file: you will not see the modified value.

Question 6

Marc response is correct. Actually, you can print the memory address for the variables print(hex(id(libvar)) and you can see the addresses are different.

# mylib.py
libvar = None
def lib_method():
    global libvar
    print(hex(id(libvar)))

# myapp.py
from mylib import libvar, lib_method
import mylib

lib_method()
print(hex(id(libvar)))
print(hex(id(mylib.libvar)))

Question 7

script1.py

title="Hello world"

script2.py is where we using script1 variable

Method 1:

import script1
print(script1.title)

Method 2:

from script1 import title
print(title)

Question 8

In Python you can access the contents of other files like as if they
are some kind of a library, compared to other languages like java or any oop base languages , This is really cool ;

This makes accessing the contents of the file or import it to to process it or to do anything with it ; And that is the Main reason why Python is highly preferred Language for Data Science and Machine Learning etc. ;

And this is the picture of project structure

Where I am accessing variables from .env file where the API links and Secret keys reside .

General Structure:

from <File-Name> import *

Question 9

first.py:

a=5

second.py:

import first
print(first.a)

The result will be 5.

Question 10

Situation: – There is a module in my project_folder called calendar – I would like to use the built-in Calendar class from the Python libraries – When I use from calendar import Calendar it complains because it’s trying to load from my module.

I’ve done a few searches and I can’t seem to find a solution to my problem.

Any ideas without having to rename my module?

Question 11

The accepted solution contains a now-deprecated approach.

The importlib documentation here gives a good example of the more appropriate way to load a module directly from a file path for python >= 3.5:

import importlib.util
import sys

# For illustrative purposes.
import tokenize
file_path = tokenize.__file__  # returns "/path/to/tokenize.py"
module_name = tokenize.__name__  # returns "tokenize"

spec = importlib.util.spec_from_file_location(module_name, file_path)
module = importlib.util.module_from_spec(spec)
sys.modules[module_name] = module
spec.loader.exec_module(module)

So, you can load any .py file from a path and set the module name to be whatever you want. So just adjust the module_name to be whatever custom name you’d like the module to have upon importing.

To load a package instead of a single file, file_path should be the path to the package’s root __init__.py

Question 12

Changing the name of your module is not necessary. Rather, you can use absolute_import to change the importing behavior. For example with stem/socket.py I import the socket module as follows:

from __future__ import absolute_import
import socket

This only works with Python 2.5 and above; it’s enabling behavior that is the default in Python 3.0 and higher. Pylint will complain about the code but it’s perfectly valid.

Question 13

Actually, solving this is rather easy, but the implementation will always be a bit fragile, because it depends python import mechanism’s internals and they are subject to change in future versions.

(the following code shows how to load both local and non-local modules and how they may coexist)

def import_non_local(name, custom_name=None):
    import imp, sys

    custom_name = custom_name or name

    f, pathname, desc = imp.find_module(name, sys.path[1:])
    module = imp.load_module(custom_name, f, pathname, desc)
    f.close()

    return module

# Import non-local module, use a custom name to differentiate it from local
# This name is only used internally for identifying the module. We decide
# the name in the local scope by assigning it to the variable calendar.
calendar = import_non_local('calendar','std_calendar')

# import local module normally, as calendar_local
import calendar as calendar_local

print calendar.Calendar
print calendar_local

The best solution, if possible, is to avoid naming your modules with the same name as standard-library or built-in module names.

Question 14

The only way to solve this problem is to hijack the internal import machinery yourself. This is not easy, and fraught with peril. You should avoid the grail shaped beacon at all costs because the peril is too perilous.

Rename your module instead.

If you want to learn how to hijack the internal import machinery, here is where you would go about finding out how to do this:

There are sometimes good reasons to get into this peril. The reason you give is not among them. Rename your module.

If you take the perilous path, one problem you will encounter is that when you load a module it ends up with an ‘official name’ so that Python can avoid ever having to parse the contents of that module ever again. A mapping of the ‘official name’ of a module to the module object itself can be found in sys.modules.

This means that if you import calendar in one place, whatever module is imported will be thought of as the module with the official name calendar and all other attempts to import calendar anywhere else, including in other code that’s part of the main Python library, will get that calendar.

It might be possible to design a customer importer using the imputil module in Python 2.x that caused modules loaded from certain paths to look up the modules they were importing in something other than sys.modules first or something like that. But that’s an extremely hairy thing to be doing, and it won’t work in Python 3.x anyway.

There is an extremely ugly and horrible thing you can do that does not involve hooking the import mechanism. This is something you should probably not do, but it will likely work. It turns your calendar module into a hybrid of the system calendar module and your calendar module. Thanks to Boaz Yaniv for the skeleton of the function I use. Put this at the beginning of your calendar.py file:

import sys

def copy_in_standard_module_symbols(name, local_module):
    import imp

    for i in range(0, 100):
        random_name = 'random_name_%d' % (i,)
        if random_name not in sys.modules:
            break
        else:
            random_name = None
    if random_name is None:
        raise RuntimeError("Couldn't manufacture an unused module name.")
    f, pathname, desc = imp.find_module(name, sys.path[1:])
    module = imp.load_module(random_name, f, pathname, desc)
    f.close()
    del sys.modules[random_name]
    for key in module.__dict__:
        if not hasattr(local_module, key):
            setattr(local_module, key, getattr(module, key))

copy_in_standard_module_symbols('calendar', sys.modules[copy_in_standard_module_symbols.__module__])

Question 15

I’d like to offer my version, which is a combination of Boaz Yaniv’s and Omnifarious’s solution. It will import the system version of a module, with two main differences from the previous answers:

Supports the ‘dot’ notation, eg. package.module
Is a drop-in replacement for the import statement on system modules, meaning you just have to replace that one line and if there are already calls being made to the module they will work as-is

Put this somewhere accessible so you can call it (I have mine in my __init__.py file):

class SysModule(object):
    pass

def import_non_local(name, local_module=None, path=None, full_name=None, accessor=SysModule()):
    import imp, sys, os

    path = path or sys.path[1:]
    if isinstance(path, basestring):
        path = [path]

    if '.' in name:
        package_name = name.split('.')[0]
        f, pathname, desc = imp.find_module(package_name, path)
        if pathname not in __path__:
            __path__.insert(0, pathname)
        imp.load_module(package_name, f, pathname, desc)
        v = import_non_local('.'.join(name.split('.')[1:]), None, pathname, name, SysModule())
        setattr(accessor, package_name, v)
        if local_module:
            for key in accessor.__dict__.keys():
                setattr(local_module, key, getattr(accessor, key))
        return accessor
    try:
        f, pathname, desc = imp.find_module(name, path)
        if pathname not in __path__:
            __path__.insert(0, pathname)
        module = imp.load_module(name, f, pathname, desc)
        setattr(accessor, name, module)
        if local_module:
            for key in accessor.__dict__.keys():
                setattr(local_module, key, getattr(accessor, key))
            return module
        return accessor
    finally:
        try:
            if f:
                f.close()
        except:
            pass

Example

I wanted to import mysql.connection, but I had a local package already called mysql (the official mysql utilities). So to get the connector from the system mysql package, I replaced this:

import mysql.connector

With this:

import sys
from mysql.utilities import import_non_local         # where I put the above function (mysql/utilities/__init__.py)
import_non_local('mysql.connector', sys.modules[__name__])

Result

# This unmodified line further down in the file now works just fine because mysql.connector has actually become part of the namespace
self.db_conn = mysql.connector.connect(**parameters)

Question 16

Change the import path:

import sys
save_path = sys.path[:]
sys.path.remove('')
import calendar
sys.path = save_path

Question 17

I am working on a package in Python. I use virtualenv. I set the path to the root of the module in a .pth path in my virtualenv, so that I can import modules of the package while developing the code and do testing (Question 1: is it a good way to do?). This works fine (here is an example, this is the behavior I want):

(VEnvTestRc) zz@zz:~/Desktop/GitFolders/rc$ python
Python 2.7.12 (default, Jul  1 2016, 15:12:24) 
[GCC 5.4.0 20160609] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> from rc import ns
>>> exit()
(VEnvTestRc) zz@zz:~/Desktop/GitFolders/rc$ python tests/test_ns.py 
issued command: echo hello
command output: hello

However, if I try to use PyTest, I get some import error messages:

(VEnvTestRc) zz@zz:~/Desktop/GitFolders/rc$ pytest
=========================================== test session starts ============================================
platform linux2 -- Python 2.7.12, pytest-3.0.5, py-1.4.31, pluggy-0.4.0
rootdir: /home/zz/Desktop/GitFolders/rc, inifile: 
collected 0 items / 1 errors 

================================================== ERRORS ==================================================
________________________________ ERROR collecting tests/test_ns.py ________________________________
ImportError while importing test module '/home/zz/Desktop/GitFolders/rc/tests/test_ns.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
tests/test_ns.py:2: in <module>
    from rc import ns
E   ImportError: cannot import name ns
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! Interrupted: 1 errors during collection !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
========================================= 1 error in 0.09 seconds ==========================================
(VEnvTestRc) zz@zz:~/Desktop/GitFolders/rc$ which pytest
/home/zz/Desktop/VirtualEnvs/VEnvTestRc/bin/pytest

I am a bit puzzled, it looks like this indicates an import error, but Python does it fine so why is there a problem specifically with PyTest? Any suggestion to the reason / remedy (Question 2)? I googled and stack-overflowed the ‘ImportError: cannot import’ error for PyTest, but the hits I got were related to missing python path and remedy to this, which does not seem to be the problem here. Any suggestions?

Question 18

Found the answer:

DO NOT put a __init__.py file in a folder containing TESTS if you plan on using pytest. I had one such file, deleting it solved the problem.

This was actually buried in the comments to the second answer of PATH issue with pytest ‘ImportError: No module named YadaYadaYada’ so I did not see it, hope it gets more visibility here.

Question 19

I can’t say I understand why this works, but I had the same problem and the tests work fine if I run python -m pytest.

I’m in a virtualenv, with pytest also available globally:

(proj)tom@neon ~/dev/proj$ type -a python
python is /home/tom/.virtualenvs/proj/bin/python
python is /usr/bin/python

(proj)tom@neon ~/dev/proj$ python -V
Python 3.5.2

(proj)tom@neon ~/dev/proj$ type -a pytest
pytest is /home/tom/.virtualenvs/proj/bin/pytest
pytest is /usr/bin/pytest

(proj)tom@neon ~/dev/proj$ pytest --version
This is pytest version 3.5.0, imported from /home/tom/.virtualenvs/proj/lib/python3.5/site-packages/pytest.py

Question 20

I just solved this by removing the __init__.py in my project root:

.
├── __init__.py <--- removed
├── models
│   ├── __init__.py
│   ├── address.py
│   ├── appointment.py
│   └── client.py
├── requirements.txt
├── setup.cfg
├── tests
│   ├── __init__.py
│   ├── models
│   │   ├── __init__.py
│   │   ├── appointment_test.py
│   │   └── client_test.py
│   └── other_test.py
└── script.py

Question 21

I had the same problem but for another reason than the ones mentioned:

I had py.test installed globally, while the packages were installed in a virtual environment.

The solution was to install pytest in the virtual environment. (In case your shell hashes executables, as Bash does, use hash -r, or use the full path to py.test)

Question 22

This problem will happen if you have a tests.py file and a tests folder with tests/__init__.py.

During the collection pytest finds the folder, but when it tries to import the test files from the folder, tests.py file will cause the import problem.

To fix simply remove the tests.py file and put all your tests inside the tests/ folder.

For your specific case the fix will be precisely:

Remove the file /home/zz/Desktop/GitFolders/rc/tests.py
Make sure /home/zz/Desktop/GitFolders/rc/tests/__init__.py is present

Question 23

I had a similar issue, exact same error, but different cause. I was running the test code just fine, but against an old version of the module. In the previous version of my code one class existed, while the other did not. After updating my code, I should have run the following to install it.

sudo pip install ./ --upgrade

Upon installing the updated module running pytest produced the correct results (because i was using the correct code base).

Question 24

In my case, the import error occurred because the package is pointing to another package/directory with the same name and its path is one level above the folder I actually wanted. I think this also explains why some people need to remove _ init _.py while others need to add back.

I just put print(the_root_package.__path__) (after import the_root_package) in both python console and pytest scripts to compare the difference

BOTTOM LINE: When you do python, the package you import may be different from the package when you run pytest.

Question 25

Install the packages into Your virtual environment.
Then start a new shell and source Your virtual environment again.

Question 26

The answer above not work for me. I just solved it by appending the absolute path of the module which not found to the sys.path at top of the test_xxx.py (your test module), like:

import sys
sys.path.append('path')

Question 27

If it is related to python code that was originally developed in python 2.7 and now migrated into python 3.x than the problem is probably related to an import issue.

e.g. when importing an object from a file: base that is located in the same directory this will work in python 2.x:

from base import MyClass

in python 3.x you should replace with base full path or .base not doing so will cause the above problem. so try:

from .base import MyClass

Question 28

I was experiencing this issue today and solved it by calling python -m pytest from the root of my project directory.

Calling pytest from the same location still caused issues.

My Project dir is organized as:

api/
 - server/
  - tests/
      - test_routes.py
  - routes/
      - routes.py
 - app.py

The module routes was imported in my test_routes.py as: from server.routes.routes import Routes

Hope that helps!

Question 29

Another special case:

I had the problem using tox. So my program ran fine, but unittests via tox kept complaining. After installing packages (needed for the program) you need to additionally specify the packages used in the unittests in the tox.ini

[testenv]
deps =
    package1
    package2 
...

Question 30

I was getting this using VSCode. I have a conda environment. I don’t think the VScode python extension could see the updates I was making.

python c:\Users\brig\.vscode\extensions\ms-python.python-2019.9.34911\pythonFiles\testing_tools\run_adapter.py discover pytest -- -s --cache-clear test
Test Discovery failed:

I had to run pip install ./ --upgrade

Question 31

Edit your conftest.py and add following lines of code:

import os, sys
sys.path.insert(0, os.path.abspath(os.path.join(os.path.dirname(file), '..')))

And if trying to run the test case through terminal, use following ex:

python -m pytest test_some_step_file_steps.py --html=HTML_step_file_output.html --self-contained-html

Question 32

Yet another massive win for Python’s import system. I think the reason there is no consensus is that what works probably depends on your environment and the tools you are using on top of it.

I’m using this from VS Code, in the test explorer under Windows in a conda environment, Python 3.8.

The setup I have got to work is:

mypkg/
    __init__.py
    app.py
    view.py
tests/
    test_app.py
    test_view.py

Under this setup intellisense works and so does test discovery.

Note that I originally tried the following, as recommended here.

src/
    mypkg/
        __init__.py
        app.py
        view.py
tests/
    test_app.py
    test_view.py

I could find no way of getting this to work from VS Code because the src folder just blew the mind of the import system. I can imagine there is a way of getting this to work from the command line. As a relatively new convert to Python programming it gives me a nostalgic feeling of working with COM, but being slightly less fun.

Question 33

My 2 cents on this: pytest will fail at chance if you are not using virtual environments. Sometimes it will just work, sometimes not.

Therefore, the solution is:

remove pytest with pip uninstall
create your venv
activate your venv
pip install your project path in editable mode, so it will be treated by pytest as a module (otherwise, pytest wont find your internal imports). You will need a setup.py file for that
install your packages, including pytest
finally, run your tests

The code, using windows PowerShell:

pip uninstall pytest
python.exe -m venv my_env
.\my_env\Scripts\activate
(my_env) pip install -e .
(my_env) pip install pytest pytest-html pandas numpy

Then finally

(my_env) pytest --html="my_testing_report.html"

An example of setup.py, for pip install -e:

import setuptools

setuptools.setup(
    name='my_package',
    version='devel',
    author='erickfis',
    author_email='erickfis@gmail.com',
    description='My package',
    long_description='My gooood package',
    packages=setuptools.find_packages(),
    classifiers=[
        'Programming Language :: Python :: 3',
        'Operating System :: OS Independent',
    ],
    include_package_data=True
)

Question 34

I disagree with the posts saying that you must remove any __init__.py files. What you must instead do is alter the sys.path.

Run an experiment where you print sys.path when running the code normally. Then print sys.path while running the code via pytest. I think you will find that there is a difference between these two paths, hence why pytest breaks.

To fix this, insert the path from the first experiment at the 0th index of the second.

Let '/usr/exampleUser/Documents/foo' be the first element of print(sys.path) for experiment 1.

Below is code that should fix your issue:

import sys sys.path[0] = '/usr/exampleUser/Documents/foo'

Put this at the top of your file, before your actual import statement.

Source: I was dealing with this myself and the above process solved it.

Question 35

Kept everything same and just added a blank test file at the root folder .. Solved it

Here are the findings, this problem really bugged me for a while. My folder structure was

mathapp/
    - server.py  
    - configuration.py 
    - __init__.py 
    - static/ 
       - home.html  
tests/            
    - functional 
       - test_errors.py 
    - unit  
       - test_add.py

and pytest would complain with the ModuleNotFoundError and gives the HINT – make sure your test modules/packages have valid Python names.

I introduced a mock test file at the same level as mathsapp and tests directory. The file contained nothing. Now pytest does not complain.

Result without the file

$ pytest
============================= test session starts =============================
platform win32 -- Python 3.8.2, pytest-5.4.2, py-1.8.1, pluggy-0.13.1
rootdir: C:\mak2006\workspace\0github\python-rest-app-cont
collected 1 item / 1 error

=================================== ERRORS ====================================
_______________ ERROR collecting tests/functional/test_func.py ________________
ImportError while importing test module 'C:\mainak\workspace\0github\python-rest-app-cont\tests\functional\test_func.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
tests\functional\test_func.py:4: in <module>
    from mathapp.service import sum
E   ModuleNotFoundError: No module named 'mathapp'
=========================== short test summary info ===========================
ERROR tests/functional/test_func.py
!!!!!!!!!!!!!!!!!!! Interrupted: 1 error during collection !!!!!!!!!!!!!!!!!!!!
============================== 1 error in 0.24s ===============================

Results with the file

$ pytest
============================= test session starts =============================
platform win32 -- Python 3.8.2, pytest-5.4.2, py-1.8.1, pluggy-0.13.1
rootdir: C:\mak2006\workspace\0github\python-rest-app-cont
collected 2 items

tests\functional\test_func.py .                                          [ 50%]
tests\unit\test_unit.py .                                                [100%]

============================== 2 passed in 0.11s ==============================

Question 36

I solved my problem by setting the PYTHONPATH in Environment Variables for the specific configuration I’m running my tests with.

While you’re viewing the test file on PyCharm:

Ctrl+Shift+A
Type Edit Configurations
Set your PYTHONPATH under Environment > Environment variables.

Question 37

Simply put an empty conftest.py file in the project root directory, because when pytest discovers a conftest.py, it modifies sys.path so it can import stuff from the conftest module. A general directory structure can be:

Root
├── conftest.py
├── module1
│   ├── __init__.py
│   └── sample.py
└── tests
    └── test_sample.py

Question 38

Had a similar issue and it worked when I added __init__.py file under tests directory.

Question 39

It could be that Pytest is not reading the package as a Python module while Python is (likely due to path issues). Try changing the directory of the pytest script or adding the module explicitly to your PYTHONPATH.

Or it could be that you have two versions of Python installed on your machine. Check your Python source for pytest and for the python shell that you run. If they are different (i.e. Python 2 vs 3), use source activate to make sure that you are running the pytest installed for the same python that the module is installed in.

Question 40

For anyone who tried everything and still getting error,I have a work around.

In the folder where pytest is installed,go to pytest-env folder.

Open pyvenv.cfg file.

In the file change include-system-site-packages from false to true.

home = /usr/bin
include-system-site-packages = true
version = 3.6.6

Hope it works .Don’t forget to up vote.

Question 41

If you already have .pyc files, try to delete them.

Today, I encounter this problem, here is what happend:

first I run pytest in mac (this will generate pyc files) then I launch a docker container (the os is alpine), with project dir mounted, and then when I try to run pytest in container, ImportError happens. after cleaning all pyc files, no error any more.

Hope this may be helpful.

Question 42

if you need a init.py file in your folder make a copy of the folder and delete init.py in that one to run your tests it works for local projects. If you need to run test regularly see if you can move your init.py to a separate file.

Question 43

[Resolved] Before directly jumping into the solution of removing/ adding __init__.py, we might also want to look at how the imports are been done in your classes. Actually, I lost a day playing around with just __init__.py thinking that might be the problem :) However, that was quite informative.

In my case, it was the wrong way of calling classes from one python class to another python class which was throwing ImportError. Fixed the way classes/ modules to be called and it worked like charm. Hope, this helps others as well.

And yes, for similar error, we might have different solutions depending on how the code is written. Better to spend more time on self debugging. Lesson Learnt :) Happy coding!!!

Question 44

In my case I am working in a container and unfortuantely pytest has the tendency to use python2.7 rather than my python3 interpreter of choice.

In my case this worked:

python3 -m pytest

My folder structure

/
app/
-module1.py
-module2.py
-tests/
--test_module1.py
--test_module2.py
requirements.txt
README.md

Question 45

I had placed all my tests in a tests folder and was getting the same error. I solved this by adding an init.py in that folder like so:

.
|-- Pipfile
|-- Pipfile.lock
|-- README.md
|-- api
|-- app.py
|-- config.py
|-- migrations
|-- pull_request_template.md
|-- settings.py
`-- tests
    |-- __init__.py <------
    |-- conftest.py
    `-- test_sample.py

Question 46

I would like to use urllib.quote(). But python (python3) is not finding the module. Suppose, I have this line of code:

print(urllib.quote("châteu", safe=''))

How do I import urllib.quote?

import urllib or import urllib.quote both give

AttributeError: 'module' object has no attribute 'quote'

What confuses me is that urllib.request is accessible via import urllib.request

Question 47

In Python 3.x, you need to import urllib.parse.quote:

>>> import urllib.parse
>>> urllib.parse.quote("châteu", safe='')
'ch%C3%A2teu'

According to Python 2.x urllib module documentation:

NOTE

The urllib module has been split into parts and renamed in Python 3 to urllib.request, urllib.parse, and urllib.error.

Question 48

If you need to handle both Python 2.x and 3.x you can catch the exception and load the alternative.

try:
    from urllib import quote  # Python 2.X
except ImportError:
    from urllib.parse import quote  # Python 3+

You could also use the python compatibility wrapper six to handle this.

from six.moves.urllib.parse import quote

Question 49

urllib went through some changes in Python3 and can now be imported from the parse submodule

>>> from urllib.parse import quote  
>>> quote('"')                      
'%22'

Question 50

This is how I handle this, without using exceptions.

import sys
if sys.version_info.major > 2:  # Python 3 or later
    from urllib.parse import quote
else:  # Python 2
    from urllib import quote

Question 51

So i’m getting this error

Traceback (most recent call last):
  File "/Users/alex/dev/runswift/utils/sim2014/simulator.py", line 3, in <module>
    from world import World
  File "/Users/alex/dev/runswift/utils/sim2014/world.py", line 2, in <module>
    from entities.field import Field
  File "/Users/alex/dev/runswift/utils/sim2014/entities/field.py", line 2, in <module>
    from entities.goal import Goal
  File "/Users/alex/dev/runswift/utils/sim2014/entities/goal.py", line 2, in <module>
    from entities.post import Post
  File "/Users/alex/dev/runswift/utils/sim2014/entities/post.py", line 4, in <module>
    from physics import PostBody
  File "/Users/alex/dev/runswift/utils/sim2014/physics.py", line 21, in <module>
    from entities.post import Post
ImportError: cannot import name Post

and you can see that i use the same import statement further up and it works? Is there some unwritten rule about circular importing? How do i use the same class further down the call stack?

Question 52

I think the answer by jpmc26, while by no means wrong, comes down too heavily on circular imports. They can work just fine, if you set them up correctly.

The easiest way to do so is to use import my_module syntax, rather than from my_module import some_object. The former will almost always work, even if my_module included imports us back. The latter only works if my_object is already defined in my_module, which in a circular import may not be the case.

To be specific to your case: Try changing entities/post.py to do import physics and then refer to physics.PostBody rather than just PostBody directly. Similarly, change physics.py to do import entities.post and then use entities.post.Post rather than just Post.

Question 53

When you import a module (or a member of it) for the first time, the code inside the module is executed sequentially like any other code; e.g., it is not treated any differently that the body of a function. An import is just a command like any other (assignment, a function call, def, class). Assuming your imports occur at the top of the script, then here’s what’s happening:

When you try to import World from world, the world script gets executed.
The world script imports Field, which causes the entities.field script to get executed.
This process continues until you reach the entities.post script because you tried to import Post
The entities.post script causes physics module to be executed because it tries to import PostBody
Finally, physics tries to import Post from entities.post
I’m not sure whether the entities.post module exists in memory yet, but it really doesn’t matter. Either the module is not in memory, or the module doesn’t yet have a Post member because it hasn’t finished executing to define Post
Either way, an error occurs because Post is not there to be imported

So no, it’s not “working further up in the call stack”. This is a stack trace of where the error occurred, which means it errored out trying to import Post in that class. You shouldn’t use circular imports. At best, it has negligible benefit (typically, no benefit), and it causes problems like this. It burdens any developer maintaining it, forcing them to walk on egg shells to avoid breaking it. Refactor your module organization.

Question 54

To understand circular dependencies, you need to remember that Python is essentially a scripting language. Execution of statements outside methods occurs at compile time. Import statements are executed just like method calls, and to understand them you should think about them like method calls.

When you do an import, what happens depends on whether the file you are importing already exists in the module table. If it does, Python uses whatever is currently in the symbol table. If not, Python begins reading the module file, compiling/executing/importing whatever it finds there. Symbols referenced at compile time are found or not, depending on whether they have been seen, or are yet to be seen by the compiler.

Imagine you have two source files:

File X.py

def X1:
    return "x1"

from Y import Y2

def X2:
    return "x2"

File Y.py

def Y1:
    return "y1"

from X import X1

def Y2:
    return "y2"

Now suppose you compile file X.py. The compiler begins by defining the method X1, and then hits the import statement in X.py. This causes the compiler to pause compilation of X.py and begin compiling Y.py. Shortly thereafter the compiler hits the import statement in Y.py. Since X.py is already in the module table, Python uses the existing incomplete X.py symbol table to satisfy any references requested. Any symbols appearing before the import statement in X.py are now in the symbol table, but any symbols after are not. Since X1 now appears before the import statement, it is successfully imported. Python then resumes compiling Y.py. In doing so it defines Y2 and finishes compiling Y.py. It then resumes compilation of X.py, and finds Y2 in the Y.py symbol table. Compilation eventually completes w/o error.

Something very different happens if you attempt to compile Y.py from the command line. While compiling Y.py, the compiler hits the import statement before it defines Y2. Then it starts compiling X.py. Soon it hits the import statement in X.py that requires Y2. But Y2 is undefined, so the compile fails.

Please note that if you modify X.py to import Y1, the compile will always succeed, no matter which file you compile. However if you modify file Y.py to import symbol X2, neither file will compile.

Any time when module X, or any module imported by X might import the current module, do NOT use:

from X import Y

Any time you think there may be a circular import you should also avoid compile time references to variables in other modules. Consider the innocent looking code:

import X
z = X.Y

Suppose module X imports this module before this module imports X. Further suppose Y is defined in X after the import statement. Then Y will not be defined when this module is imported, and you will get a compile error. If this module imports Y first, you can get away with it. But when one of your co-workers innocently changes the order of definitions in a third module, the code will break.

In some cases you can resolve circular dependencies by moving an import statement down below symbol definitions needed by other modules. In the examples above, definitions before the import statement never fail. Definitions after the import statement sometimes fail, depending on the order of compilation. You can even put import statements at the end of a file, so long as none of the imported symbols are needed at compile time.

Note that moving import statements down in a module obscures what you are doing. Compensate for this with a comment at the top of your module something like the following:

#import X   (actual import moved down to avoid circular dependency)

In general this is a bad practice, but sometimes it is difficult to avoid.

Question 55

For those of you who, like me, come to this issue from Django, you should know that the docs provide a solution: https://docs.djangoproject.com/en/1.10/ref/models/fields/#foreignkey

“…To refer to models defined in another application, you can explicitly specify a model with the full application label. For example, if the Manufacturer model above is defined in another application called production, you’d need to use:

class Car(models.Model):
    manufacturer = models.ForeignKey(
        'production.Manufacturer',
        on_delete=models.CASCADE,
)

This sort of reference can be useful when resolving circular import dependencies between two applications.…”

Question 56

I was able to import the module within the function (only) that would require the objects from this module:

def my_func():
    import Foo
    foo_instance = Foo()

Question 57

If you run into this issue in a fairly complex app it can be cumbersome to refactor all your imports. PyCharm offers a quickfix for this that will automatically change all usage of the imported symbols as well.

Question 58

I was using the following:

from module import Foo

foo_instance = Foo()

but to get rid of circular reference I did the following and it worked:

import module.foo

foo_instance = foo.Foo()

Question 59

I have just joined a project with a rather large existing code base. We develop in linux and do not use and IDE. We run through the command line. I’m trying to figure out how to get python to search for the right path when I run project modules. For instance, when I run something like:

python someprojectfile.py

I get

ImportError: no module named core.'somemodule'

I get this for all of my imports to I assume it’s an issue with the path.

TLDR:

How do I get Python to search ~/codez/project/ and all the files and folders for *.py files during import statements.

Question 60

There are a few possible ways to do this:

Set the environment variable PYTHONPATH to a colon-separated list of directories to search for imported modules.
In your program, use sys.path.append('/path/to/search') to add the names of directories you want Python to search for imported modules. sys.path is just the list of directories Python searches every time it gets asked to import a module, and you can alter it as needed (although I wouldn’t recommend removing any of the standard directories!). Any directories you put in the environment variable PYTHONPATH will be inserted into sys.path when Python starts up.
Use site.addsitedir to add a directory to sys.path. The difference between this and just plain appending is that when you use addsitedir, it also looks for .pth files within that directory and uses them to possibly add additional directories to sys.path based on the contents of the files. See the documentation for more detail.

Which one of these you want to use depends on your situation. Remember that when you distribute your project to other users, they typically install it in such a manner that the Python code files will be automatically detected by Python’s importer (i.e. packages are usually installed in the site-packages directory), so if you mess with sys.path in your code, that may be unnecessary and might even have adverse effects when that code runs on another computer. For development, I would venture a guess that setting PYTHONPATH is usually the best way to go.

However, when you’re using something that just runs on your own computer (or when you have nonstandard setups, e.g. sometimes in web app frameworks), it’s not entirely uncommon to do something like

import sys
from os.path import dirname
sys.path.append(dirname(__file__))

Question 61

You should also read about python packages here: http://docs.python.org/tutorial/modules.html.

From your example, I would guess that you really have a package at ~/codez/project. The file __init__.py in a python directory maps a directory into a namespace. If your subdirectories all have an __init__.py file, then you only need to add the base directory to your PYTHONPATH. For example:

PYTHONPATH=$PYTHONPATH:$HOME/adaifotis/project

In addition to testing your PYTHONPATH environment variable, as David explains, you can test it in python like this:

$ python
>>> import project                      # should work if PYTHONPATH set
>>> import sys
>>> for line in sys.path: print line    # print current python path

…

Question 62

I know this thread is a bit old, but it took me some time to get to the heart of this, so I wanted to share.

In my project, I had the main script in a parent directory, and, to differentiate the modules, I put all the supporting modules in a sub-folder called “modules”. In my main script, I import these modules like this (for a module called report.py):

from modules.report import report, reportError

If I call my main script, this works. HOWEVER, I wanted to test each module by including a main() in each, and calling each directly, as:

python modules/report.py

Now Python complains that it can’t find “a module called modules”. The key here is that, by default, Python includes the folder of the script in its search path, BUT NOT THE CWD. So what this error says, really, is “I can’t find a modules subfolder”. The is because there is no “modules” subdirectory from the directory where the report.py module resides.

I find that the neatest solution to this is to append the CWD in Python search path by including this at the top:

import sys

sys.path.append(".")

Now Python searches the CWD (current directory), finds the “modules” sub-folder, and all is well.

Question 63

I read this question looking for an answer, and didn’t like any of them.

So I wrote a quick and dirty solution. Just put this somewhere on your sys.path, and it’ll add any directory under folder (from the current working directory), or under abspath:

#using.py

import sys, os.path

def all_from(folder='', abspath=None):
    """add all dirs under `folder` to sys.path if any .py files are found.
    Use an abspath if you'd rather do it that way.

    Uses the current working directory as the location of using.py. 
    Keep in mind that os.walk goes *all the way* down the directory tree.
    With that, try not to use this on something too close to '/'

    """
    add = set(sys.path)
    if abspath is None:
        cwd = os.path.abspath(os.path.curdir)
        abspath = os.path.join(cwd, folder)
    for root, dirs, files in os.walk(abspath):
        for f in files:
            if f[-3:] in '.py':
                add.add(root)
                break
    for i in add: sys.path.append(i)

>>> import using, sys, pprint
>>> using.all_from('py') #if in ~, /home/user/py/
>>> pprint.pprint(sys.path)
[
#that was easy
]

And I like it because I can have a folder for some random tools and not have them be a part of packages or anything, and still get access to some (or all) of them in a couple lines of code.

Question 64

The easiest way I find is to create a file “any_name.pth” and put it in your folder “\Lib\site-packages”. You should find that folder wherever python is installed.

In that file, put a list of directories where you want to keep modules for importing. For instance, make a line in that file like this:

C:\Users\example…\example

You will be able to tell it works by running this in python:

import sys
for line in sys: print line

You will see your directory printed out, amongst others from where you can also import. Now you can import a “mymodule.py” file that sits in that directory as easily as:

import mymodule

This will not import subfolders. For that you could imagine creating a python script to create a .pth file containing all sub folders of a folder you define. Have it run at startup perhaps.

问题：从另一个文件导入变量？

回答 0

回答 1

回答 2

回答 3

回答 4

回答 5

回答 6

回答 7

问题：存在相同名称的模块时从内置库导入

回答 0

回答 1

回答 2

回答 3

回答 4

例

结果

Example

Result

回答 5

问题：pytest无法导入模块，而python可以

回答 0

回答 1

回答 2

回答 3

回答 4

回答 5

回答 6

回答 7

回答 8

回答 9

回答 10

回答 11

回答 12

回答 13

回答 14

回答 15

回答 16

回答 17

回答 18

回答 19

回答 20

回答 21

回答 22

回答 23

回答 24

回答 25

回答 26

回答 27

问题：Python：导入urllib.quote

回答 0

回答 1

回答 2

回答 3

问题：Python循环导入？

回答 0

回答 1

回答 2

回答 3

回答 4

回答 5

回答 6

问题：将Python搜索路径扩展到其他来源

回答 0

回答 1

回答 2

回答 3

回答 4

回答 5

问题：如何从另一个模块更改模块变量？

回答 0

回答 1

回答 2

问题：从子文件夹导入模块

回答 0

回答 1

回答 2

回答 3

回答 4

回答 5