使用登录多个模块

问题:使用登录多个模块

我有一个具有以下结构的小型python项目-

Project 
 -- pkg01
   -- test01.py
 -- pkg02
   -- test02.py
 -- logging.conf

我计划使用默认的日志记录模块将消息打印到stdout和日志文件。要使用日志记录模块,需要进行一些初始化-

import logging.config

logging.config.fileConfig('logging.conf')
logger = logging.getLogger('pyApp')

logger.info('testing')

目前,在开始记录消息之前,我会在每个模块中执行此初始化。是否可以只在一个地方执行一次初始化,以便通过记录整个项目来重复使用相同的设置?

I have a small python project that has the following structure –

Project 
 -- pkg01
   -- test01.py
 -- pkg02
   -- test02.py
 -- logging.conf

I plan to use the default logging module to print messages to stdout and a log file. To use the logging module, some initialization is required –

import logging.config

logging.config.fileConfig('logging.conf')
logger = logging.getLogger('pyApp')

logger.info('testing')

At present, I perform this initialization in every module before I start logging messages. Is it possible to perform this initialization only once in one place such that the same settings are reused by logging all over the project?


回答 0

最佳实践是在每个模块中都定义一个记录器,如下所示:

import logging
logger = logging.getLogger(__name__)

在模块顶部附近,然后在模块中的其他代码中执行例如

logger.debug('My message with %s', 'variable data')

如果您需要在模块内细分日志记录活动,请使用例如

loggerA = logging.getLogger(__name__ + '.A')
loggerB = logging.getLogger(__name__ + '.B')

并登录loggerAloggerB视情况而定。

在您的一个或多个主程序中,执行例如:

def main():
    "your program code"

if __name__ == '__main__':
    import logging.config
    logging.config.fileConfig('/path/to/logging.conf')
    main()

要么

def main():
    import logging.config
    logging.config.fileConfig('/path/to/logging.conf')
    # your program code

if __name__ == '__main__':
    main()

这里用于从多个模块记录,并在这里用于代码日志配置将被用作其它的代码库模块。

更新:调用时fileConfig(),您可能想指定disable_existing_loggers=False是否使用Python 2.6或更高版本(有关更多信息,请参阅文档)。缺省值是True为了向后兼容,fileConfig()除非所有现有记录器或其祖先在配置中被明确命名,否则它将被禁用。将该值设置为False,将保留现有记录器。如果使用的是Python 2.7 / Python 3.2或更高版本,则不妨考虑该dictConfig()API更好,fileConfig()因为它可以更好地控制配置。

Best practice is, in each module, to have a logger defined like this:

import logging
logger = logging.getLogger(__name__)

near the top of the module, and then in other code in the module do e.g.

logger.debug('My message with %s', 'variable data')

If you need to subdivide logging activity inside a module, use e.g.

loggerA = logging.getLogger(__name__ + '.A')
loggerB = logging.getLogger(__name__ + '.B')

and log to loggerA and loggerB as appropriate.

In your main program or programs, do e.g.:

def main():
    "your program code"

if __name__ == '__main__':
    import logging.config
    logging.config.fileConfig('/path/to/logging.conf')
    main()

or

def main():
    import logging.config
    logging.config.fileConfig('/path/to/logging.conf')
    # your program code

if __name__ == '__main__':
    main()

See here for logging from multiple modules, and here for logging configuration for code which will be used as a library module by other code.

Update: When calling fileConfig(), you may want to specify disable_existing_loggers=False if you’re using Python 2.6 or later (see the docs for more information). The default value is True for backward compatibility, which causes all existing loggers to be disabled by fileConfig() unless they or their ancestor are explicitly named in the configuration. With the value set to False, existing loggers are left alone. If using Python 2.7/Python 3.2 or later, you may wish to consider the dictConfig() API which is better than fileConfig() as it gives more control over the configuration.


回答 1

实际上,每个记录器都是父级程序包记录器的子级(即package.subpackage.module从继承配置package.subpackage),因此您只需要做的就是配置根记录器。这可以通过logging.config.fileConfig(您自己的记录器配置)或logging.basicConfig(设置根记录器)来实现。在您的输入模块中登录安装程序(__main__.py或者您想要运行的任何程序,例如main_script.py__init__.py也可以)

使用basicConfig:

# package/__main__.py
import logging
import sys

logging.basicConfig(stream=sys.stdout, level=logging.INFO)

使用fileConfig:

# package/__main__.py
import logging
import logging.config

logging.config.fileConfig('logging.conf')

然后使用以下命令创建每个记录器:

# package/submodule.py
# or
# package/subpackage/submodule.py
import logging
log = logging.getLogger(__name__)

log.info("Hello logging!")

有关更多信息,请参见高级日志记录教程

Actually every logger is a child of the parent’s package logger (i.e. package.subpackage.module inherits configuration from package.subpackage), so all you need to do is just to configure the root logger. This can be achieved by logging.config.fileConfig (your own config for loggers) or logging.basicConfig (sets the root logger). Setup logging in your entry module (__main__.py or whatever you want to run, for example main_script.py. __init__.py works as well)

using basicConfig:

# package/__main__.py
import logging
import sys

logging.basicConfig(stream=sys.stdout, level=logging.INFO)

using fileConfig:

# package/__main__.py
import logging
import logging.config

logging.config.fileConfig('logging.conf')

and then create every logger using:

# package/submodule.py
# or
# package/subpackage/submodule.py
import logging
log = logging.getLogger(__name__)

log.info("Hello logging!")

For more information see Advanced Logging Tutorial.


回答 2

我总是这样做如下。

使用一个python文件将我的日志配置为名为“ log_conf.py”的单例模式

#-*-coding:utf-8-*-

import logging.config

def singleton(cls):
    instances = {}
    def get_instance():
        if cls not in instances:
            instances[cls] = cls()
        return instances[cls]
    return get_instance()

@singleton
class Logger():
    def __init__(self):
        logging.config.fileConfig('logging.conf')
        self.logr = logging.getLogger('root')

在另一个模块中,只需导入配置。

from log_conf import Logger

Logger.logr.info("Hello World")

这是一种简单有效的日志记录模式。

I always do it as below.

Use a single python file to config my log as singleton pattern which named ‘log_conf.py

#-*-coding:utf-8-*-

import logging.config

def singleton(cls):
    instances = {}
    def get_instance():
        if cls not in instances:
            instances[cls] = cls()
        return instances[cls]
    return get_instance()

@singleton
class Logger():
    def __init__(self):
        logging.config.fileConfig('logging.conf')
        self.logr = logging.getLogger('root')

In another module, just import the config.

from log_conf import Logger

Logger.logr.info("Hello World")

This is a singleton pattern to log, simply and efficiently.


回答 3

这些答案中有几个建议您在模块的顶部进行操作

import logging
logger = logging.getLogger(__name__)

据我了解,这被认为是非常糟糕的做法。原因是默认情况下,文件配置将禁用所有现有记录器。例如

#my_module
import logging

logger = logging.getLogger(__name__)

def foo():
    logger.info('Hi, foo')

class Bar(object):
    def bar(self):
        logger.info('Hi, bar')

在您的主模块中:

#main
import logging

# load my module - this now configures the logger
import my_module

# This will now disable the logger in my module by default, [see the docs][1] 
logging.config.fileConfig('logging.ini')

my_module.foo()
bar = my_module.Bar()
bar.bar()

现在,在logging.ini中指定的日志将为空,因为现有的记录器已被fileconfig调用禁用。

虽然当然可以解决此问题(disable_existing_Loggers = False),但实际上,您的库中的许多客户端将不了解此行为,并且不会收到您的日志。始终通过在本地调用logging.getLogger来使您的客户容易。帽子小贴士:我从这里了解了这种行为 Victor Lin的网站上

因此,好的做法是始终在本地调用logging.getLogger。例如

#my_module
import logging

logger = logging.getLogger(__name__)

def foo():
    logging.getLogger(__name__).info('Hi, foo')

class Bar(object):
    def bar(self):
        logging.getLogger(__name__).info('Hi, bar')    

另外,如果在主目录中使用fileconfig,请设置disable_existing_loggers = False,以防万一您的库设计人员使用模块级记录器实例。

Several of these answers suggest that at the top of a module you you do

import logging
logger = logging.getLogger(__name__)

It is my understanding that this is considered very bad practice. The reason is that the file config will disable all existing loggers by default. E.g.

#my_module
import logging

logger = logging.getLogger(__name__)

def foo():
    logger.info('Hi, foo')

class Bar(object):
    def bar(self):
        logger.info('Hi, bar')

And in your main module :

#main
import logging

# load my module - this now configures the logger
import my_module

# This will now disable the logger in my module by default, [see the docs][1] 
logging.config.fileConfig('logging.ini')

my_module.foo()
bar = my_module.Bar()
bar.bar()

Now the log specified in logging.ini will be empty, as the existing logger was disabled by fileconfig call.

While is is certainly possible to get around this (disable_existing_Loggers=False), realistically many clients of your library will not know about this behavior, and will not receive your logs. Make it easy for your clients by always calling logging.getLogger locally. Hat Tip : I learned about this behavior from Victor Lin’s Website.

So good practice is instead to always call logging.getLogger locally. E.g.

#my_module
import logging

logger = logging.getLogger(__name__)

def foo():
    logging.getLogger(__name__).info('Hi, foo')

class Bar(object):
    def bar(self):
        logging.getLogger(__name__).info('Hi, bar')    

Also, if you use fileconfig in your main, set disable_existing_loggers=False, just in case your library designers use module level logger instances.


回答 4

对我来说,在多个模块中使用日志记录库实例的一种简单方法是解决方案:

base_logger.py

import logging

logger = logging
logger.basicConfig(format='%(asctime)s - %(message)s', level=logging.INFO)

其他的文件

from base_logger import logger

if __name__ == '__main__':
    logger.info("This is an info message")

A simple way of using one instance of logging library in multiple modules for me was following solution:

base_logger.py

import logging

logger = logging
logger.basicConfig(format='%(asctime)s - %(message)s', level=logging.INFO)

Other files

from base_logger import logger

if __name__ == '__main__':
    logger.info("This is an info message")

回答 5

抛出另一种解决方案。

在我模块的init .py中,我有类似以下内容:

# mymodule/__init__.py
import logging

def get_module_logger(mod_name):
  logger = logging.getLogger(mod_name)
  handler = logging.StreamHandler()
  formatter = logging.Formatter(
        '%(asctime)s %(name)-12s %(levelname)-8s %(message)s')
  handler.setFormatter(formatter)
  logger.addHandler(handler)
  logger.setLevel(logging.DEBUG)
  return logger

然后在每个模块中我需要一个记录器,我这样做:

# mymodule/foo.py
from [modname] import get_module_logger
logger = get_module_logger(__name__)

如果缺少日志,则可以通过其来源来区分其来源。

Throwing in another solution.

In my module’s init.py I have something like:

# mymodule/__init__.py
import logging

def get_module_logger(mod_name):
  logger = logging.getLogger(mod_name)
  handler = logging.StreamHandler()
  formatter = logging.Formatter(
        '%(asctime)s %(name)-12s %(levelname)-8s %(message)s')
  handler.setFormatter(formatter)
  logger.addHandler(handler)
  logger.setLevel(logging.DEBUG)
  return logger

Then in each module I need a logger, I do:

# mymodule/foo.py
from [modname] import get_module_logger
logger = get_module_logger(__name__)

When the logs are missed, you can differentiate their source by the module they came from.


回答 6

您也可以提出这样的建议!

def get_logger(name=None):
    default = "__app__"
    formatter = logging.Formatter('%(levelname)s: %(asctime)s %(funcName)s(%(lineno)d) -- %(message)s',
                              datefmt='%Y-%m-%d %H:%M:%S')
    log_map = {"__app__": "app.log", "__basic_log__": "file1.log", "__advance_log__": "file2.log"}
    if name:
        logger = logging.getLogger(name)
    else:
        logger = logging.getLogger(default)
    fh = logging.FileHandler(log_map[name])
    fh.setFormatter(formatter)
    logger.addHandler(fh)
    logger.setLevel(logging.DEBUG)
    return logger

现在,如果以上内容在单独的模块中定义并且在需要记录的其他模块中导入,则可以在同一模块和整个项目中使用多个记录器。

a=get_logger("__app___")
b=get_logger("__basic_log__")
a.info("Starting logging!")
b.debug("Debug Mode")

You could also come up with something like this!

def get_logger(name=None):
    default = "__app__"
    formatter = logging.Formatter('%(levelname)s: %(asctime)s %(funcName)s(%(lineno)d) -- %(message)s',
                              datefmt='%Y-%m-%d %H:%M:%S')
    log_map = {"__app__": "app.log", "__basic_log__": "file1.log", "__advance_log__": "file2.log"}
    if name:
        logger = logging.getLogger(name)
    else:
        logger = logging.getLogger(default)
    fh = logging.FileHandler(log_map[name])
    fh.setFormatter(formatter)
    logger.addHandler(fh)
    logger.setLevel(logging.DEBUG)
    return logger

Now you could use multiple loggers in same module and across whole project if the above is defined in a separate module and imported in other modules were logging is required.

a=get_logger("__app___")
b=get_logger("__basic_log__")
a.info("Starting logging!")
b.debug("Debug Mode")

回答 7

@Yarkee的解决方案似乎更好。我想补充一点-

class Singleton(type):
    _instances = {}

    def __call__(cls, *args, **kwargs):
        if cls not in cls._instances.keys():
            cls._instances[cls] = super(Singleton, cls).__call__(*args, **kwargs)
        return cls._instances[cls]


class LoggerManager(object):
    __metaclass__ = Singleton

    _loggers = {}

    def __init__(self, *args, **kwargs):
        pass

    @staticmethod
    def getLogger(name=None):
        if not name:
            logging.basicConfig()
            return logging.getLogger()
        elif name not in LoggerManager._loggers.keys():
            logging.basicConfig()
            LoggerManager._loggers[name] = logging.getLogger(str(name))
        return LoggerManager._loggers[name]    


log=LoggerManager().getLogger("Hello")
log.setLevel(level=logging.DEBUG)

因此LoggerManager可以插入整个应用程序。希望它有意义和有价值。

@Yarkee’s solution seemed better. I would like to add somemore to it –

class Singleton(type):
    _instances = {}

    def __call__(cls, *args, **kwargs):
        if cls not in cls._instances.keys():
            cls._instances[cls] = super(Singleton, cls).__call__(*args, **kwargs)
        return cls._instances[cls]


class LoggerManager(object):
    __metaclass__ = Singleton

    _loggers = {}

    def __init__(self, *args, **kwargs):
        pass

    @staticmethod
    def getLogger(name=None):
        if not name:
            logging.basicConfig()
            return logging.getLogger()
        elif name not in LoggerManager._loggers.keys():
            logging.basicConfig()
            LoggerManager._loggers[name] = logging.getLogger(str(name))
        return LoggerManager._loggers[name]    


log=LoggerManager().getLogger("Hello")
log.setLevel(level=logging.DEBUG)

So LoggerManager can be a pluggable to the entire application. Hope it makes sense and value.


回答 8

有几个答案。我最终得到了一个对我有意义的类似但又不同的解决方案,也许对您也很有意义。我的主要目标是能够按其级别将日志传递给处理程序(将调试级别的日志传递到控制台,将警告及以上信息传递给文件):

from flask import Flask
import logging
from logging.handlers import RotatingFileHandler

app = Flask(__name__)

# make default logger output everything to the console
logging.basicConfig(level=logging.DEBUG)

rotating_file_handler = RotatingFileHandler(filename="logs.log")
rotating_file_handler.setLevel(logging.INFO)

app.logger.addHandler(rotating_file_handler)

创建了一个名为logger.py的实用程序文件:

import logging

def get_logger(name):
    return logging.getLogger("flask.app." + name)

flask.app是flask中的硬编码值。应用程序记录器始终以flask.app作为模块名称。

现在,在每个模块中,我都可以在以下模式下使用它:

from logger import get_logger
logger = get_logger(__name__)

logger.info("new log")

这将以最小的努力为“ app.flask.MODULE_NAME”创建一个新日志。

There are several answers. i ended up with a similar yet different solution that makes sense to me, maybe it will make sense to you as well. My main objective was to be able to pass logs to handlers by their level (debug level logs to the console, warnings and above to files):

from flask import Flask
import logging
from logging.handlers import RotatingFileHandler

app = Flask(__name__)

# make default logger output everything to the console
logging.basicConfig(level=logging.DEBUG)

rotating_file_handler = RotatingFileHandler(filename="logs.log")
rotating_file_handler.setLevel(logging.INFO)

app.logger.addHandler(rotating_file_handler)

created a nice util file named logger.py:

import logging

def get_logger(name):
    return logging.getLogger("flask.app." + name)

the flask.app is a hardcoded value in flask. the application logger is always starting with flask.app as its the module’s name.

now, in each module, i’m able to use it in the following mode:

from logger import get_logger
logger = get_logger(__name__)

logger.info("new log")

This will create a new log for “app.flask.MODULE_NAME” with minimum effort.


回答 9

最佳实践是分别创建一个模块,该模块只有一个方法,我们的任务是将记录程序处理程序提供给调用方法。将此文件另存为m_logger.py

import logger, logging

def getlogger():
    # logger
    logger = logging.getLogger(__name__)
    logger.setLevel(logging.DEBUG)
    # create console handler and set level to debug
    #ch = logging.StreamHandler()
    ch = logging.FileHandler(r'log.txt')
    ch.setLevel(logging.DEBUG)
    # create formatter
    formatter = logging.Formatter('%(asctime)s - %(levelname)s - %(message)s')
    # add formatter to ch
    ch.setFormatter(formatter)
    # add ch to logger
    logger.addHandler(ch)
    return logger

现在,在需要记录器处理程序时调用getlogger()方法。

from m_logger import getlogger
logger = getlogger()
logger.info('My mssg')

The best practice would be to create a module separately which has only one method whose task we be to give a logger handler to the the calling method. Save this file as m_logger.py

import logger, logging

def getlogger():
    # logger
    logger = logging.getLogger(__name__)
    logger.setLevel(logging.DEBUG)
    # create console handler and set level to debug
    #ch = logging.StreamHandler()
    ch = logging.FileHandler(r'log.txt')
    ch.setLevel(logging.DEBUG)
    # create formatter
    formatter = logging.Formatter('%(asctime)s - %(levelname)s - %(message)s')
    # add formatter to ch
    ch.setFormatter(formatter)
    # add ch to logger
    logger.addHandler(ch)
    return logger

Now call the getlogger() method whenever logger handler is needed.

from m_logger import getlogger
logger = getlogger()
logger.info('My mssg')

回答 10

python的新手,所以我不知道这是否可取,但是它对不重写样板非常有用。

您的项目必须具有init .py,以便可以将其作为模块加载

# Put this in your module's __init__.py
import logging.config
import sys

# I used this dictionary test, you would put:
# logging.config.fileConfig('logging.conf')
# The "" entry in loggers is the root logger, tutorials always 
# use "root" but I can't get that to work
logging.config.dictConfig({
    "version": 1,
    "formatters": {
        "default": {
            "format": "%(asctime)s %(levelname)s %(name)s %(message)s"
        },
    },
    "handlers": {
        "console": {
            "level": 'DEBUG',
            "class": "logging.StreamHandler",
            "stream": "ext://sys.stdout"
        }
    },
    "loggers": {
        "": {
            "level": "DEBUG",
            "handlers": ["console"]
        }
    }
})

def logger():
    # Get the name from the caller of this function
    return logging.getLogger(sys._getframe(1).f_globals['__name__'])

sys._getframe(1)建议来自这里

然后在其他任何文件中使用记录器:

from [your module name here] import logger

logger().debug("FOOOOOOOOO!!!")

注意事项:

  1. 您必须将文件作为模块运行,否则import [your module]将无法工作:
    • python -m [your module name].[your filename without .py]
  2. 程序入口点的记录程序的名称为__main__,但是使用的任何解决方案__name__都会出现该问题。

New to python so I don’t know if this is advisable, but it works great for not re-writing boilerplate.

Your project must have an init.py so it can be loaded as a module

# Put this in your module's __init__.py
import logging.config
import sys

# I used this dictionary test, you would put:
# logging.config.fileConfig('logging.conf')
# The "" entry in loggers is the root logger, tutorials always 
# use "root" but I can't get that to work
logging.config.dictConfig({
    "version": 1,
    "formatters": {
        "default": {
            "format": "%(asctime)s %(levelname)s %(name)s %(message)s"
        },
    },
    "handlers": {
        "console": {
            "level": 'DEBUG',
            "class": "logging.StreamHandler",
            "stream": "ext://sys.stdout"
        }
    },
    "loggers": {
        "": {
            "level": "DEBUG",
            "handlers": ["console"]
        }
    }
})

def logger():
    # Get the name from the caller of this function
    return logging.getLogger(sys._getframe(1).f_globals['__name__'])

sys._getframe(1) suggestion comes from here

Then to use your logger in any other file:

from [your module name here] import logger

logger().debug("FOOOOOOOOO!!!")

Caveats:

  1. You must run your files as modules, otherwise import [your module] won’t work:
    • python -m [your module name].[your filename without .py]
  2. The name of the logger for the entry point of your program will be __main__, but any solution using __name__ will have that issue.