Python日志记录-禁用导入模块的日志记录

问题:Python日志记录-禁用导入模块的日志记录

我正在使用Python日志记录模块,并且想禁用由导入的第三方模块打印的日志消息。例如,我正在使用类似以下内容的东西:

logger = logging.getLogger()
logger.setLevel(level=logging.DEBUG)
fh = logging.StreamHandler()
fh_formatter = logging.Formatter('%(asctime)s %(levelname)s %(lineno)d:%(filename)s(%(process)d) - %(message)s')
fh.setFormatter(fh_formatter)
logger.addHandler(fh)

当执行logger.debug(“ my message!”)时,这会打印出我的调试消息,但是它也会从我导入的任何模块(例如请求和许多其他东西)中打印出调试消息。

我只想查看我感兴趣的模块中的日志消息。是否可以使日志记录模块执行此操作?

理想情况下,我希望能够告诉记录器打印来自“ ModuleX,ModuleY”的消息,而忽略所有其他消息。

我看了以下内容,但是我不想在每次调用导入函数之前都禁用/启用日志记录: logging-如何忽略导入的模块日志?

I’m using the Python logging module, and would like to disable log messages printed by the third party modules that I import. For example, I’m using something like the following:

logger = logging.getLogger()
logger.setLevel(level=logging.DEBUG)
fh = logging.StreamHandler()
fh_formatter = logging.Formatter('%(asctime)s %(levelname)s %(lineno)d:%(filename)s(%(process)d) - %(message)s')
fh.setFormatter(fh_formatter)
logger.addHandler(fh)

This prints out my debug messages when I do a logger.debug(“my message!”), but it also prints out the debug messages from any module I import (such as requests, and a number of other things).

I’d like to see only the log messages from modules I’m interested in. Is it possible to make the logging module do this?

Ideally, I’d like to be able tell the logger to print messages from “ModuleX, ModuleY” and ignore all others.

I looked at the following, but I don’t want to have to disable/enable logging before every call to an imported function: logging – how to ignore imported module logs?


回答 0

问题在于,getLogger不带参数的调用会返回记录器,因此当您将级别logging.DEBUG设置为时,还将为使用该记录器的其他模块设置级别。

您可以通过使用root记录器来解决此问题。为此,只需将名称作为参数传递,例如模块的名称:

logger = logging.getLogger('my_module_name')
# as before

这将创建一个新的记录器,因此不会无意中更改其他模块的记录级别。


显然,您必须使用logger.debug而不是,logging.debug因为后者是一个方便的函数,它调用debug了根记录器的方法。

在“高级日志记录教程”中对此进行了提及。它还允许您以简单的方式知道哪个模块触发了日志消息。

The problem is that calling getLogger without arguments returns the root logger so when you set the level to logging.DEBUG you are also setting the level for other modules that use that logger.

You can solve this by simply not using the root logger. To do this just pass a name as argument, for example the name of your module:

logger = logging.getLogger('my_module_name')
# as before

this will create a new logger and thus it wont inadvertently change logging level for other modules.


Obviously you have to use logger.debug instead of logging.debug since the latter is a convenience function that calls the debug method of the root logger.

This is mentioned in the Advanced Logging Tutorial. It also allows you to know which module triggered the log message in a simple way.


回答 1

如果要使用pythonlogging包,则通常会在使用它的每个模块中定义一个记录器。

logger = logging.getLogger(__name__)

许多流行的python软件包都可以做到这一点,包括requests。如果程序包使用此约定,则很容易为其启用/禁用日志记录,因为记录器名称将与该程序包相同(或者是该记录器的子代)。您甚至可以将其记录到与其他记录器相同的文件中。

logger = logging.getLogger(__name__)
logger.setLevel(logging.DEBUG)

requests_logger = logging.getLogger('requests')
requests_logger.setLevel(logging.DEBUG)

handler = logging.StreamHandler()
handler.setLevel(logging.DEBUG)
logger.addHandler(handler)
requests_logger.addHandler(handler)

If you’re going to use the python logging package, it’s a common convention to define a logger in every module that uses it.

logger = logging.getLogger(__name__)

Many popular python packages do this, including requests. If a package uses this convention, it’s easy to enable/disable logging for it, because the logger name will be the same name as the package (or will be a child of that logger). You can even log it to the same file as your other loggers.

logger = logging.getLogger(__name__)
logger.setLevel(logging.DEBUG)

requests_logger = logging.getLogger('requests')
requests_logger.setLevel(logging.DEBUG)

handler = logging.StreamHandler()
handler.setLevel(logging.DEBUG)
logger.addHandler(handler)
requests_logger.addHandler(handler)

回答 2

不知道这是否适合发布,但是我被困了很长时间并且想帮助任何遇到相同问题的人,因为我在其他任何地方都找不到它!

尽管遵循了日志记录高级教程故障排除中非常简单的文档,但我还是从matplotlib获取调试日志。我在main()一个文件中启动记录器,然后导入一个函数以从另一个文件(我已导入matplotlib)中创建绘图。

对我有用的是导入之前设置matplotlib的级别,而不是像在我的主文件中的其他模块之后那样设置。这对我来说似乎是违反直觉的,因此,如果有人了解如何设置尚未导入的记录器的配置,我很想知道这是如何工作的。谢谢!

在我的主文件中:

import logging
import requests
logger = logging.getLogger(__name__)
logger.setLevel(logging.DEBUG)
logging.getLogger('requests').setLevel(logging.DEBUG)

def main():
  ...

在我的plot.py档案中:

import logging
logging.getLogger('matplotlib').setLevel(logging.WARNING)
import matplotlib.pyplot as plt

def generatePlot():
  ...

Not sure if this is appropriate to post, but I was stuck for a long time & wanted to help out anyone with the same issue, as I hadn’t found it anywhere else!

I was getting debug logs from matplotlib despite following the pretty straightforward documentation at the logging advanced tutorial and the troubleshooting. I was initiating my logger in main() of one file and importing a function to create a plot from another file (where I had imported matplotlib).

What worked for me was setting the level of matplotlib before importing it, rather than after as I had for other modules in my main file. This seemed counterintuitive to me so if anyone has insight into how you can set the config for a logger that hasn’t been imported yet I’d be curious to find out how this works. Thanks!

In my main file:

import logging
import requests
logger = logging.getLogger(__name__)
logger.setLevel(logging.DEBUG)
logging.getLogger('requests').setLevel(logging.DEBUG)

def main():
  ...

In my plot.py file:

import logging
logging.getLogger('matplotlib').setLevel(logging.WARNING)
import matplotlib.pyplot as plt

def generatePlot():
  ...

回答 3

@Bakuriu非常优雅地解释了该功能。相反,您可以使用该getLogger()方法来检索和重新配置/禁用不需要的记录器。

我还想添加该logging.fileConfig()方法接受一个名为的参数disable_existing_loggers,该参数将禁用以前定义的任何记录器(即,在导入的模块中)。

@Bakuriu quite elegantly explains the function. Conversely, you can use the getLogger() method to retrieve and reconfigure/disable the unwanted loggers.

I also wanted to add the logging.fileConfig() method accepts a parameter called disable_existing_loggers which will disable any loggers previously defined (i.e., in imported modules).


回答 4

这将禁用所有现有记录器,例如由导入模块创建的记录器,同时仍使用根记录器(而不必加载外部文件)。

logging.config.dictConfig({
    'version': 1,
    'disable_existing_loggers': True,
})

请注意,您需要导入所有您不想首先登录的模块!否则,这些将不被视为“现有记录器”。然后,它将禁用那些模块中的所有记录器。这可能会导致您也错过重要的错误!

有关使用相关选项进行配置的更多详细示例,请参见https://gist.github.com/st4lk/6287746是一个(部分工作的)示例,该示例使用YAML对该coloredlog库进行配置。

This disables all existing loggers, such as those created by imported modules, while still using the root logger (and without having to load an external file).

logging.config.dictConfig({
    'version': 1,
    'disable_existing_loggers': True,
})

Note that you need to import all modules you don’t want logged first! Otherwise those won’t be considered as “existing loggers”. It will then disable all loggers from those modules. This might lead you to also miss out on important errors!

For more detailed examples using related options for configuration, see https://gist.github.com/st4lk/6287746, and here is a (partially working) example using YAML for config with the coloredlog library.


回答 5

您可以使用类似:

logging.getLogger("imported_module").setLevel(logging.WARNING)
logging.getLogger("my_own_logger_name").setLevel(logging.DEBUG)

这会将我自己模块的日志级别设置为DEBUG,同时防止导入的模块使用同一级别。

注意: "imported_module"可以替换为imported_module.__name__(不带引号),如果您喜欢这样做,"my_own_logger_name"可以将其替换为__name__

You could use something like:

logging.getLogger("imported_module").setLevel(logging.WARNING)
logging.getLogger("my_own_logger_name").setLevel(logging.DEBUG)

This will set my own module’s log level to DEBUG, while preventing the imported module from using the same level.

Note: "imported_module" can be replaced with imported_module.__name__ (without quotes), and "my_own_logger_name" can be replaced by __name__ if that’s the way you prefer to do it.


回答 6

我有同样的问题。我有一个logging_config.py文件,我将其导入所有其他py文件中。在logging_config.py文件中,我将root logger的日志记录级别设置为ERROR(默认情况下为警告):

logging.basicConfig(
    handlers=[
        RotatingFileHandler('logs.log',maxBytes=1000, backupCount=2),
        logging.StreamHandler(), #print to console
    ],
    level=logging.ERROR
)

在其他模块中,我导入logging_config.py并声明一个新的记录器,并将其级别设置为debug:

log = logging.getLogger(__name__)
log.setLevel(logging.DEBUG)

这样,我登录py文件中的所有内容都会被记录下来,但是不会记录由urllib,request,boto3等导入模块在调试和信息级别记录的内容。如果这些导入模块中存在某些错误,则将其记录下来,因为我将根记录程序级别设置为ERROR。

I had the same problem. I have a logging_config.py file which I import in all other py files. In logging_config.py file I set root logger logging level to ERROR (by default its warning):

logging.basicConfig(
    handlers=[
        RotatingFileHandler('logs.log',maxBytes=1000, backupCount=2),
        logging.StreamHandler(), #print to console
    ],
    level=logging.ERROR
)

In other modules I import logging_config.py and declare a new logger and set its level to debug:

log = logging.getLogger(__name__)
log.setLevel(logging.DEBUG)

This way everything I log in my py files is logged, but stuff logged at debug and info level by imported modules like urllib, request,boto3 etc is not logged. If there is some error in those import module then its logged, since I set root loggers level to ERROR.


回答 7

另一个要考虑的是繁殖Logger类的属性。

例如,用于处理肥皂呼叫的py-suds库,甚至置为ERROR

logging.getLogger('suds.client').setLevel(logging.ERROR)
logging.getLogger('suds.transport').setLevel(logging.ERROR)
logging.getLogger('suds.xsdschema').setLevel(logging.ERROR)
logging.getLogger('suds.wsdl').setLevel(logging.ERROR)

记录有关名为sxbasics.py的模块的日志大量的日志

因为默认情况下日志的传播为True,所以设置为False,我恢复了514MB的日志。

import logging
logging.getLogger("suds").propagate = False
logging.getLogger('suds.client').setLevel(logging.ERROR)
logging.getLogger('suds.transport').setLevel(logging.ERROR)
logging.getLogger('suds.xsdschema').setLevel(logging.ERROR)
logging.getLogger('suds.wsdl').setLevel(logging.ERROR)

Another thing to consider is the propagate property of the Logger class.

For example, py-suds library for handling soap calls, even put to ERROR

logging.getLogger('suds.client').setLevel(logging.ERROR)
logging.getLogger('suds.transport').setLevel(logging.ERROR)
logging.getLogger('suds.xsdschema').setLevel(logging.ERROR)
logging.getLogger('suds.wsdl').setLevel(logging.ERROR)

logs logs about a module called sxbasics.py creationg a huge amount of logs

that because the propagation of the logs is True by default, setting to False, instead, i recovered 514MB of logs.

import logging
logging.getLogger("suds").propagate = False
logging.getLogger('suds.client').setLevel(logging.ERROR)
logging.getLogger('suds.transport').setLevel(logging.ERROR)
logging.getLogger('suds.xsdschema').setLevel(logging.ERROR)
logging.getLogger('suds.wsdl').setLevel(logging.ERROR)