在Django中优雅地设置Python日志记录

问题:在Django中优雅地设置Python日志记录

我还没有找到对自己满意的Django设置Python日志记录的方法。我的要求很简单:

  • 不同事件的不同日志处理程序-也就是说,我希望能够登录到不同的文件
  • 轻松访问我模块中的记录器。该模块应该可以轻松找到其记录器。
  • 应该容易适用于命令行模块。系统的一部分是独立的命令行或守护进程。这些模块应易于使用日志记录。

我当前的设置是使用logging.conf文件,然后在我登录的每个模块中记录设置。感觉不对。

您是否有喜欢的日志记录设置?请详细说明:如何设置配置(logging.conf在代码中使用或设置),在何处/何时启动记录器,以及如何在模块中访问它们等等。

I have yet to find a way of setting up Python logging with Django that I’m happy with. My requirements are fairly simple:

  • Different log handlers for different events – that is, I want to be able to log to different files
  • Easy access to loggers in my modules. The module should be able to find its logger with little effort.
  • Should be easily applicable to command-line modules. Parts of the system are stand-alone command line or daemon processes. Logging should be easily usable with these modules.

My current setup is to use a logging.conf file and setup logging in each module I log from. It doesn’t feel right.

Do you have a logging setup that you like? Please detail it: how do you setup the configuration (do you use logging.conf or set it up in code), where/when do you initiate the loggers, and how do you get access to them in your modules, etc.


回答 0

到目前为止,我发现的最好方法是在settings.py中初始化日志记录设置-其他地方。您可以使用配置文件,也可以分步进行编程-这仅取决于您的要求。关键是我通常使用级别和有时记录日志的方式将我想要的处理程序添加到根记录器中。过滤器将我想要的事件获取到适当的文件,控制台,系统日志等。您当然可以将处理程序添加到任何其他记录器中同样,但根据我的经验,通常不需要这样做。

在每个模块中,我使用

logger = logging.getLogger(__name__)

并将其用于记录模块中的事件(并且,如果我想进一步区分),请使用记录器,该记录器是上面创建的记录器的子级。

如果我的应用程序可能会在未配置settings.py中的日志记录的站点中使用,请在以下位置定义NullHandler:

#someutils.py

class NullHandler(logging.Handler):
    def emit(self, record):
        pass

null_handler = NullHandler()

并确保将其实例添加到我的使用日志记录的应用程序的模块中创建的所有记录器中。(注意:NullHandler已经在Python 3.1的日志记录包中,并且将在Python 2.7中使用。)

logger = logging.getLogger(__name__)
logger.addHandler(someutils.null_handler)

这样做是为了确保您的模块在未配置settings.py中的日志记录的站点中正常运行,并且不会收到任何令人讨厌的“找不到记录器XYZ处理程序”的消息(这是有关潜在警告的警告)错误配置的日志记录)。

通过这种方式可以满足您规定的要求:

  • 您可以像当前一样为不同的事件设置不同的日志处理程序。
  • 轻松访问模块中的记录器-使用getLogger(__name__)
  • 容易适用于命令行模块-它们也可以导入settings.py

更新:请注意,从1.3版开始,Django现在合并了对logging的支持

The best way I’ve found so far is to initialize logging setup in settings.py – nowhere else. You can either use a configuration file or do it programmatically step-by-step – it just depends on your requirements. The key thing is that I usually add the handlers I want to the root logger, using levels and sometimes logging.Filters to get the events I want to the appropriate files, console, syslogs etc. You can of course add handlers to any other loggers too, but there isn’t commonly a need for this in my experience.

In each module, I define a logger using

logger = logging.getLogger(__name__)

and use that for logging events in the module (and, if I want to differentiate further) use a logger which is a child of the logger created above.

If my app is going to be potentially used in a site which doesn’t configure logging in settings.py, I define a NullHandler somewhere as follows:

#someutils.py

class NullHandler(logging.Handler):
    def emit(self, record):
        pass

null_handler = NullHandler()

and ensure that an instance of it is added to all loggers created in the modules in my apps which use logging. (Note: NullHandler is already in the logging package for Python 3.1, and will be in Python 2.7.) So:

logger = logging.getLogger(__name__)
logger.addHandler(someutils.null_handler)

This is done to ensure that your modules play nicely in a site which doesn’t configure logging in settings.py, and that you don’t get any annoying “No handlers could be found for logger X.Y.Z” messages (which are warnings about potentially misconfigured logging).

Doing it this way meets your stated requirements:

  • You can set up different log handlers for different events, as you currently do.
  • Easy access to loggers in your modules – use getLogger(__name__).
  • Easily applicable to command-line modules – they also import settings.py.

Update: Note that as of version 1.3, Django now incorporates support for logging.


回答 1

我知道这已经是一个解决的答案,但是按照django> = 1.3,有一个新的日志记录设置。

从旧到新并不是自动的,所以我想在这里写下来。

当然,请查看django文档以了解更多信息。

这是基本的配置文件,默认情况下是使用django-admin createproject v1.3创建的-里程可能会随着最新的django版本而变化:

LOGGING = {
    'version': 1,
    'disable_existing_loggers': False,
    'handlers': {
        'mail_admins': {
            'level': 'ERROR',
            'class': 'django.utils.log.AdminEmailHandler',
        }
    },
    'loggers': {
        'django.request': {
            'handlers': ['mail_admins'],
            'level': 'ERROR',
            'propagate': True,
        }
    }
}

该结构基于标准的Python日志dictConfig,该命令规定了以下块:

  • formatters -相应的值将是一个dict,其中每个键是一个格式化程序ID,每个值是一个描述如何配置相应的Formatter实例的dict。
  • filters -相应的值将是一个dict,其中每个键是一个过滤器ID,每个值是一个描述如何配置相应的Filter实例的dict。
  • handlers-相应的值将是一个dict,其中每个键是一个处理程序ID,每个值是一个描述如何配置相应的Handler实例的dict。每个处理程序具有以下键:

    • class(必填)。这是处理程序类的完全限定名称。
    • level(可选的)。处理程序的级别。
    • formatter(可选的)。此处理程序的格式化程序的ID。
    • filters(可选的)。此处理程序的过滤器ID的列表。

我通常至少这样做:

  • 添加一个.log文件
  • 配置我的应用程序以写入此日志

转换为:

LOGGING = {
    'version': 1,
    'disable_existing_loggers': False,
    'formatters': {
        'verbose': {
            'format': '%(levelname)s %(asctime)s %(module)s %(process)d %(thread)d %(message)s'
        },
        'simple': {
            'format': '%(levelname)s %(message)s'
        },
    },
    'filters': {
        'require_debug_false': {
            '()': 'django.utils.log.RequireDebugFalse'
        }
    },
    'handlers': {
        'null': {
            'level':'DEBUG',
            'class':'django.utils.log.NullHandler',
        },
        'console':{
            'level': 'DEBUG',
            'class': 'logging.StreamHandler',
            'formatter': 'simple'
        },
        # I always add this handler to facilitate separating loggings
        'log_file':{
            'level': 'DEBUG',
            'class': 'logging.handlers.RotatingFileHandler',
            'filename': os.path.join(VAR_ROOT, 'logs/django.log'),
            'maxBytes': '16777216', # 16megabytes
            'formatter': 'verbose'
        },
        'mail_admins': {
            'level': 'ERROR',
            'filters': ['require_debug_false'],
            'class': 'django.utils.log.AdminEmailHandler',
            'include_html': True,
        }
    },
    'loggers': {
        'django.request': {
            'handlers': ['mail_admins'],
            'level': 'ERROR',
            'propagate': True,
        },
        'apps': { # I keep all my of apps under 'apps' folder, but you can also add them one by one, and this depends on how your virtualenv/paths are set
            'handlers': ['log_file'],
            'level': 'INFO',
            'propagate': True,
        },
    },
    # you can also shortcut 'loggers' and just configure logging for EVERYTHING at once
    'root': {
        'handlers': ['console', 'mail_admins'],
        'level': 'INFO'
    },
}

编辑

请参阅现在始终记录请求异常故障单#16288

我更新了上面的示例conf,以明确包含针对mail_admins的正确过滤器,以便默认情况下,当debug为True时,不发送电子邮件。

您应该添加一个过滤器:

'filters': {
    'require_debug_false': {
        '()': 'django.utils.log.RequireDebugFalse'
    }
},

并将其应用于mail_admins处理程序:

    'mail_admins': {
        'level': 'ERROR',
        'filters': ['require_debug_false'],
        'class': 'django.utils.log.AdminEmailHandler',
        'include_html': True,
    }

否则,django.core.handers.base.handle_uncaught_exception如果settings.DEBUG为True,则不会将错误传递给django.request记录器。

如果您在Django 1.5中不执行此操作,则会得到一个

DeprecationWarning:您在’mail_admins’日志记录处理程序上未定义过滤器:添加隐式的debug-false-only过滤器

但是在django 1.4和django 1.5中,一切仍然可以正常工作。

**结束编辑**

该conf受到django doc中示例conf的强烈启发,但添加了日志文件部分。

我经常还会执行以下操作:

LOG_LEVEL = 'DEBUG' if DEBUG else 'INFO'

...
    'level': LOG_LEVEL
...

然后在我的python代码中,我总是添加一个NullHandler,以防万一没有定义任何日志配置。这样可以避免未指定任何处理程序的警告。对于不一定只在Django(ref)中调用的库特别有用

import logging
# Get an instance of a logger
logger = logging.getLogger(__name__)
class NullHandler(logging.Handler): #exists in python 3.1
    def emit(self, record):
        pass
nullhandler = logger.addHandler(NullHandler())

# here you can also add some local logger should you want: to stdout with streamhandler, or to a local file...

[…]

logger.warning('etc.etc.')

希望这可以帮助!

I know this is a solved answer already, but as per django >= 1.3 there’s a new logging setting.

Moving from old to new is not automatic, so I thought i’ll write it down here.

And of course checkout the django doc for some more.

This is the basic conf, created by default with django-admin createproject v1.3 – mileage might change with latest django versions:

LOGGING = {
    'version': 1,
    'disable_existing_loggers': False,
    'handlers': {
        'mail_admins': {
            'level': 'ERROR',
            'class': 'django.utils.log.AdminEmailHandler',
        }
    },
    'loggers': {
        'django.request': {
            'handlers': ['mail_admins'],
            'level': 'ERROR',
            'propagate': True,
        }
    }
}

This structure is based upon the standard Python logging dictConfig, that dictates the following blocks:

  • formatters – the corresponding value will be a dict in which each key is a formatter id and each value is a dict describing how to configure the corresponding Formatter instance.
  • filters – the corresponding value will be a dict in which each key is a filter id and each value is a dict describing how to configure the corresponding Filter instance.
  • handlers – the corresponding value will be a dict in which each key is a handler id and each value is a dict describing how to configure the corresponding Handler instance. Each handler has the following keys:

    • class (mandatory). This is the fully qualified name of the handler class.
    • level (optional). The level of the handler.
    • formatter (optional). The id of the formatter for this handler.
    • filters (optional). A list of ids of the filters for this handler.

I usually do at least this:

  • add a .log file
  • configure my apps to write to this log

Which translates into:

LOGGING = {
    'version': 1,
    'disable_existing_loggers': False,
    'formatters': {
        'verbose': {
            'format': '%(levelname)s %(asctime)s %(module)s %(process)d %(thread)d %(message)s'
        },
        'simple': {
            'format': '%(levelname)s %(message)s'
        },
    },
    'filters': {
        'require_debug_false': {
            '()': 'django.utils.log.RequireDebugFalse'
        }
    },
    'handlers': {
        'null': {
            'level':'DEBUG',
            'class':'django.utils.log.NullHandler',
        },
        'console':{
            'level': 'DEBUG',
            'class': 'logging.StreamHandler',
            'formatter': 'simple'
        },
        # I always add this handler to facilitate separating loggings
        'log_file':{
            'level': 'DEBUG',
            'class': 'logging.handlers.RotatingFileHandler',
            'filename': os.path.join(VAR_ROOT, 'logs/django.log'),
            'maxBytes': '16777216', # 16megabytes
            'formatter': 'verbose'
        },
        'mail_admins': {
            'level': 'ERROR',
            'filters': ['require_debug_false'],
            'class': 'django.utils.log.AdminEmailHandler',
            'include_html': True,
        }
    },
    'loggers': {
        'django.request': {
            'handlers': ['mail_admins'],
            'level': 'ERROR',
            'propagate': True,
        },
        'apps': { # I keep all my of apps under 'apps' folder, but you can also add them one by one, and this depends on how your virtualenv/paths are set
            'handlers': ['log_file'],
            'level': 'INFO',
            'propagate': True,
        },
    },
    # you can also shortcut 'loggers' and just configure logging for EVERYTHING at once
    'root': {
        'handlers': ['console', 'mail_admins'],
        'level': 'INFO'
    },
}

edit

See request exceptions are now always logged and Ticket #16288:

I updated the above sample conf to explicitly include the correct filter for mail_admins so that, by default, emails are not sent when debug is True.

You should add a filter:

'filters': {
    'require_debug_false': {
        '()': 'django.utils.log.RequireDebugFalse'
    }
},

and apply it to the mail_admins handler:

    'mail_admins': {
        'level': 'ERROR',
        'filters': ['require_debug_false'],
        'class': 'django.utils.log.AdminEmailHandler',
        'include_html': True,
    }

Otherwise the django.core.handers.base.handle_uncaught_exception doesn’t pass errors to the ‘django.request’ logger if settings.DEBUG is True.

If you don’t do this in Django 1.5 you’ll get a

DeprecationWarning: You have no filters defined on the ‘mail_admins’ logging handler: adding implicit debug-false-only filter

but things will still work correctly BOTH in django 1.4 and django 1.5.

** end edit **

That conf is strongly inspired by the sample conf in the django doc, but adding the log file part.

I often also do the following:

LOG_LEVEL = 'DEBUG' if DEBUG else 'INFO'

...
    'level': LOG_LEVEL
...

Then in my python code I always add a NullHandler in case no logging conf is defined whatsoever. This avoid warnings for no Handler specified. Especially useful for libs that are not necessarily called only in Django (ref)

import logging
# Get an instance of a logger
logger = logging.getLogger(__name__)
class NullHandler(logging.Handler): #exists in python 3.1
    def emit(self, record):
        pass
nullhandler = logger.addHandler(NullHandler())

# here you can also add some local logger should you want: to stdout with streamhandler, or to a local file...

[…]

logger.warning('etc.etc.')

Hope this helps!


回答 2

我们urls.py使用logging.ini文件初始化顶级日志记录。

的位置在logging.ini中提供settings.py,仅此而已。

然后每个模块都执行

logger = logging.getLogger(__name__)

为了区分测试,开发和生产实例,我们有不同的logging.ini文件。在大多数情况下,我们有一个“控制台日志”,该日志仅发送到带有错误的stderr。我们有一个“应用程序日志”,它使用常规的滚动日志文件进入日志目录。

We initialize logging in the top-level urls.py by using a logging.ini file.

The location of the logging.ini is provided in settings.py, but that’s all.

Each module then does

logger = logging.getLogger(__name__)

To distinguish testing, development and production instances, we have different logging.ini files. For the most part, we have a “console log” that goes to stderr with Errors only. We have an “application log” that uses a regular rolling log file that goes to a logs directory.


回答 3

我目前正在使用自己创建的日志系统。它使用CSV格式进行记录。

django-csvlog

该项目仍然没有完整的文档,但是我正在研究它。

I am currently using a logging system, which I created myself. It uses CSV format for logging.

django-csvlog

This project still doesn’t have full documentation, but I am working on it.