标签归档:error-logging

如何在Django网站上记录服务器错误

问题:如何在Django网站上记录服务器错误

因此,在进行开发时,我可以设置settings.DEBUGTrue,如果发生错误,我可以看到格式正确,具有良好的堆栈跟踪和请求信息。

但是在某种生产站点上,我更愿意使用DEBUG=False并向访问者展示一些标准错误500页,其中包含我目前正在修复此bug的信息;)
同时,我想以某种方式记录所有这些信息(堆栈跟踪和请求信息)存储到服务器上的文件中-因此我可以将其输出到控制台并观看错误滚动,每小时将日志发送给我或类似的东西。

您会为django站点推荐什么样的日志记录解决方案,这些解决方案可以满足那些简单的要求?我有作为fcgi服务器运行的应用程序,并且我使用apache Web服务器作为前端(尽管考虑使用lighttpd)。

So, when playing with the development I can just set settings.DEBUG to True and if an error occures I can see it nicely formatted, with good stack trace and request information.

But on kind of production site I’d rather use DEBUG=False and show visitors some standard error 500 page with information that I’m working on fixing this bug at this moment ;)
At the same time I’d like to have some way of logging all those information (stack trace and request info) to a file on my server – so I can just output it to my console and watch errors scroll, email the log to me every hour or something like this.

What logging solutions would you recomend for a django-site, that would meet those simple requirements? I have the application running as fcgi server and I’m using apache web server as frontend (although thinking of going to lighttpd).


回答 0

好的,当时DEBUG = False,Django会自动将所有错误的完整回溯邮件发送给ADMINS设置中列出的每个人,这几乎可以免费为您提供通知。如果您想要更细粒度的控件,则可以编写一个中间件类并将其添加到设置中,该中间件类定义了一个名为的方法process_exception(),该方法可以访问所引发的异常:

http://docs.djangoproject.com/en/dev/topics/http/middleware/#process-exception

process_exception()然后,您的方法可以执行您想要的任何类型的日志记录:写入控制台,写入文件等,等等。

编辑:尽管它的用处不大,但是您也可以侦听got_request_exception信号,该信号将在请求处理期间遇到异常时发送:

http://docs.djangoproject.com/en/dev/ref/signals/#got-request-exception

但是,这不能使您访问异常对象,因此中间件方法更容易使用。

Well, when DEBUG = False, Django will automatically mail a full traceback of any error to each person listed in the ADMINS setting, which gets you notifications pretty much for free. If you’d like more fine-grained control, you can write and add to your settings a middleware class which defines a method named process_exception(), which will have access to the exception that was raised:

http://docs.djangoproject.com/en/dev/topics/http/middleware/#process-exception

Your process_exception() method can then perform whatever type of logging you’d like: writing to console, writing to a file, etc., etc.

Edit: though it’s a bit less useful, you can also listen for the got_request_exception signal, which will be sent whenever an exception is encountered during request processing:

http://docs.djangoproject.com/en/dev/ref/signals/#got-request-exception

This does not give you access to the exception object, however, so the middleware method is much easier to work with.


回答 1

如前所述,Django Sentry是一个不错的选择,但是要正确设置它(作为一个单独的网站)需要进行一些工作。如果您只想将所有内容记录到一个简单的文本文件中,请在此处输入以下记录配置settings.py

LOGGING = {
    'version': 1,
    'disable_existing_loggers': False,
    'handlers': {
        # Include the default Django email handler for errors
        # This is what you'd get without configuring logging at all.
        'mail_admins': {
            'class': 'django.utils.log.AdminEmailHandler',
            'level': 'ERROR',
             # But the emails are plain text by default - HTML is nicer
            'include_html': True,
        },
        # Log to a text file that can be rotated by logrotate
        'logfile': {
            'class': 'logging.handlers.WatchedFileHandler',
            'filename': '/var/log/django/myapp.log'
        },
    },
    'loggers': {
        # Again, default Django configuration to email unhandled exceptions
        'django.request': {
            'handlers': ['mail_admins'],
            'level': 'ERROR',
            'propagate': True,
        },
        # Might as well log any errors anywhere else in Django
        'django': {
            'handlers': ['logfile'],
            'level': 'ERROR',
            'propagate': False,
        },
        # Your own app - this assumes all your logger names start with "myapp."
        'myapp': {
            'handlers': ['logfile'],
            'level': 'WARNING', # Or maybe INFO or DEBUG
            'propagate': False
        },
    },
}

Django Sentry is a good way to go, as already mentioned, but there is a bit of work involved in setting it up properly (as a separate website). If you just want to log everything to a simple text file here’s the logging configuration to put in your settings.py

LOGGING = {
    'version': 1,
    'disable_existing_loggers': False,
    'handlers': {
        # Include the default Django email handler for errors
        # This is what you'd get without configuring logging at all.
        'mail_admins': {
            'class': 'django.utils.log.AdminEmailHandler',
            'level': 'ERROR',
             # But the emails are plain text by default - HTML is nicer
            'include_html': True,
        },
        # Log to a text file that can be rotated by logrotate
        'logfile': {
            'class': 'logging.handlers.WatchedFileHandler',
            'filename': '/var/log/django/myapp.log'
        },
    },
    'loggers': {
        # Again, default Django configuration to email unhandled exceptions
        'django.request': {
            'handlers': ['mail_admins'],
            'level': 'ERROR',
            'propagate': True,
        },
        # Might as well log any errors anywhere else in Django
        'django': {
            'handlers': ['logfile'],
            'level': 'ERROR',
            'propagate': False,
        },
        # Your own app - this assumes all your logger names start with "myapp."
        'myapp': {
            'handlers': ['logfile'],
            'level': 'WARNING', # Or maybe INFO or DEBUG
            'propagate': False
        },
    },
}

回答 2

另一个答案中提到的django-db-log已替换为:

https://github.com/dcramer/django-sentry

django-db-log, mentioned in another answer, has been replaced with:

https://github.com/dcramer/django-sentry


回答 3

显然,James是正确的,但是如果您想在数据存储区中记录异常,则已经有一些开源解决方案可用:

1)CrashLog是一个不错的选择:http : //code.google.com/p/django-crashlog/

2)Db-Log也是一个不错的选择:http : //code.google.com/p/django-db-log/

两者有什么区别?我几乎看不到任何东西,所以只要一个就足够了。

我都用过,而且它们运作良好。

Obviously James is correct, but if you wanted to log exceptions in a datastore, there are a few open source solutions already available:

1) CrashLog is a good choice: http://code.google.com/p/django-crashlog/

2) Db-Log is a good choice as well: http://code.google.com/p/django-db-log/

What is the difference between the two? Almost nothing that I can see, so either one will suffice.

I’ve used both and they work well.


回答 4

自EMP提交最有用的代码以来,已经过去了一段时间。我刚刚实现了它,并在尝试使用一些manage.py选项进行尝试以查找错误时,我收到了弃用警告,以表明在当前版本的Django(1.5。?)中,现在需要require_debug_false过滤器mail_admins处理程序所需。

这是修改后的代码:

LOGGING = {
    'version': 1,
    'disable_existing_loggers': False,
    'filters': {
         'require_debug_false': {
             '()': 'django.utils.log.RequireDebugFalse'
         }
     },
    'handlers': {
        # Include the default Django email handler for errors
        # This is what you'd get without configuring logging at all.
        'mail_admins': {
            'class': 'django.utils.log.AdminEmailHandler',
            'level': 'ERROR',
            'filters': ['require_debug_false'],
             # But the emails are plain text by default - HTML is nicer
            'include_html': True,
        },
        # Log to a text file that can be rotated by logrotate
        'logfile': {
            'class': 'logging.handlers.WatchedFileHandler',
            'filename': '/home/username/public_html/djangoprojectname/logfilename.log'
        },
    },
    'loggers': {
        # Again, default Django configuration to email unhandled exceptions
        'django.request': {
            'handlers': ['mail_admins'],
            'level': 'ERROR',
            'propagate': True,
        },
        # Might as well log any errors anywhere else in Django
        'django': {
            'handlers': ['logfile'],
            'level': 'ERROR',
            'propagate': False,
        },
        # Your own app - this assumes all your logger names start with "myapp."
        'myapp': {
            'handlers': ['logfile'],
            'level': 'DEBUG', # Or maybe INFO or WARNING
            'propagate': False
        },
    },
}

Some time has passed since EMP’s most helpful code submission. I just now implemented it, and while thrashing around with some manage.py option, to try to chase down a bug, I got a deprecation warning to the effect that with my current version of Django (1.5.?) a require_debug_false filter is now needed for the mail_admins handler.

Here is the revised code:

LOGGING = {
    'version': 1,
    'disable_existing_loggers': False,
    'filters': {
         'require_debug_false': {
             '()': 'django.utils.log.RequireDebugFalse'
         }
     },
    'handlers': {
        # Include the default Django email handler for errors
        # This is what you'd get without configuring logging at all.
        'mail_admins': {
            'class': 'django.utils.log.AdminEmailHandler',
            'level': 'ERROR',
            'filters': ['require_debug_false'],
             # But the emails are plain text by default - HTML is nicer
            'include_html': True,
        },
        # Log to a text file that can be rotated by logrotate
        'logfile': {
            'class': 'logging.handlers.WatchedFileHandler',
            'filename': '/home/username/public_html/djangoprojectname/logfilename.log'
        },
    },
    'loggers': {
        # Again, default Django configuration to email unhandled exceptions
        'django.request': {
            'handlers': ['mail_admins'],
            'level': 'ERROR',
            'propagate': True,
        },
        # Might as well log any errors anywhere else in Django
        'django': {
            'handlers': ['logfile'],
            'level': 'ERROR',
            'propagate': False,
        },
        # Your own app - this assumes all your logger names start with "myapp."
        'myapp': {
            'handlers': ['logfile'],
            'level': 'DEBUG', # Or maybe INFO or WARNING
            'propagate': False
        },
    },
}

回答 5

我的fcgi脚本有一个烦人的问题。它发生在django开始之前。缺少伐木非常痛苦。无论如何,将stderr重定向到文件作为第一件事很有帮助:

#!/home/user/env/bin/python
sys.stderr = open('/home/user/fcgi_errors', 'a')

I just had an annoying problem with my fcgi script. It occurred before django even started. The lack of logging is sooo painful. Anyway, redirecting stderr to a file as the very first thing helped a lot:

#!/home/user/env/bin/python
sys.stderr = open('/home/user/fcgi_errors', 'a')

使Python记录器将除日志文件外的所有消息输出到stdout

问题:使Python记录器将除日志文件外的所有消息输出到stdout

除了应该将日志文件放到哪里,是否有一种方法可以使用该logging模块使Python日志自动输出到stdout ?例如,我想所有呼叫,,去他们预期的地方,但除了总是被复制到。这是为了避免重复消息,例如:logger.warninglogger.criticallogger.errorstdout

mylogger.critical("something failed")
print "something failed"

Is there a way to make Python logging using the logging module automatically output things to stdout in addition to the log file where they are supposed to go? For example, I’d like all calls to logger.warning, logger.critical, logger.error to go to their intended places but in addition always be copied to stdout. This is to avoid duplicating messages like:

mylogger.critical("something failed")
print "something failed"

回答 0

所有日志记录输出均由处理程序处理;只需添加一个logging.StreamHandler()在根记录器中即可。

这是配置流处理程序(使用stdout而不是默认值stderr)并将其添加到根记录器的示例:

import logging
import sys

root = logging.getLogger()
root.setLevel(logging.DEBUG)

handler = logging.StreamHandler(sys.stdout)
handler.setLevel(logging.DEBUG)
formatter = logging.Formatter('%(asctime)s - %(name)s - %(levelname)s - %(message)s')
handler.setFormatter(formatter)
root.addHandler(handler)

All logging output is handled by the handlers; just add a logging.StreamHandler() to the root logger.

Here’s an example configuring a stream handler (using stdout instead of the default stderr) and adding it to the root logger:

import logging
import sys

root = logging.getLogger()
root.setLevel(logging.DEBUG)

handler = logging.StreamHandler(sys.stdout)
handler.setLevel(logging.DEBUG)
formatter = logging.Formatter('%(asctime)s - %(name)s - %(levelname)s - %(message)s')
handler.setFormatter(formatter)
root.addHandler(handler)

回答 1

登录到stdout的最简单方法:

import logging
import sys
logging.basicConfig(stream=sys.stdout, level=logging.DEBUG)

The simplest way to log to stdout:

import logging
import sys
logging.basicConfig(stream=sys.stdout, level=logging.DEBUG)

回答 2

可以使用多个处理程序。

import logging
import auxiliary_module

# create logger with 'spam_application'
log = logging.getLogger('spam_application')
log.setLevel(logging.DEBUG)

# create formatter and add it to the handlers
formatter = logging.Formatter('%(asctime)s - %(name)s - %(levelname)s - %(message)s')

# create file handler which logs even debug messages
fh = logging.FileHandler('spam.log')
fh.setLevel(logging.DEBUG)
fh.setFormatter(formatter)
log.addHandler(fh)

# create console handler with a higher log level
ch = logging.StreamHandler()
ch.setLevel(logging.ERROR)
ch.setFormatter(formatter)
log.addHandler(ch)

log.info('creating an instance of auxiliary_module.Auxiliary')
a = auxiliary_module.Auxiliary()
log.info('created an instance of auxiliary_module.Auxiliary')

log.info('calling auxiliary_module.Auxiliary.do_something')
a.do_something()
log.info('finished auxiliary_module.Auxiliary.do_something')

log.info('calling auxiliary_module.some_function()')
auxiliary_module.some_function()
log.info('done with auxiliary_module.some_function()')

# remember to close the handlers
for handler in log.handlers:
    handler.close()
    log.removeFilter(handler)

请参阅:https : //docs.python.org/2/howto/logging-cookbook.html

It’s possible using multiple handlers.

import logging
import auxiliary_module

# create logger with 'spam_application'
log = logging.getLogger('spam_application')
log.setLevel(logging.DEBUG)

# create formatter and add it to the handlers
formatter = logging.Formatter('%(asctime)s - %(name)s - %(levelname)s - %(message)s')

# create file handler which logs even debug messages
fh = logging.FileHandler('spam.log')
fh.setLevel(logging.DEBUG)
fh.setFormatter(formatter)
log.addHandler(fh)

# create console handler with a higher log level
ch = logging.StreamHandler()
ch.setLevel(logging.ERROR)
ch.setFormatter(formatter)
log.addHandler(ch)

log.info('creating an instance of auxiliary_module.Auxiliary')
a = auxiliary_module.Auxiliary()
log.info('created an instance of auxiliary_module.Auxiliary')

log.info('calling auxiliary_module.Auxiliary.do_something')
a.do_something()
log.info('finished auxiliary_module.Auxiliary.do_something')

log.info('calling auxiliary_module.some_function()')
auxiliary_module.some_function()
log.info('done with auxiliary_module.some_function()')

# remember to close the handlers
for handler in log.handlers:
    handler.close()
    log.removeFilter(handler)

Please see: https://docs.python.org/2/howto/logging-cookbook.html


回答 3

您可以为file和stdout创建两个处理程序,然后创建一个handlers参数为的记录器basicConfig。如果两个处理程序具有相同的log_level和format输出,则可能会很有用:

import logging
import sys

file_handler = logging.FileHandler(filename='tmp.log')
stdout_handler = logging.StreamHandler(sys.stdout)
handlers = [file_handler, stdout_handler]

logging.basicConfig(
    level=logging.DEBUG, 
    format='[%(asctime)s] {%(filename)s:%(lineno)d} %(levelname)s - %(message)s',
    handlers=handlers
)

logger = logging.getLogger('LOGGER_NAME')

You could create two handlers for file and stdout and then create one logger with handlers argument to basicConfig. It could be useful if you have the same log_level and format output for both handlers:

import logging
import sys

file_handler = logging.FileHandler(filename='tmp.log')
stdout_handler = logging.StreamHandler(sys.stdout)
handlers = [file_handler, stdout_handler]

logging.basicConfig(
    level=logging.DEBUG, 
    format='[%(asctime)s] {%(filename)s:%(lineno)d} %(levelname)s - %(message)s',
    handlers=handlers
)

logger = logging.getLogger('LOGGER_NAME')

回答 4

登录文件和stderr的最简单方法:

import logging

logging.basicConfig(filename="logfile.txt")
stderrLogger=logging.StreamHandler()
stderrLogger.setFormatter(logging.Formatter(logging.BASIC_FORMAT))
logging.getLogger().addHandler(stderrLogger)

The simplest way to log to file and to stderr:

import logging

logging.basicConfig(filename="logfile.txt")
stderrLogger=logging.StreamHandler()
stderrLogger.setFormatter(logging.Formatter(logging.BASIC_FORMAT))
logging.getLogger().addHandler(stderrLogger)

回答 5

这是基于强大的,但记录不完整的解决方案logging.config.dictConfig的方法。而不是将所有日志消息发送到stdout,而是将日志级别ERROR和更高级别的消息发送stderrstdout。如果系统的其他部分正在收听stderr或,则此功能很有用stdout

import logging
import logging.config
import sys

class _ExcludeErrorsFilter(logging.Filter):
    def filter(self, record):
        """Filters out log messages with log level ERROR (numeric value: 40) or higher."""
        return record.levelno < 40


config = {
    'version': 1,
    'filters': {
        'exclude_errors': {
            '()': _ExcludeErrorsFilter
        }
    },
    'formatters': {
        # Modify log message format here or replace with your custom formatter class
        'my_formatter': {
            'format': '(%(process)d) %(asctime)s %(name)s (line %(lineno)s) | %(levelname)s %(message)s'
        }
    },
    'handlers': {
        'console_stderr': {
            # Sends log messages with log level ERROR or higher to stderr
            'class': 'logging.StreamHandler',
            'level': 'ERROR',
            'formatter': 'my_formatter',
            'stream': sys.stderr
        },
        'console_stdout': {
            # Sends log messages with log level lower than ERROR to stdout
            'class': 'logging.StreamHandler',
            'level': 'DEBUG',
            'formatter': 'my_formatter',
            'filters': ['exclude_errors'],
            'stream': sys.stdout
        },
        'file': {
            # Sends all log messages to a file
            'class': 'logging.FileHandler',
            'level': 'DEBUG',
            'formatter': 'my_formatter',
            'filename': 'my.log',
            'encoding': 'utf8'
        }
    },
    'root': {
        # In general, this should be kept at 'NOTSET'.
        # Otherwise it would interfere with the log levels set for each handler.
        'level': 'NOTSET',
        'handlers': ['console_stderr', 'console_stdout', 'file']
    },
}

logging.config.dictConfig(config)

Here is a solution based on the powerful but poorly documented logging.config.dictConfig method. Instead of sending every log message to stdout, it sends messages with log level ERROR and higher to stderr and everything else to stdout. This can be useful if other parts of the system are listening to stderr or stdout.

import logging
import logging.config
import sys

class _ExcludeErrorsFilter(logging.Filter):
    def filter(self, record):
        """Filters out log messages with log level ERROR (numeric value: 40) or higher."""
        return record.levelno < 40


config = {
    'version': 1,
    'filters': {
        'exclude_errors': {
            '()': _ExcludeErrorsFilter
        }
    },
    'formatters': {
        # Modify log message format here or replace with your custom formatter class
        'my_formatter': {
            'format': '(%(process)d) %(asctime)s %(name)s (line %(lineno)s) | %(levelname)s %(message)s'
        }
    },
    'handlers': {
        'console_stderr': {
            # Sends log messages with log level ERROR or higher to stderr
            'class': 'logging.StreamHandler',
            'level': 'ERROR',
            'formatter': 'my_formatter',
            'stream': sys.stderr
        },
        'console_stdout': {
            # Sends log messages with log level lower than ERROR to stdout
            'class': 'logging.StreamHandler',
            'level': 'DEBUG',
            'formatter': 'my_formatter',
            'filters': ['exclude_errors'],
            'stream': sys.stdout
        },
        'file': {
            # Sends all log messages to a file
            'class': 'logging.FileHandler',
            'level': 'DEBUG',
            'formatter': 'my_formatter',
            'filename': 'my.log',
            'encoding': 'utf8'
        }
    },
    'root': {
        # In general, this should be kept at 'NOTSET'.
        # Otherwise it would interfere with the log levels set for each handler.
        'level': 'NOTSET',
        'handlers': ['console_stderr', 'console_stdout', 'file']
    },
}

logging.config.dictConfig(config)

回答 6

由于没有人共享两个整齐的班轮,我将分享我自己的:

logging.basicConfig(filename='logs.log', level=logging.DEBUG, format="%(asctime)s:%(levelname)s: %(message)s")
logging.getLogger().addHandler(logging.StreamHandler())

Since no one has shared a neat two liner, I will share my own:

logging.basicConfig(filename='logs.log', level=logging.DEBUG, format="%(asctime)s:%(levelname)s: %(message)s")
logging.getLogger().addHandler(logging.StreamHandler())

回答 7

这是一个非常简单的示例:

import logging
l = logging.getLogger("test")

# Add a file logger
f = logging.FileHandler("test.log")
l.addHandler(f)

# Add a stream logger
s = logging.StreamHandler()
l.addHandler(s)

# Send a test message to both -- critical will always log
l.critical("test msg")

输出将在标准输出和文件中显示“ test msg”。

Here’s an extremely simple example:

import logging
l = logging.getLogger("test")

# Add a file logger
f = logging.FileHandler("test.log")
l.addHandler(f)

# Add a stream logger
s = logging.StreamHandler()
l.addHandler(s)

# Send a test message to both -- critical will always log
l.critical("test msg")

The output will show “test msg” on stdout and also in the file.


Sentry 哨兵是跨平台的应用程序监控,重点是错误报告



用户和日志提供线索。哨兵提供答案。

什么是哨兵?

哨兵是一项帮助你实时监控和修复崩溃的服务。服务器使用Python,但它包含用于在任何应用程序中从任何语言发送事件的完整API



官方哨兵SDK

资源