带回溯的日志异常

问题:带回溯的日志异常

如何记录我的Python错误?

try:
    do_something()
except:
    # How can I log my exception here, complete with its traceback?

How can I log my Python errors?

try:
    do_something()
except:
    # How can I log my exception here, complete with its traceback?

回答 0

使用logging.exception从内except:处理/块与跟踪信息,与消息前缀一起记录当前异常。

import logging
LOG_FILENAME = '/tmp/logging_example.out'
logging.basicConfig(filename=LOG_FILENAME, level=logging.DEBUG)

logging.debug('This message should go to the log file')

try:
    run_my_stuff()
except:
    logging.exception('Got exception on main handler')
    raise

现在查看日志文件/tmp/logging_example.out

DEBUG:root:This message should go to the log file
ERROR:root:Got exception on main handler
Traceback (most recent call last):
  File "/tmp/teste.py", line 9, in <module>
    run_my_stuff()
NameError: name 'run_my_stuff' is not defined

Use logging.exception from within the except: handler/block to log the current exception along with the trace information, prepended with a message.

import logging
LOG_FILENAME = '/tmp/logging_example.out'
logging.basicConfig(filename=LOG_FILENAME, level=logging.DEBUG)

logging.debug('This message should go to the log file')

try:
    run_my_stuff()
except:
    logging.exception('Got exception on main handler')
    raise

Now looking at the log file, /tmp/logging_example.out:

DEBUG:root:This message should go to the log file
ERROR:root:Got exception on main handler
Traceback (most recent call last):
  File "/tmp/teste.py", line 9, in <module>
    run_my_stuff()
NameError: name 'run_my_stuff' is not defined

回答 1

使用exc_info选项可能更好,但仍会显示警告或错误标题:

try:
    # coode in here
except Exception as e:
    logging.error(e, exc_info=True)

Use exc_info options may be better, remains warning or error title:

try:
    # coode in here
except Exception as e:
    logging.error(e, exc_info=True)

回答 2

最近,我的工作要求我记录应用程序中的所有回溯/异常。我尝试了其他人在网上发布的许多技术,例如上面的一种,但选择了另一种方法。覆盖traceback.print_exception

我在http://www.bbarrows.com/上写了一篇文章,它很容易阅读,但我也会将其粘贴在这里。

当任务是记录我们的软件在野外可能遇到的所有异常时,我尝试了多种不同的技术来记录我们的python异常回溯。起初,我认为python系统异常挂钩sys.excepthook将是插入日志记录代码的理想场所。我正在尝试类似的东西:

import traceback
import StringIO
import logging
import os, sys

def my_excepthook(excType, excValue, traceback, logger=logger):
    logger.error("Logging an uncaught exception",
                 exc_info=(excType, excValue, traceback))

sys.excepthook = my_excepthook  

这适用于主线程,但是我很快发现我的sys.excepthook在进程启动的任何新线程中都不存在。这是一个很大的问题,因为大多数事情都发生在该项目的线程中。

仔细阅读并阅读大量文档后,我发现最有用的信息来自Python问题跟踪器。

线程的第一篇文章显示了一个sys.excepthook跨线程不持久的工作示例(如下所示)。显然,这是预期的行为。

import sys, threading

def log_exception(*args):
    print 'got exception %s' % (args,)
sys.excepthook = log_exception

def foo():
    a = 1 / 0

threading.Thread(target=foo).start()

该Python Issue线程上的消息确实导致了2条建议的hack。可以将子类Thread并将run方法包装在我们自己的tryexcept块中以捕获和记录异常,或者将Monkey补丁threading.Thread.run以您自己的tryexcept块中的方式运行,以阻止和记录异常。

Thread我看来,第一种子类化方法在您的代码中似乎不太优雅,因为您必须在Thread想要拥有日志记录线程的任何地方导入和使用自定义类。最终这很麻烦,因为我不得不搜索我们的整个代码库,并Threads用此自定义替换所有常规代码Thread。但是,很清楚这Thread是在做什么,如果自定义日志代码出了问题,则对于某人来说,诊断和调试将更容易。定制日志记录线程可能如下所示:

class TracebackLoggingThread(threading.Thread):
    def run(self):
        try:
            super(TracebackLoggingThread, self).run()
        except (KeyboardInterrupt, SystemExit):
            raise
        except Exception, e:
            logger = logging.getLogger('')
            logger.exception("Logging an uncaught exception")

Monkey修补的第二种方法threading.Thread.run很好,因为我可以立即运行一次,__main__并在所有异常中检测日志记录代码。Monkey修补可能会令人讨厌调试,因为它会更改某些功能的预期功能。来自Python问题跟踪器的建议补丁为:

def installThreadExcepthook():
    """
    Workaround for sys.excepthook thread bug
    From
http://spyced.blogspot.com/2007/06/workaround-for-sysexcepthook-bug.html

(https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1230540&group_id=5470).
    Call once from __main__ before creating any threads.
    If using psyco, call psyco.cannotcompile(threading.Thread.run)
    since this replaces a new-style class method.
    """
    init_old = threading.Thread.__init__
    def init(self, *args, **kwargs):
        init_old(self, *args, **kwargs)
        run_old = self.run
        def run_with_except_hook(*args, **kw):
            try:
                run_old(*args, **kw)
            except (KeyboardInterrupt, SystemExit):
                raise
            except:
                sys.excepthook(*sys.exc_info())
        self.run = run_with_except_hook
    threading.Thread.__init__ = init

直到我开始测试异常日志记录时,我才意识到自己在处理所有错误。

为了测试,我放置了一个

raise Exception("Test")

在我的代码中的某处。但是,包装一个称为该方法的方法是一种尝试,除了打印出回溯并吞没了异常的块。这非常令人沮丧,因为我看到回溯将打印输出到STDOUT,但是没有被记录下来。然后我决定,记录回溯的一种更简单的方法就是Monkey补丁所有Python代码用来打印回溯的方法traceback.print_exception。我最终得到了类似于以下内容的东西:

def add_custom_print_exception():
    old_print_exception = traceback.print_exception
    def custom_print_exception(etype, value, tb, limit=None, file=None):
        tb_output = StringIO.StringIO()
        traceback.print_tb(tb, limit, tb_output)
        logger = logging.getLogger('customLogger')
        logger.error(tb_output.getvalue())
        tb_output.close()
        old_print_exception(etype, value, tb, limit=None, file=None)
    traceback.print_exception = custom_print_exception

此代码将回溯写到字符串缓冲区,并将其记录到日志记录错误中。我有一个自定义日志记录处理程序,它设置了’customLogger’记录器,该记录器将使用ERROR级日志并将其发送回家进行分析。

My job recently tasked me with logging all the tracebacks/exceptions from our application. I tried numerous techniques that others had posted online such as the one above but settled on a different approach. Overriding traceback.print_exception.

I have a write up at http://www.bbarrows.com/ That would be much easier to read but Ill paste it in here as well.

When tasked with logging all the exceptions that our software might encounter in the wild I tried a number of different techniques to log our python exception tracebacks. At first I thought that the python system exception hook, sys.excepthook would be the perfect place to insert the logging code. I was trying something similar to:

import traceback
import StringIO
import logging
import os, sys

def my_excepthook(excType, excValue, traceback, logger=logger):
    logger.error("Logging an uncaught exception",
                 exc_info=(excType, excValue, traceback))

sys.excepthook = my_excepthook  

This worked for the main thread but I soon found that the my sys.excepthook would not exist across any new threads my process started. This is a huge issue because most everything happens in threads in this project.

After googling and reading plenty of documentation the most helpful information I found was from the Python Issue tracker.

The first post on the thread shows a working example of the sys.excepthook NOT persisting across threads (as shown below). Apparently this is expected behavior.

import sys, threading

def log_exception(*args):
    print 'got exception %s' % (args,)
sys.excepthook = log_exception

def foo():
    a = 1 / 0

threading.Thread(target=foo).start()

The messages on this Python Issue thread really result in 2 suggested hacks. Either subclass Thread and wrap the run method in our own try except block in order to catch and log exceptions or monkey patch threading.Thread.run to run in your own try except block and log the exceptions.

The first method of subclassing Thread seems to me to be less elegant in your code as you would have to import and use your custom Thread class EVERYWHERE you wanted to have a logging thread. This ended up being a hassle because I had to search our entire code base and replace all normal Threads with this custom Thread. However, it was clear as to what this Thread was doing and would be easier for someone to diagnose and debug if something went wrong with the custom logging code. A custome logging thread might look like this:

class TracebackLoggingThread(threading.Thread):
    def run(self):
        try:
            super(TracebackLoggingThread, self).run()
        except (KeyboardInterrupt, SystemExit):
            raise
        except Exception, e:
            logger = logging.getLogger('')
            logger.exception("Logging an uncaught exception")

The second method of monkey patching threading.Thread.run is nice because I could just run it once right after __main__ and instrument my logging code in all exceptions. Monkey patching can be annoying to debug though as it changes the expected functionality of something. The suggested patch from the Python Issue tracker was:

def installThreadExcepthook():
    """
    Workaround for sys.excepthook thread bug
    From
http://spyced.blogspot.com/2007/06/workaround-for-sysexcepthook-bug.html

(https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1230540&group_id=5470).
    Call once from __main__ before creating any threads.
    If using psyco, call psyco.cannotcompile(threading.Thread.run)
    since this replaces a new-style class method.
    """
    init_old = threading.Thread.__init__
    def init(self, *args, **kwargs):
        init_old(self, *args, **kwargs)
        run_old = self.run
        def run_with_except_hook(*args, **kw):
            try:
                run_old(*args, **kw)
            except (KeyboardInterrupt, SystemExit):
                raise
            except:
                sys.excepthook(*sys.exc_info())
        self.run = run_with_except_hook
    threading.Thread.__init__ = init

It was not until I started testing my exception logging I realized that I was going about it all wrong.

To test I had placed a

raise Exception("Test")

somewhere in my code. However, wrapping a a method that called this method was a try except block that printed out the traceback and swallowed the exception. This was very frustrating because I saw the traceback bring printed to STDOUT but not being logged. It was I then decided that a much easier method of logging the tracebacks was just to monkey patch the method that all python code uses to print the tracebacks themselves, traceback.print_exception. I ended up with something similar to the following:

def add_custom_print_exception():
    old_print_exception = traceback.print_exception
    def custom_print_exception(etype, value, tb, limit=None, file=None):
        tb_output = StringIO.StringIO()
        traceback.print_tb(tb, limit, tb_output)
        logger = logging.getLogger('customLogger')
        logger.error(tb_output.getvalue())
        tb_output.close()
        old_print_exception(etype, value, tb, limit=None, file=None)
    traceback.print_exception = custom_print_exception

This code writes the traceback to a String Buffer and logs it to logging ERROR. I have a custom logging handler set up the ‘customLogger’ logger which takes the ERROR level logs and send them home for analysis.


回答 3

您可以通过将处理程序分配给来记录主线程上所有未捕获的异常sys.excepthook,也许使用exc_infoPython的记录函数参数

import sys
import logging

logging.basicConfig(filename='/tmp/foobar.log')

def exception_hook(exc_type, exc_value, exc_traceback):
    logging.error(
        "Uncaught exception",
        exc_info=(exc_type, exc_value, exc_traceback)
    )

sys.excepthook = exception_hook

raise Exception('Boom')

如果你的程序使用的线程,然而,然后记下创建的线程使用threading.Thread不会触发sys.excepthook时未捕获的异常在他们里面发生,在指出问题1230540 Python的问题跟踪器。已经有人提出了一些可以解决此限制的技巧,例如Monkey修补Thread.__init__程序self.run用另run一种方法覆盖,该方法将原始文件包装在一个try块中并sys.excepthook从该except块内部进行调用。或者,您可以手动将每个线程的入口点包装在try/ except自己中。

You can log all uncaught exceptions on the main thread by assigning a handler to sys.excepthook, perhaps using the exc_info parameter of Python’s logging functions:

import sys
import logging

logging.basicConfig(filename='/tmp/foobar.log')

def exception_hook(exc_type, exc_value, exc_traceback):
    logging.error(
        "Uncaught exception",
        exc_info=(exc_type, exc_value, exc_traceback)
    )

sys.excepthook = exception_hook

raise Exception('Boom')

If your program uses threads, however, then note that threads created using threading.Thread will not trigger sys.excepthook when an uncaught exception occurs inside them, as noted in Issue 1230540 on Python’s issue tracker. Some hacks have been suggested there to work around this limitation, like monkey-patching Thread.__init__ to overwrite self.run with an alternative run method that wraps the original in a try block and calls sys.excepthook from inside the except block. Alternatively, you could just manually wrap the entry point for each of your threads in try/except yourself.


回答 4

未捕获的异常消息将发送到STDERR,因此,您可以使用用于运行Python脚本的任何Shell将STDERR发送到文件,而不是在Python本身中实现日志记录。在Bash脚本中,您可以使用输出重定向来执行此操作,如BASH指南中所述

例子

将错误附加到文件,其他输出到终端:

./test.py 2>> mylog.log

用交错的STDOUT和STDERR输出覆盖文件:

./test.py &> mylog.log

Uncaught exception messages go to STDERR, so instead of implementing your logging in Python itself you could send STDERR to a file using whatever shell you’re using to run your Python script. In a Bash script, you can do this with output redirection, as described in the BASH guide.

Examples

Append errors to file, other output to the terminal:

./test.py 2>> mylog.log

Overwrite file with interleaved STDOUT and STDERR output:

./test.py &> mylog.log

回答 5

我在寻找什么:

import sys
import traceback

exc_type, exc_value, exc_traceback = sys.exc_info()
traceback_in_var = traceback.format_tb(exc_traceback)

看到:

What I was looking for:

import sys
import traceback

exc_type, exc_value, exc_traceback = sys.exc_info()
traceback_in_var = traceback.format_tb(exc_traceback)

See:


回答 6

您可以使用记录器在任何级别(调试,信息等)获取回溯。请注意,使用logging.exception,级别为ERROR。

# test_app.py
import sys
import logging

logging.basicConfig(level="DEBUG")

def do_something():
    raise ValueError(":(")

try:
    do_something()
except Exception:
    logging.debug("Something went wrong", exc_info=sys.exc_info())
DEBUG:root:Something went wrong
Traceback (most recent call last):
  File "test_app.py", line 10, in <module>
    do_something()
  File "test_app.py", line 7, in do_something
    raise ValueError(":(")
ValueError: :(

编辑:

这也可以工作(使用python 3.6)

logging.debug("Something went wrong", exc_info=True)

You can get the traceback using a logger, at any level (DEBUG, INFO, …). Note that using logging.exception, the level is ERROR.

# test_app.py
import sys
import logging

logging.basicConfig(level="DEBUG")

def do_something():
    raise ValueError(":(")

try:
    do_something()
except Exception:
    logging.debug("Something went wrong", exc_info=sys.exc_info())
DEBUG:root:Something went wrong
Traceback (most recent call last):
  File "test_app.py", line 10, in <module>
    do_something()
  File "test_app.py", line 7, in do_something
    raise ValueError(":(")
ValueError: :(

EDIT:

This works too (using python 3.6)

logging.debug("Something went wrong", exc_info=True)

回答 7

这是使用sys.excepthook的版本

import traceback
import sys

logger = logging.getLogger()

def handle_excepthook(type, message, stack):
     logger.error(f'An unhandled exception occured: {message}. Traceback: {traceback.format_tb(stack)}')

sys.excepthook = handle_excepthook

Here is a version that uses sys.excepthook

import traceback
import sys

logger = logging.getLogger()

def handle_excepthook(type, message, stack):
     logger.error(f'An unhandled exception occured: {message}. Traceback: {traceback.format_tb(stack)}')

sys.excepthook = handle_excepthook

回答 8

也许不那么时尚,但是更容易:

#!/bin/bash
log="/var/log/yourlog"
/path/to/your/script.py 2>&1 | (while read; do echo "$REPLY" >> $log; done)

maybe not as stylish, but easier:

#!/bin/bash
log="/var/log/yourlog"
/path/to/your/script.py 2>&1 | (while read; do echo "$REPLY" >> $log; done)

回答 9

这是取自python 2.6文档的一个简单示例:

import logging
LOG_FILENAME = '/tmp/logging_example.out'
logging.basicConfig(filename=LOG_FILENAME,level=logging.DEBUG,)

logging.debug('This message should go to the log file')

Heres a simple example taken from the python 2.6 documentation:

import logging
LOG_FILENAME = '/tmp/logging_example.out'
logging.basicConfig(filename=LOG_FILENAME,level=logging.DEBUG,)

logging.debug('This message should go to the log file')