TransactionManagementError“使用信号时,您只能在’atomic’块的末尾才能执行查询”,但仅限于单元测试期间

问题:TransactionManagementError“使用信号时,您只能在’atomic’块的末尾才能执行查询”,但仅限于单元测试期间

尝试保存Django用户模型实例时,我收到TransactionManagementError,并在其post_save信号中,保存了一些将用户作为外键的模型。

使用信号时,上下文和错误与此问题django TransactionManagementError非常相似

但是,在这种情况下,错误仅在单元测试时发生。

它在手动测试中效果很好,但是单元测试失败。

有什么我想念的吗?

以下是代码片段:

views.py

@csrf_exempt
def mobileRegister(request):
    if request.method == 'GET':
        response = {"error": "GET request not accepted!!"}
        return HttpResponse(json.dumps(response), content_type="application/json",status=500)
    elif request.method == 'POST':
        postdata = json.loads(request.body)
        try:
            # Get POST data which is to be used to save the user
            username = postdata.get('phone')
            password = postdata.get('password')
            email = postdata.get('email',"")
            first_name = postdata.get('first_name',"")
            last_name = postdata.get('last_name',"")
            user = User(username=username, email=email,
                        first_name=first_name, last_name=last_name)
            user._company = postdata.get('company',None)
            user._country_code = postdata.get('country_code',"+91")
            user.is_verified=True
            user._gcm_reg_id = postdata.get('reg_id',None)
            user._gcm_device_id = postdata.get('device_id',None)
            # Set Password for the user
            user.set_password(password)
            # Save the user
            user.save()

信号

def create_user_profile(sender, instance, created, **kwargs):
    if created:
        company = None
        companycontact = None
        try:   # Try to make userprofile with company and country code provided
            user = User.objects.get(id=instance.id)
            rand_pass = random.randint(1000, 9999)
            company = Company.objects.get_or_create(name=instance._company,user=user)
            companycontact = CompanyContact.objects.get_or_create(contact_type="Owner",company=company,contact_number=instance.username)
            profile = UserProfile.objects.get_or_create(user=instance,phone=instance.username,verification_code=rand_pass,company=company,country_code=instance._country_code)
            gcmDevice = GCMDevice.objects.create(registration_id=instance._gcm_reg_id,device_id=instance._gcm_reg_id,user=instance)
        except Exception, e:
            pass

tests.py

class AuthTestCase(TestCase):
    fixtures = ['nextgencatalogs/fixtures.json']
    def setUp(self):
        self.user_data={
            "phone":"0000000000",
            "password":"123",
            "first_name":"Gaurav",
            "last_name":"Toshniwal"
            }

    def test_registration_api_get(self):
        response = self.client.get("/mobileRegister/")
        self.assertEqual(response.status_code,500)

    def test_registration_api_post(self):
        response = self.client.post(path="/mobileRegister/",
                                    data=json.dumps(self.user_data),
                                    content_type="application/json")
        self.assertEqual(response.status_code,201)
        self.user_data['username']=self.user_data['phone']
        user = User.objects.get(username=self.user_data['username'])
        # Check if the company was created
        company = Company.objects.get(user__username=self.user_data['phone'])
        self.assertIsInstance(company,Company)
        # Check if the owner's contact is the same as the user's phone number
        company_contact = CompanyContact.objects.get(company=company,contact_type="owner")
        self.assertEqual(user.username,company_contact[0].contact_number)

追溯:

======================================================================
ERROR: test_registration_api_post (nextgencatalogs.apps.catalogsapp.tests.AuthTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/gauravtoshniwal1989/Developer/Web/Server/ngc/nextgencatalogs/apps/catalogsapp/tests.py", line 29, in test_registration_api_post
    user = User.objects.get(username=self.user_data['username'])
  File "/Users/gauravtoshniwal1989/Developer/Web/Server/ngc/ngcvenv/lib/python2.7/site-packages/django/db/models/manager.py", line 151, in get
    return self.get_queryset().get(*args, **kwargs)
  File "/Users/gauravtoshniwal1989/Developer/Web/Server/ngc/ngcvenv/lib/python2.7/site-packages/django/db/models/query.py", line 301, in get
    num = len(clone)
  File "/Users/gauravtoshniwal1989/Developer/Web/Server/ngc/ngcvenv/lib/python2.7/site-packages/django/db/models/query.py", line 77, in __len__
    self._fetch_all()
  File "/Users/gauravtoshniwal1989/Developer/Web/Server/ngc/ngcvenv/lib/python2.7/site-packages/django/db/models/query.py", line 854, in _fetch_all
    self._result_cache = list(self.iterator())
  File "/Users/gauravtoshniwal1989/Developer/Web/Server/ngc/ngcvenv/lib/python2.7/site-packages/django/db/models/query.py", line 220, in iterator
    for row in compiler.results_iter():
  File "/Users/gauravtoshniwal1989/Developer/Web/Server/ngc/ngcvenv/lib/python2.7/site-packages/django/db/models/sql/compiler.py", line 710, in results_iter
    for rows in self.execute_sql(MULTI):
  File "/Users/gauravtoshniwal1989/Developer/Web/Server/ngc/ngcvenv/lib/python2.7/site-packages/django/db/models/sql/compiler.py", line 781, in execute_sql
    cursor.execute(sql, params)
  File "/Users/gauravtoshniwal1989/Developer/Web/Server/ngc/ngcvenv/lib/python2.7/site-packages/django/db/backends/util.py", line 47, in execute
    self.db.validate_no_broken_transaction()
  File "/Users/gauravtoshniwal1989/Developer/Web/Server/ngc/ngcvenv/lib/python2.7/site-packages/django/db/backends/__init__.py", line 365, in validate_no_broken_transaction
    "An error occurred in the current transaction. You can't "
TransactionManagementError: An error occurred in the current transaction. You can't execute queries until the end of the 'atomic' block.

----------------------------------------------------------------------

I am getting TransactionManagementError when trying to save a Django User model instance and in its post_save signal, I’m saving some models that have the user as the foreign key.

The context and error is pretty similar to this question django TransactionManagementError when using signals

However, in this case, the error occurs only while unit testing.

It works well in manual testing, but unit tests fails.

Is there anything that I’m missing?

Here are the code snippets:

views.py

@csrf_exempt
def mobileRegister(request):
    if request.method == 'GET':
        response = {"error": "GET request not accepted!!"}
        return HttpResponse(json.dumps(response), content_type="application/json",status=500)
    elif request.method == 'POST':
        postdata = json.loads(request.body)
        try:
            # Get POST data which is to be used to save the user
            username = postdata.get('phone')
            password = postdata.get('password')
            email = postdata.get('email',"")
            first_name = postdata.get('first_name',"")
            last_name = postdata.get('last_name',"")
            user = User(username=username, email=email,
                        first_name=first_name, last_name=last_name)
            user._company = postdata.get('company',None)
            user._country_code = postdata.get('country_code',"+91")
            user.is_verified=True
            user._gcm_reg_id = postdata.get('reg_id',None)
            user._gcm_device_id = postdata.get('device_id',None)
            # Set Password for the user
            user.set_password(password)
            # Save the user
            user.save()

signal.py

def create_user_profile(sender, instance, created, **kwargs):
    if created:
        company = None
        companycontact = None
        try:   # Try to make userprofile with company and country code provided
            user = User.objects.get(id=instance.id)
            rand_pass = random.randint(1000, 9999)
            company = Company.objects.get_or_create(name=instance._company,user=user)
            companycontact = CompanyContact.objects.get_or_create(contact_type="Owner",company=company,contact_number=instance.username)
            profile = UserProfile.objects.get_or_create(user=instance,phone=instance.username,verification_code=rand_pass,company=company,country_code=instance._country_code)
            gcmDevice = GCMDevice.objects.create(registration_id=instance._gcm_reg_id,device_id=instance._gcm_reg_id,user=instance)
        except Exception, e:
            pass

tests.py

class AuthTestCase(TestCase):
    fixtures = ['nextgencatalogs/fixtures.json']
    def setUp(self):
        self.user_data={
            "phone":"0000000000",
            "password":"123",
            "first_name":"Gaurav",
            "last_name":"Toshniwal"
            }

    def test_registration_api_get(self):
        response = self.client.get("/mobileRegister/")
        self.assertEqual(response.status_code,500)

    def test_registration_api_post(self):
        response = self.client.post(path="/mobileRegister/",
                                    data=json.dumps(self.user_data),
                                    content_type="application/json")
        self.assertEqual(response.status_code,201)
        self.user_data['username']=self.user_data['phone']
        user = User.objects.get(username=self.user_data['username'])
        # Check if the company was created
        company = Company.objects.get(user__username=self.user_data['phone'])
        self.assertIsInstance(company,Company)
        # Check if the owner's contact is the same as the user's phone number
        company_contact = CompanyContact.objects.get(company=company,contact_type="owner")
        self.assertEqual(user.username,company_contact[0].contact_number)

Traceback:

======================================================================
ERROR: test_registration_api_post (nextgencatalogs.apps.catalogsapp.tests.AuthTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/gauravtoshniwal1989/Developer/Web/Server/ngc/nextgencatalogs/apps/catalogsapp/tests.py", line 29, in test_registration_api_post
    user = User.objects.get(username=self.user_data['username'])
  File "/Users/gauravtoshniwal1989/Developer/Web/Server/ngc/ngcvenv/lib/python2.7/site-packages/django/db/models/manager.py", line 151, in get
    return self.get_queryset().get(*args, **kwargs)
  File "/Users/gauravtoshniwal1989/Developer/Web/Server/ngc/ngcvenv/lib/python2.7/site-packages/django/db/models/query.py", line 301, in get
    num = len(clone)
  File "/Users/gauravtoshniwal1989/Developer/Web/Server/ngc/ngcvenv/lib/python2.7/site-packages/django/db/models/query.py", line 77, in __len__
    self._fetch_all()
  File "/Users/gauravtoshniwal1989/Developer/Web/Server/ngc/ngcvenv/lib/python2.7/site-packages/django/db/models/query.py", line 854, in _fetch_all
    self._result_cache = list(self.iterator())
  File "/Users/gauravtoshniwal1989/Developer/Web/Server/ngc/ngcvenv/lib/python2.7/site-packages/django/db/models/query.py", line 220, in iterator
    for row in compiler.results_iter():
  File "/Users/gauravtoshniwal1989/Developer/Web/Server/ngc/ngcvenv/lib/python2.7/site-packages/django/db/models/sql/compiler.py", line 710, in results_iter
    for rows in self.execute_sql(MULTI):
  File "/Users/gauravtoshniwal1989/Developer/Web/Server/ngc/ngcvenv/lib/python2.7/site-packages/django/db/models/sql/compiler.py", line 781, in execute_sql
    cursor.execute(sql, params)
  File "/Users/gauravtoshniwal1989/Developer/Web/Server/ngc/ngcvenv/lib/python2.7/site-packages/django/db/backends/util.py", line 47, in execute
    self.db.validate_no_broken_transaction()
  File "/Users/gauravtoshniwal1989/Developer/Web/Server/ngc/ngcvenv/lib/python2.7/site-packages/django/db/backends/__init__.py", line 365, in validate_no_broken_transaction
    "An error occurred in the current transaction. You can't "
TransactionManagementError: An error occurred in the current transaction. You can't execute queries until the end of the 'atomic' block.

----------------------------------------------------------------------

回答 0

我本人也遇到了同样的问题。这是由于在较新版本的Django中如何处理事务的古怪之处,加上故意触发异常的单元测试。

我有一个单元测试,通过有意触发IntegrityError异常来检查以确保实施了唯一的列约束:

def test_constraint(self):
    try:
        # Duplicates should be prevented.
        models.Question.objects.create(domain=self.domain, slug='barks')
        self.fail('Duplicate question allowed.')
    except IntegrityError:
        pass

    do_more_model_stuff()

在Django 1.4中,这可以正常工作。但是,在Django 1.5 / 1.6中,每个测试都包装在一个事务中,因此,如果发生异常,它将破坏该事务,直到您明确地将其回滚为止。因此,该事务中任何进一步的ORM操作(例如my do_more_model_stuff())都将因该django.db.transaction.TransactionManagementError异常而失败。

就像注释中提到的caio一样,解决方案是使用以下方式捕获您的异常transaction.atomic

from django.db import transaction
def test_constraint(self):
    try:
        # Duplicates should be prevented.
        with transaction.atomic():
            models.Question.objects.create(domain=self.domain, slug='barks')
        self.fail('Duplicate question allowed.')
    except IntegrityError:
        pass

这将防止故意抛出的异常破坏整个单元测试的事务。

I ran into this same problem myself. This is caused by a quirk in how transactions are handled in the newer versions of Django coupled with a unittest that intentionally triggers an exception.

I had a unittest that checked to make sure a unique column constraint was enforced by purposefully triggering an IntegrityError exception:

def test_constraint(self):
    try:
        # Duplicates should be prevented.
        models.Question.objects.create(domain=self.domain, slug='barks')
        self.fail('Duplicate question allowed.')
    except IntegrityError:
        pass

    do_more_model_stuff()

In Django 1.4, this works fine. However, in Django 1.5/1.6, each test is wrapped in a transaction, so if an exception occurs, it breaks the transaction until you explicitly roll it back. Therefore, any further ORM operations in that transaction, such as my do_more_model_stuff(), will fail with that django.db.transaction.TransactionManagementError exception.

Like caio mentioned in the comments, the solution is to capture your exception with transaction.atomic like:

from django.db import transaction
def test_constraint(self):
    try:
        # Duplicates should be prevented.
        with transaction.atomic():
            models.Question.objects.create(domain=self.domain, slug='barks')
        self.fail('Duplicate question allowed.')
    except IntegrityError:
        pass

That will prevent the purposefully-thrown exception from breaking the entire unittest’s transaction.


回答 1

由于@mkoistinen从未发表过他的评论(答案),因此我将发布他的建议,这样人们就不必再仔细研究评论了。

考虑只将测试类声明为TransactionTestCase而不是TestCase。

文档:TransactionTestCase可以调用提交和回滚,并观察这些调用对数据库的影响。

Since @mkoistinen never made his comment, an answer, I’ll post his suggestion so people won’t have to dig through comments.

consider just declaring your test class as a TransactionTestCase rather than just TestCase.

From the Django docs: A TransactionTestCase may call commit and rollback and observe the effects of these calls on the database.


回答 2

如果使用pytest-django,则可以传递transaction=Truedjango_db装饰器以避免此错误。

请参阅https://pytest-django.readthedocs.io/en/latest/database.html#testing-transactions

Django本身具有TransactionTestCase,可让您测试事务并在两次测试之间刷新数据库以隔离它们。缺点是,由于需要刷新数据库,因此这些测试的建立速度要慢得多。pytest-django也支持这种测试风格,您可以使用django_db标记的参数进行选择:

@pytest.mark.django_db(transaction=True)
def test_spam():
    pass  # test relying on transactions

If using pytest-django you can pass transaction=True to the django_db decorator to avoid this error.

See https://pytest-django.readthedocs.io/en/latest/database.html#testing-transactions

Django itself has the TransactionTestCase which allows you to test transactions and will flush the database between tests to isolate them. The downside of this is that these tests are much slower to set up due to the required flushing of the database. pytest-django also supports this style of tests, which you can select using an argument to the django_db mark:

@pytest.mark.django_db(transaction=True)
def test_spam():
    pass  # test relying on transactions

回答 3

对我来说,建议的修复程序不起作用。在我的测试中,我打开了一些Popen用于分析/皮棉迁移的子流程(例如,一项测试检查是否没有模型更改)。

对我来说,从而SimpleTestCase不是继承TestCase确实可以解决问题。

请注意,SimpleTestCase不允许使用数据库。

尽管这不能回答最初的问题,但我希望这对某些人有帮助。

For me, the proposed fixes did not work. In my tests, I open some subprocesses with Popen to analyze/lint migrations (e.g. one test checks if there are no model changes).

For me, subclassing from SimpleTestCase instead of TestCase did do the trick.

Note that SimpleTestCase doesn’t allow to use the database.

While this does not answer the original question, I hope this helps some people anyway.


回答 4

这是基于此问题的答案的另一种方法:

with transaction.atomic():
    self.assertRaises(IntegrityError, models.Question.objects.create, **{'domain':self.domain, 'slug':'barks'})

Here is another way to do it, based on the answer to this question:

with transaction.atomic():
    self.assertRaises(IntegrityError, models.Question.objects.create, **{'domain':self.domain, 'slug':'barks'})

回答 5

我在使用Django 1.9.7在create_test_data函数中运行单元测试时遇到此错误。它在Django的早期版本中工作。

它看起来像这样:

cls.localauth,_ = Organisation.objects.get_or_create(organisation_type=cls.orgtypeLA, name='LA for test', email_general='test@test.com', address='test', postcode='test', telephone='test')
cls.chamber,_ = Organisation.objects.get_or_create(organisation_type=cls.orgtypeC, name='chamber for test', email_general='test@test.com', address='test', postcode='test', telephone='test')
cls.lawfirm,_ = Organisation.objects.get_or_create(organisation_type=cls.orgtypeL, name='lawfirm for test', email_general='test@test.com', address='test', postcode='test', telephone='test')

cls.chamber.active = True
cls.chamber.save()

cls.localauth.active = True
cls.localauth.save()    <---- error here

cls.lawfirm.active = True
cls.lawfirm.save()

我的解决方案是改用update_or_create:

cls.localauth,_ = Organisation.objects.update_or_create(organisation_type=cls.orgtypeLA, name='LA for test', email_general='test@test.com', address='test', postcode='test', telephone='test', defaults={'active': True})
cls.chamber,_ = Organisation.objects.update_or_create(organisation_type=cls.orgtypeC, name='chamber for test', email_general='test@test.com', address='test', postcode='test', telephone='test', defaults={'active': True})
cls.lawfirm,_ = Organisation.objects.update_or_create(organisation_type=cls.orgtypeL, name='lawfirm for test', email_general='test@test.com', address='test', postcode='test', telephone='test', defaults={'active': True})

I was getting this error on running unit tests in my create_test_data function using django 1.9.7. It worked in earlier versions of django.

It looked like this:

cls.localauth,_ = Organisation.objects.get_or_create(organisation_type=cls.orgtypeLA, name='LA for test', email_general='test@test.com', address='test', postcode='test', telephone='test')
cls.chamber,_ = Organisation.objects.get_or_create(organisation_type=cls.orgtypeC, name='chamber for test', email_general='test@test.com', address='test', postcode='test', telephone='test')
cls.lawfirm,_ = Organisation.objects.get_or_create(organisation_type=cls.orgtypeL, name='lawfirm for test', email_general='test@test.com', address='test', postcode='test', telephone='test')

cls.chamber.active = True
cls.chamber.save()

cls.localauth.active = True
cls.localauth.save()    <---- error here

cls.lawfirm.active = True
cls.lawfirm.save()

My solution was to use update_or_create instead:

cls.localauth,_ = Organisation.objects.update_or_create(organisation_type=cls.orgtypeLA, name='LA for test', email_general='test@test.com', address='test', postcode='test', telephone='test', defaults={'active': True})
cls.chamber,_ = Organisation.objects.update_or_create(organisation_type=cls.orgtypeC, name='chamber for test', email_general='test@test.com', address='test', postcode='test', telephone='test', defaults={'active': True})
cls.lawfirm,_ = Organisation.objects.update_or_create(organisation_type=cls.orgtypeL, name='lawfirm for test', email_general='test@test.com', address='test', postcode='test', telephone='test', defaults={'active': True})

回答 6

我有同样的问题,但with transaction.atomic()TransactionTestCase我没有工作。

python manage.py test -r而不是python manage.py test对我来说没关系,也许执行顺序很关键

然后我找到了有关要在其中执行测试的Order的文档,其中提到了哪个测试将首先运行。

因此,我使用TestCase进行数据库交互,unittest.TestCase进行其他简单测试,现在可以使用!

I have the same issue, but with transaction.atomic() and TransactionTestCase didn’t work for me.

python manage.py test -r instead of python manage.py test is ok for me, maybe the order of execution is crucial

then i find a doc about Order in which tests are executed, It mentions which test will run first.

So, i use TestCase for database interaction, unittest.TestCase for other simple test, it works now!


回答 7

@kdazzle的答案是正确的。我没有尝试过,因为人们说“ Django的TestCase类是TransactionTestCase的一个更常用的子类”,所以我认为它的用途是相同的。但是Jahongir Rahmonov博客对此进行了更好的解释:

TestCase类将测试包装在两个嵌套的atomic()块内:一个用于整个类,一个用于每个测试。这是应该使用TransactionTestCase的地方。它不使用atomic()块包装测试,因此您可以测试需要事务的特殊方法,而不会出现任何问题。

编辑:这没有用,我想是的,但是没有。

他们可以在4年内解决此问题………………………………

The answer of @kdazzle is correct. I didnt try it because people said that ‘Django’s TestCase class is a more commonly used subclass of TransactionTestCase’ so I thought it was the same use one or another. But the blog of Jahongir Rahmonov explained it better:

the TestCase class wraps the tests within two nested atomic() blocks: one for the whole class and one for each test. This is where TransactionTestCase should be used. It does not wrap the tests with atomic() block and thus you can test your special methods that require a transaction without any problem.

EDIT: It didn’t work, I thought yes, but NO.

In 4 years they could fixed this…………………………………


回答 8

def test_wrong_user_country_db_constraint(self):
        """
        Check whether or not DB constraint doesnt allow to save wrong country code in DB.
        """
        self.test_user_data['user_country'] = 'XX'
        expected_constraint_name = "country_code_within_list_of_countries_check"

        with transaction.atomic():
            with self.assertRaisesRegex(IntegrityError, expected_constraint_name) as cm:
                get_user_model().objects.create_user(**self.test_user_data)

        self.assertFalse(
            get_user_model().objects.filter(email=self.test_user_data['email']).exists()
        )
with transaction.atomic() seems do the job correct
def test_wrong_user_country_db_constraint(self):
        """
        Check whether or not DB constraint doesnt allow to save wrong country code in DB.
        """
        self.test_user_data['user_country'] = 'XX'
        expected_constraint_name = "country_code_within_list_of_countries_check"

        with transaction.atomic():
            with self.assertRaisesRegex(IntegrityError, expected_constraint_name) as cm:
                get_user_model().objects.create_user(**self.test_user_data)

        self.assertFalse(
            get_user_model().objects.filter(email=self.test_user_data['email']).exists()
        )
with transaction.atomic() seems do the job correct

回答 9

我遇到过同样的问题。

就我而言,我正在这样做

author.tasks.add(tasks)

所以将其转换为

author.tasks.add(*tasks)

删除了该错误。

I had the same issue.

In My Case I was doing this

author.tasks.add(tasks)

so converting it to

author.tasks.add(*tasks)

Removed that error.