Python 实用宝典

Question 1

Which are the most advanced frameworks and tools there are available for python for practicing Behavior Driven Development? Especially finding similar tools as rspec and mocha for ruby would be great.

Question 2

Ian Bicking recommends using doctest for behavior driven design:

I personally tend to use nose and voidspace mock in a behavior driven design style. Specifically, the spec plugin for nose is excellent for BDD.

Question 3

Lettuce means to be a cucumber-like tool for python: http://lettuce.it/

You can grab the source at github.com/gabrielfalcao/lettuce

Question 4

I really recommend behave.

Looking for a Cucumber clone for Python, I started using lettuce, but found it a pretty clumsily designed replica. Very Unpythonic.

Then I discovered behave, and have been really happy with it.

Question 5

I recommend you to use a set of tools developed to help programmers in the practice of BDD and TDD. This tool set is composed by: pycukes, specloud, ludibrio and should-dsl.

Should-DSL will give you RSpec-like expectations. Everything you can do with RSpec expectation API, should-dsl does too. You can grab the latestversion from Github.

SpecLoud helps you on running BDD-like unittests. You can install it by doing

pip install specloud

Ludibrio is a library for test doubles (Mocks, Stubs and Dummies). Install it via

pip install ludibrio

And PyCukes is the main tool for BDD. It will run the Scenarios, etc. Again,

pip install pycukes

For more info please read the tools documentation at PyPi.

Question 6

Great post and answers. Just wanted to update to include Freshen in this list as I read pycukes is discontinued. A good post about using BDD and Django with Freshen is here.

Question 7

You can use “sure” for expressive assertions (just like in RSpec)

Question 8

The Pyccuracy project is an effort to provide a domain-specific language for BDD in Python.

Unlike doctest, which works at the API level, it encodes higher-level operations such as loading a web page and submitting a form. I haven’t used it but it looks somewhat promising if that is what you’re looking for.

Question 9

I like Pyccuracy a lot. I’m implementing it on a mid sized project these days.

Question 10

Try out pyspecs. Making tests easy to read and constantly running during development were two of my main goals in creating this project.

Test Code:

from pyspecs import given, when, then, and_, the, this

with given.two_operands:
    a = 2
    b = 3

    with when.supplied_to_the_add_function:
        total = a + b

        with then.the_total_should_be_mathmatically_correct:
            the(total).should.equal(5)

        with and_.the_total_should_be_greater_than_either_operand:
            the(total).should.be_greater_than(a)
            the(total).should.be_greater_than(b)

    with when.supplied_to_the_subtract_function:
        difference = b - a

        with then.the_difference_should_be_mathmatically_correct:
            the(difference).should.equal(1)

Console Output:

# run_pyspecs.py

  | • given two operands 
  |   • when supplied to the add function 
  |     • then the total should be mathmatically correct 
  |     • and the total should be greater than either operand 
  |   • when supplied to the subtract function 
  |     • then the difference should be mathmatically correct 

(ok) 6 passed (6 steps, 1 scenarios in 0.0002 seconds)

Question 11

I am probably completely missing the point, but what I retained of the original BDD paper was that BDD was just TDD repackaged to emphasize some best practices.

If my interpretation is correct, you can get a BDD framework just by renaming methods around in any xUnit implementation. So just go ahead and use the standard library’s unittest.

EDIT: A quick google turned up a Behaviour module in the Cheese Shop. Further searching for BDD there did not find anything else.

Question 12

I need for testing purposes to populate few hundred email boxes with various messages, and was going to use smtplib for that. But among other things I need to be able to send messages not only TO specific mailboxes, but CC and BCC them as well. It does not look like smtplib supports CC-ing and BCC-ing while sending emails.

Looking for suggestions how to do CC or BCC sending messages from the python script.

(And — no, I’m not creating a script to spam anyone outside of my testing environment.)

Question 13

Email headers don’t matter to the smtp server. Just add the CC and BCC recipients to the toaddrs when you send your email. For CC, add them to the CC header.

toaddr = 'buffy@sunnydale.k12.ca.us'
cc = ['alexander@sunydale.k12.ca.us','willow@sunnydale.k12.ca.us']
bcc = ['chairman@slayerscouncil.uk']
fromaddr = 'giles@sunnydale.k12.ca.us'
message_subject = "disturbance in sector 7"
message_text = "Three are dead in an attack in the sewers below sector 7."
message = "From: %s\r\n" % fromaddr
        + "To: %s\r\n" % toaddr
        + "CC: %s\r\n" % ",".join(cc)
        + "Subject: %s\r\n" % message_subject
        + "\r\n" 
        + message_text
toaddrs = [toaddr] + cc + bcc
server = smtplib.SMTP('smtp.sunnydale.k12.ca.us')
server.set_debuglevel(1)
server.sendmail(fromaddr, toaddrs, message)
server.quit()

Question 14

Key thing is to add the recipients as a list of email ids in your sendmail call.

import smtplib
from email.mime.multipart import MIMEMultipart

me = "user63503@gmail.com"
to = "someone@gmail.com"
cc = "anotherperson@gmail.com,someone@yahoo.com"
bcc = "bccperson1@gmail.com,bccperson2@yahoo.com"

rcpt = cc.split(",") + bcc.split(",") + [to]
msg = MIMEMultipart('alternative')
msg['Subject'] = "my subject"
msg['To'] = to
msg['Cc'] = cc
msg.attach(my_msg_body)
server = smtplib.SMTP("localhost") # or your smtp server
server.sendmail(me, rcpt, msg.as_string())
server.quit()

Question 15

Don’t add the bcc header.

See this: http://mail.python.org/pipermail/email-sig/2004-September/000151.html

And this: “””Notice that the second argument to sendmail(), the recipients, is passed as a list. You can include any number of addresses in the list to have the message delivered to each of them in turn. Since the envelope information is separate from the message headers, you can even BCC someone by including them in the method argument but not in the message header.””” from http://pymotw.com/2/smtplib

toaddr = 'buffy@sunnydale.k12.ca.us'
cc = ['alexander@sunydale.k12.ca.us','willow@sunnydale.k12.ca.us']
bcc = ['chairman@slayerscouncil.uk']
fromaddr = 'giles@sunnydale.k12.ca.us'
message_subject = "disturbance in sector 7"
message_text = "Three are dead in an attack in the sewers below sector 7."
message = "From: %s\r\n" % fromaddr
    + "To: %s\r\n" % toaddr
    + "CC: %s\r\n" % ",".join(cc)
    # don't add this, otherwise "to and cc" receivers will know who are the bcc receivers
    # + "BCC: %s\r\n" % ",".join(bcc)
    + "Subject: %s\r\n" % message_subject
    + "\r\n" 
    + message_text
toaddrs = [toaddr] + cc + bcc
server = smtplib.SMTP('smtp.sunnydale.k12.ca.us')
server.set_debuglevel(1)
server.sendmail(fromaddr, toaddrs, message)
server.quit()

Question 16

As of Python 3.2, released Nov 2011, the smtplib has a new function send_message instead of just sendmail, which makes dealing with To/CC/BCC easier. Pulling from the Python official email examples, with some slight modifications, we get:

# Import smtplib for the actual sending function
import smtplib

# Import the email modules we'll need
from email.message import EmailMessage

# Open the plain text file whose name is in textfile for reading.
with open(textfile) as fp:
    # Create a text/plain message
    msg = EmailMessage()
    msg.set_content(fp.read())

# me == the sender's email address
# you == the recipient's email address
# them == the cc's email address
# they == the bcc's email address
msg['Subject'] = 'The contents of %s' % textfile
msg['From'] = me
msg['To'] = you
msg['Cc'] = them
msg['Bcc'] = they


# Send the message via our own SMTP server.
s = smtplib.SMTP('localhost')
s.send_message(msg)
s.quit()

Using the headers work fine, because send_message respects BCC as outlined in the documentation:

send_message does not transmit any Bcc or Resent-Bcc headers that may appear in msg

With sendmail it was common to add the CC headers to the message, doing something such as:

msg['Bcc'] = blind.email@adrress.com

Or

msg = "From: from.email@address.com" +
      "To: to.email@adress.com" +
      "BCC: hidden.email@address.com" +
      "Subject: You've got mail!" +
      "This is the message body"

The problem is, the sendmail function treats all those headers the same, meaning they’ll get sent (visibly) to all To: and BCC: users, defeating the purposes of BCC. The solution, as shown in many of the other answers here, was to not include BCC in the headers, and instead only in the list of emails passed to sendmail.

The caveat is that send_message requires a Message object, meaning you’ll need to import a class from email.message instead of merely passing strings into sendmail.

Question 17

The distinction between TO, CC and BCC occurs only in the text headers. At the SMTP level, everybody is a recipient.

TO – There is a TO: header with this recipient’s address

CC – There is a CC: header with this recipient’s address

BCC – This recipient isn’t mentioned in the headers at all, but is still a recipient.

If you have

TO: abc@company.com
CC: xyz@company.com
BCC: boss@company.com

You have three recipients. The headers in the email body will include only the TO: and CC:

Question 18

You can try MIMEText

msg = MIMEText('text')
msg['to'] = 
msg['cc'] =

then send msg.as_string()

https://docs.python.org/3.6/library/email.examples.html

Question 19

It did not worked for me until i created:

#created cc string
cc = ""someone@domain.com;
#added cc to header
msg['Cc'] = cc

and than added cc in recipient [list] like:

s.sendmail(me, [you,cc], msg.as_string())

Question 20

None of the above things worked for me as I had multiple recipients both in ‘to’ and ‘cc’. So I tried like below:

recipients = ['abc@gmail.com', 'xyz@gmail.com']
cc_recipients = ['lmn@gmail.com', 'pqr@gmail.com']
MESSAGE['To'] = ", ".join(recipients)
MESSAGE['Cc'] = ", ".join(cc_recipients)

and extend the ‘recipients’ with ‘cc_recipients’ and send mail in trivial way

recipients.extend(cc_recipients)
server.sendmail(FROM,recipients,MESSAGE.as_string())

Question 21

I’ve a python script which works just as it should, but I need to write the execution time. I’ve googled that I should use timeit but I can’t seem to get it to work.

My Python script looks like this:

import sys
import getopt
import timeit
import random
import os
import re
import ibm_db
import time
from string import maketrans
myfile = open("results_update.txt", "a")

for r in range(100):
    rannumber = random.randint(0, 100)

    update = "update TABLE set val = %i where MyCount >= '2010' and MyCount < '2012' and number = '250'" % rannumber
    #print rannumber

    conn = ibm_db.pconnect("dsn=myDB","usrname","secretPWD")

for r in range(5):
    print "Run %s\n" % r        
    ibm_db.execute(query_stmt)
 query_stmt = ibm_db.prepare(conn, update)

myfile.close()
ibm_db.close(conn)

What I need is the time it takes to execute the query and write it to the file results_update.txt. The purpose is to test an update statement for my database with different indexes and tuning mechanisms.

Question 22

You can use time.time() or time.clock() before and after the block you want to time.

import time

t0 = time.time()
code_block
t1 = time.time()

total = t1-t0

This method is not as exact as timeit (it does not average several runs) but it is straightforward.

time.time() (in Windows and Linux) and time.clock() (in Linux) are not precise enough for fast functions (you get total = 0). In this case or if you want to average the time elapsed by several runs, you have to manually call the function multiple times (As I think you already do in you example code and timeit does automatically when you set its number argument)

import time

def myfast():
   code

n = 10000
t0 = time.time()
for i in range(n): myfast()
t1 = time.time()

total_n = t1-t0

In Windows, as Corey stated in the comment, time.clock() has much higher precision (microsecond instead of second) and is preferred over time.time().

Question 23

If you are profiling your code and can use IPython, it has the magic function %timeit.

%%timeit operates on cells.

In [2]: %timeit cos(3.14)
10000000 loops, best of 3: 160 ns per loop

In [3]: %%timeit
   ...: cos(3.14)
   ...: x = 2 + 3
   ...: 
10000000 loops, best of 3: 196 ns per loop

Question 24

Quite apart from the timing, this code you show is simply incorrect: you execute 100 connections (completely ignoring all but the last one), and then when you do the first execute call you pass it a local variable query_stmt which you only initialize after the execute call.

First, make your code correct, without worrying about timing yet: i.e. a function that makes or receives a connection and performs 100 or 500 or whatever number of updates on that connection, then closes the connection. Once you have your code working correctly is the correct point at which to think about using timeit on it!

Specifically, if the function you want to time is a parameter-less one called foobar you can use timeit.timeit (2.6 or later — it’s more complicated in 2.5 and before):

timeit.timeit('foobar()', number=1000)

You’d better specify the number of runs because the default, a million, may be high for your use case (leading to spending a lot of time in this code;-).

Question 25

Focus on one specific thing. Disk I/O is slow, so I’d take that out of the test if all you are going to tweak is the database query.

And if you need to time your database execution, look for database tools instead, like asking for the query plan, and note that performance varies not only with the exact query and what indexes you have, but also with the data load (how much data you have stored).

That said, you can simply put your code in a function and run that function with timeit.timeit():

def function_to_repeat():
    # ...

duration = timeit.timeit(function_to_repeat, number=1000)

This would disable the garbage collection, repeatedly call the function_to_repeat() function, and time the total duration of those calls using timeit.default_timer(), which is the most accurate available clock for your specific platform.

You should move setup code out of the repeated function; for example, you should connect to the database first, then time only the queries. Use the setup argument to either import or create those dependencies, and pass them into your function:

def function_to_repeat(var1, var2):
    # ...

duration = timeit.timeit(
    'function_to_repeat(var1, var2)',
    'from __main__ import function_to_repeat, var1, var2', 
    number=1000)

would grab the globals function_to_repeat, var1 and var2 from your script and pass those to the function each repetition.

Question 26

I see the question has already been answered, but still want to add my 2 cents for the same.

I have also faced similar scenario in which I have to test the execution times for several approaches and hence written a small script, which calls timeit on all functions written in it.

The script is also available as github gist here.

Hope it will help you and others.

from random import random
import types

def list_without_comprehension():
    l = []
    for i in xrange(1000):
        l.append(int(random()*100 % 100))
    return l

def list_with_comprehension():
    # 1K random numbers between 0 to 100
    l = [int(random()*100 % 100) for _ in xrange(1000)]
    return l


# operations on list_without_comprehension
def sort_list_without_comprehension():
    list_without_comprehension().sort()

def reverse_sort_list_without_comprehension():
    list_without_comprehension().sort(reverse=True)

def sorted_list_without_comprehension():
    sorted(list_without_comprehension())


# operations on list_with_comprehension
def sort_list_with_comprehension():
    list_with_comprehension().sort()

def reverse_sort_list_with_comprehension():
    list_with_comprehension().sort(reverse=True)

def sorted_list_with_comprehension():
    sorted(list_with_comprehension())


def main():
    objs = globals()
    funcs = []
    f = open("timeit_demo.sh", "w+")

    for objname in objs:
        if objname != 'main' and type(objs[objname]) == types.FunctionType:
            funcs.append(objname)
    funcs.sort()
    for func in funcs:
        f.write('''echo "Timing: %(funcname)s"
python -m timeit "import timeit_demo; timeit_demo.%(funcname)s();"\n\n
echo "------------------------------------------------------------"
''' % dict(
                funcname = func,
                )
            )

    f.close()

if __name__ == "__main__":
    main()

    from os import system

    #Works only for *nix platforms
    system("/bin/bash timeit_demo.sh")

    #un-comment below for windows
    #system("cmd timeit_demo.sh")

Question 27

Here’s a simple wrapper for steven’s answer. This function doesn’t do repeated runs/averaging, just saves you from having to repeat the timing code everywhere :)

'''function which prints the wall time it takes to execute the given command'''
def time_func(func, *args): #*args can take 0 or more 
  import time
  start_time = time.time()
  func(*args)
  end_time = time.time()
  print("it took this long to run: {}".format(end_time-start_time))

Question 28

The testing suite doesn’t make an attempt at using the imported timeit so it’s hard to tell what the intent was. Nonetheless, this is a canonical answer so a complete example of timeit seems in order, elaborating on Martijn’s answer.

The docs for timeit offer many examples and flags worth checking out. The basic usage on the command line is:

$ python -mtimeit "all(True for _ in range(1000))"
2000 loops, best of 5: 161 usec per loop
$ python -mtimeit "all([True for _ in range(1000)])"
2000 loops, best of 5: 116 usec per loop

Run with -h to see all options. Python MOTW has a great section on timeit that shows how to run modules via import and multiline code strings from the command line.

In script form, I typically use it like this:

import argparse
import copy
import dis
import inspect
import random
import sys
import timeit

def test_slice(L):
    L[:]

def test_copy(L):
    L.copy()

def test_deepcopy(L):
    copy.deepcopy(L)

if __name__ == "__main__":
    parser = argparse.ArgumentParser()
    parser.add_argument("--n", type=int, default=10 ** 5)
    parser.add_argument("--trials", type=int, default=100)
    parser.add_argument("--dis", action="store_true")
    args = parser.parse_args()
    n = args.n
    trials = args.trials
    namespace = dict(L = random.sample(range(n), k=n))
    funcs_to_test = [x for x in locals().values() 
                     if callable(x) and x.__module__ == __name__]
    print(f"{'-' * 30}\nn = {n}, {trials} trials\n{'-' * 30}\n")

    for func in funcs_to_test:
        fname = func.__name__
        fargs = ", ".join(inspect.signature(func).parameters)
        stmt = f"{fname}({fargs})"
        setup = f"from __main__ import {fname}"
        time = timeit.timeit(stmt, setup, number=trials, globals=namespace)
        print(inspect.getsource(globals().get(fname)))

        if args.dis:
            dis.dis(globals().get(fname))

        print(f"time (s) => {time}\n{'-' * 30}\n")

You can pretty easily drop in the functions and arguments you need. Use caution when using impure functions and take care of state.

Sample output:

$ python benchmark.py --n 10000
------------------------------
n = 10000, 100 trials
------------------------------

def test_slice(L):
    L[:]

time (s) => 0.015502399999999972
------------------------------

def test_copy(L):
    L.copy()

time (s) => 0.01651419999999998
------------------------------

def test_deepcopy(L):
    copy.deepcopy(L)

time (s) => 2.136012
------------------------------

Question 29

Can anyone tell me why this isn’t working?

>>> import mock
>>> @mock.patch('datetime.date.today')
... def today(cls):
...  return date(2010, 1, 1)
...
>>> from datetime import date
>>> date.today()
datetime.date(2010, 12, 19)

Perhaps someone could suggest a better way?

Question 30

There are a few problems.

First of all, the way you’re using mock.patch isn’t quite right. When used as a decorator, it replaces the given function/class (in this case, datetime.date.today) with a Mock object only within the decorated function. So, only within your today() will datetime.date.today be a different function, which doesn’t appear to be what you want.

What you really want seems to be more like this:

@mock.patch('datetime.date.today')
def test():
    datetime.date.today.return_value = date(2010, 1, 1)
    print datetime.date.today()

Unfortunately, this won’t work:

>>> test()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "build/bdist.macosx-10.6-universal/egg/mock.py", line 557, in patched
  File "build/bdist.macosx-10.6-universal/egg/mock.py", line 620, in __enter__
TypeError: can't set attributes of built-in/extension type 'datetime.date'

This fails because Python built-in types are immutable – see this answer for more details.

In this case, I would subclass datetime.date myself and create the right function:

import datetime
class NewDate(datetime.date):
    @classmethod
    def today(cls):
        return cls(2010, 1, 1)
datetime.date = NewDate

And now you could do:

>>> datetime.date.today()
NewDate(2010, 1, 1)

Question 31

Another option is to use https://github.com/spulec/freezegun/

Install it:

pip install freezegun

And use it:

from freezegun import freeze_time

@freeze_time("2012-01-01")
def test_something():

    from datetime import datetime
    print(datetime.now()) #  2012-01-01 00:00:00

    from datetime import date
    print(date.today()) #  2012-01-01

It also affects other datetime calls in method calls from other modules:

other_module.py:

from datetime import datetime

def other_method():
    print(datetime.now())

main.py:

from freezegun import freeze_time

@freeze_time("2012-01-01")
def test_something():

    import other_module
    other_module.other_method()

And finally:

$ python main.py
# 2012-01-01

Question 32

For what it’s worth, the Mock docs talk about datetime.date.today specifically, and it’s possible to do this without having to create a dummy class:

https://docs.python.org/3/library/unittest.mock-examples.html#partial-mocking

>>> from datetime import date
>>> with patch('mymodule.date') as mock_date:
...     mock_date.today.return_value = date(2010, 10, 8)
...     mock_date.side_effect = lambda *args, **kw: date(*args, **kw)
...
...     assert mymodule.date.today() == date(2010, 10, 8)
...     assert mymodule.date(2009, 6, 8) == date(2009, 6, 8)
...

Question 33

I guess I came a little late for this but I think the main problem here is that you’re patching datetime.date.today directly and, according to the documentation, this is wrong.

You should patch the reference imported in the file where the tested function is, for example.

Let’s say you have a functions.py file where you have the following:

import datetime

def get_today():
    return datetime.date.today()

then, in your test, you should have something like this

import datetime
import unittest

from functions import get_today
from mock import patch, Mock

class GetTodayTest(unittest.TestCase):

    @patch('functions.datetime')
    def test_get_today(self, datetime_mock):
        datetime_mock.date.today = Mock(return_value=datetime.strptime('Jun 1 2005', '%b %d %Y'))
        value = get_today()
        # then assert your thing...

Hope this helps a little bit.

Question 34

To add to Daniel G’s solution:

from datetime import date

class FakeDate(date):
    "A manipulable date replacement"
    def __new__(cls, *args, **kwargs):
        return date.__new__(date, *args, **kwargs)

This creates a class which, when instantiated, will return a normal datetime.date object, but which is also able to be changed.

@mock.patch('datetime.date', FakeDate)
def test():
    from datetime import date
    FakeDate.today = classmethod(lambda cls: date(2010, 1, 1))
    return date.today()

test() # datetime.date(2010, 1, 1)

Question 35

I faced the same situation a couple of days ago, and my solution was to define a function in the module to test and just mock that:

def get_date_now():
    return datetime.datetime.now()

Today I found out about FreezeGun, and it seems to cover this case beautifully

from freezegun import freeze_time
import datetime
import unittest


@freeze_time("2012-01-14")
def test():
    assert datetime.datetime.now() == datetime.datetime(2012, 1, 14)

Question 36

The easiest way for me is doing this:

import datetime
from unittest.mock import Mock, patch

def test():
    datetime_mock = Mock(wraps=datetime.datetime)
    datetime_mock.now.return_value = datetime.datetime(1999, 1, 1)
    with patch('datetime.datetime', new=datetime_mock):
        assert datetime.datetime.now() == datetime.datetime(1999, 1, 1)

CAUTION for this solution: all functionality from datetime module from the target_module will stop working.

Question 37

You can use the following approach, based on Daniel G solution. This one has advantage of not breaking type checking with isinstance(d, datetime.date).

import mock

def fixed_today(today):
    from datetime import date

    class FakeDateType(type):
        def __instancecheck__(self, instance):
            return isinstance(instance, date)

    class FakeDate(date):
        __metaclass__ = FakeDateType

        def __new__(cls, *args, **kwargs):
            return date.__new__(date, *args, **kwargs)

        @staticmethod
        def today():
            return today

    return mock.patch("datetime.date", FakeDate)

Basically, we replace C-based datetime.date class with our own python subclass, that produces original datetime.date instances and responds to isinstance() queries exactly as native datetime.date.

Use it as context manager in your tests:

with fixed_today(datetime.date(2013, 11, 22)):
    # run the code under test
    # note, that these type checks will not break when patch is active:
    assert isinstance(datetime.date.today(), datetime.date)

Similar approach can be used to mock datetime.datetime.now() function.

Question 38

Generally speaking, you would have datetime or perhaps datetime.date imported into a module somewhere. A more effective way of mocking the method would be to patch it on the module that is importing it. Example:

a.py

from datetime import date

def my_method():
    return date.today()

Then for your test, the mock object itself would be passed as an argument to the test method. You would set up the mock with the result value you want, and then call your method under test. Then you would assert that your method did what you want.

>>> import mock
>>> import a
>>> @mock.patch('a.date')
... def test_my_method(date_mock):
...     date_mock.today.return_value = mock.sentinel.today
...     result = a.my_method()
...     print result
...     date_mock.today.assert_called_once_with()
...     assert mock.sentinel.today == result
...
>>> test_my_method()
sentinel.today

A word of warning. It is most certainly possible to go overboard with mocking. When you do, it makes your tests longer, harder to understand, and impossible to maintain. Before you mock a method as simple as datetime.date.today, ask yourself if you really need to mock it. If your test is short and to the point and works fine without mocking the function, you may just be looking at an internal detail of the code you’re testing rather than an object you need to mock.

Question 39

Here’s another way to mock datetime.date.today() with an added bonus that the rest of datetime functions continue to work, as the mock object is configured to wrap the original datetime module:

from unittest import mock, TestCase

import foo_module

class FooTest(TestCase):

    @mock.patch(f'{foo_module.__name__}.datetime', wraps=datetime)
    def test_something(self, mock_datetime):
        # mock only datetime.date.today()
        mock_datetime.date.today.return_value = datetime.date(2019, 3, 15)
        # other calls to datetime functions will be forwarded to original datetime

Note the wraps=datetime argument to mock.patch() – when the foo_module uses other datetime functions besides date.today() they will be forwarded to the original wrapped datetime module.

Question 40

Several solutions are discussed in http://blog.xelnor.net/python-mocking-datetime/. In summary:

Mock object – Simple and efficient but breaks isinstance() checks:

target = datetime.datetime(2009, 1, 1)
with mock.patch.object(datetime, 'datetime', mock.Mock(wraps=datetime.datetime)) as patched:
    patched.now.return_value = target
    print(datetime.datetime.now())

Mock class

import datetime
import mock

real_datetime_class = datetime.datetime

def mock_datetime_now(target, dt):
    class DatetimeSubclassMeta(type):
        @classmethod
        def __instancecheck__(mcs, obj):
            return isinstance(obj, real_datetime_class)

    class BaseMockedDatetime(real_datetime_class):
        @classmethod
        def now(cls, tz=None):
            return target.replace(tzinfo=tz)

        @classmethod
        def utcnow(cls):
            return target

    # Python2 & Python3 compatible metaclass
    MockedDatetime = DatetimeSubclassMeta('datetime', (BaseMockedDatetime,), {})

    return mock.patch.object(dt, 'datetime', MockedDatetime)

Use as:

with mock_datetime_now(target, datetime):
   ....

Question 41

Maybe you could use your own “today()” method that you will patch where needed. Example with mocking utcnow() can be found here: https://bitbucket.org/k_bx/blog/src/tip/source/en_posts/2012-07-13-double-call-hack.rst?at=default

Question 42

I implemented @user3016183 method using a custom decorator:

def changeNow(func, newNow = datetime(2015, 11, 23, 12, 00, 00)):
    """decorator used to change datetime.datetime.now() in the tested function."""
    def retfunc(self):                             
        with mock.patch('mymodule.datetime') as mock_date:                         
            mock_date.now.return_value = newNow
            mock_date.side_effect = lambda *args, **kw: datetime(*args, **kw)
            func(self)
    return retfunc

I thought that might help someone one day…

Question 43

It’s possible to mock functions from datetime module without adding side_effects

import mock
from datetime import datetime
from where_datetime_used import do

initial_date = datetime.strptime('2018-09-27', "%Y-%m-%d")
with mock.patch('where_datetime_used.datetime') as mocked_dt:
    mocked_dt.now.return_value = initial_date
    do()

Question 44

For those of you using pytest with mocker here is how I mocked datetime.datetime.now() which is very similar to the original question.

test_get_now(mocker):
    datetime_mock = mocker.patch("blackline_accounts_import.datetime",)
    datetime_mock.datetime.now.return_value=datetime.datetime(2019,3,11,6,2,0,0)

    now == function_being_tested()  # run function

    assert now == datetime.datetime(2019,3,11,6,2,0,0)

Essentially the mock has to be set to return the specified date. You aren’t able to patch over datetime’s object directly.

Question 45

I made this work by importing datetime as realdatetime and replacing the methods I needed in the mock with the real methods:

import datetime as realdatetime

@mock.patch('datetime')
def test_method(self, mock_datetime):
    mock_datetime.today = realdatetime.today
    mock_datetime.now.return_value = realdatetime.datetime(2019, 8, 23, 14, 34, 8, 0)

Question 46

You can mock datetime using this:

In the module sources.py:

import datetime


class ShowTime:
    def current_date():
        return datetime.date.today().strftime('%Y-%m-%d')

In your tests.py:

from unittest import TestCase, mock
import datetime


class TestShowTime(TestCase):
    def setUp(self) -> None:
        self.st = sources.ShowTime()
        super().setUp()

    @mock.patch('sources.datetime.date')
    def test_current_date(self, date_mock):
        date_mock.today.return_value = datetime.datetime(year=2019, month=10, day=1)
        current_date = self.st.current_date()
        self.assertEqual(current_date, '2019-10-01')

Question 47

CPython actually implements the datetime module using both a pure-Python Lib/datetime.py and a C-optimized Modules/_datetimemodule.c. The C-optimized version cannot be patched but the pure-Python version can.

At the bottom of the pure-Python implementation in Lib/datetime.py is this code:

try:
    from _datetime import *  # <-- Import from C-optimized module.
except ImportError:
    pass

This code imports all the C-optimized definitions and effectively replaces all the pure-Python definitions. We can force CPython to use the pure-Python implementation of the datetime module by doing:

import datetime
import importlib
import sys

sys.modules["_datetime"] = None
importlib.reload(datetime)

By setting sys.modules["_datetime"] = None, we tell Python to ignore the C-optimized module. Then we reload the module which causes the import from _datetime to fail. Now the pure-Python definitions remain and can be patched normally.

If you’re using Pytest then include the snippet above in conftest.py and you can patch datetime objects normally.

Question 48

I currently have a few unit tests which share a common set of tests. Here’s an example:

import unittest

class BaseTest(unittest.TestCase):

    def testCommon(self):
        print 'Calling BaseTest:testCommon'
        value = 5
        self.assertEquals(value, 5)

class SubTest1(BaseTest):

    def testSub1(self):
        print 'Calling SubTest1:testSub1'
        sub = 3
        self.assertEquals(sub, 3)


class SubTest2(BaseTest):

    def testSub2(self):
        print 'Calling SubTest2:testSub2'
        sub = 4
        self.assertEquals(sub, 4)

if __name__ == '__main__':
    unittest.main()

The output of the above is:

Calling BaseTest:testCommon
.Calling BaseTest:testCommon
.Calling SubTest1:testSub1
.Calling BaseTest:testCommon
.Calling SubTest2:testSub2
.
----------------------------------------------------------------------
Ran 5 tests in 0.000s

OK

Is there a way to rewrite the above so that the very first testCommon is not called?

EDIT: Instead of running 5 tests above, I want it to run only 4 tests, 2 from the SubTest1 and another 2 from SubTest2. It seems that Python unittest is running the original BaseTest on its own and I need a mechanism to prevent that from happening.

Question 49

Use multiple inheritance, so your class with common tests doesn’t itself inherit from TestCase.

import unittest

class CommonTests(object):
    def testCommon(self):
        print 'Calling BaseTest:testCommon'
        value = 5
        self.assertEquals(value, 5)

class SubTest1(unittest.TestCase, CommonTests):

    def testSub1(self):
        print 'Calling SubTest1:testSub1'
        sub = 3
        self.assertEquals(sub, 3)


class SubTest2(unittest.TestCase, CommonTests):

    def testSub2(self):
        print 'Calling SubTest2:testSub2'
        sub = 4
        self.assertEquals(sub, 4)

if __name__ == '__main__':
    unittest.main()

Question 50

Do not use multiple inheritance, it will bite you later.

Instead you can just move your base class into the separate module or wrap it with the blank class:

class BaseTestCases:

    class BaseTest(unittest.TestCase):

        def testCommon(self):
            print('Calling BaseTest:testCommon')
            value = 5
            self.assertEqual(value, 5)


class SubTest1(BaseTestCases.BaseTest):

    def testSub1(self):
        print('Calling SubTest1:testSub1')
        sub = 3
        self.assertEqual(sub, 3)


class SubTest2(BaseTestCases.BaseTest):

    def testSub2(self):
        print('Calling SubTest2:testSub2')
        sub = 4
        self.assertEqual(sub, 4)

if __name__ == '__main__':
    unittest.main()

The output:

Calling BaseTest:testCommon
.Calling SubTest1:testSub1
.Calling BaseTest:testCommon
.Calling SubTest2:testSub2
.
----------------------------------------------------------------------
Ran 4 tests in 0.001s

OK

Question 51

You can solve this problem with a single command:

del(BaseTest)

So the code would look like this:

import unittest

class BaseTest(unittest.TestCase):

    def testCommon(self):
        print 'Calling BaseTest:testCommon'
        value = 5
        self.assertEquals(value, 5)

class SubTest1(BaseTest):

    def testSub1(self):
        print 'Calling SubTest1:testSub1'
        sub = 3
        self.assertEquals(sub, 3)


class SubTest2(BaseTest):

    def testSub2(self):
        print 'Calling SubTest2:testSub2'
        sub = 4
        self.assertEquals(sub, 4)

del(BaseTest)

if __name__ == '__main__':
    unittest.main()

Question 52

Matthew Marshall’s answer is great, but it requires that you inherit from two classes in each of your test cases, which is error-prone. Instead, I use this (python>=2.7):

class BaseTest(unittest.TestCase):

    @classmethod
    def setUpClass(cls):
        if cls is BaseTest:
            raise unittest.SkipTest("Skip BaseTest tests, it's a base class")
        super(BaseTest, cls).setUpClass()

Question 53

What are you trying to achieve? If you have common test code (assertions, template tests, etc), then place them in methods which aren’t prefixed with test so unittest won’t load them.

import unittest

class CommonTests(unittest.TestCase):
      def common_assertion(self, foo, bar, baz):
          # whatever common code
          self.assertEqual(foo(bar), baz)

class BaseTest(CommonTests):

    def testCommon(self):
        print 'Calling BaseTest:testCommon'
        value = 5
        self.assertEquals(value, 5)

class SubTest1(CommonTests):

    def testSub1(self):
        print 'Calling SubTest1:testSub1'
        sub = 3
        self.assertEquals(sub, 3)

class SubTest2(CommonTests):

    def testSub2(self):
        print 'Calling SubTest2:testSub2'
        sub = 4
        self.assertEquals(sub, 4)

if __name__ == '__main__':
    unittest.main()

Question 54

Matthew’s answer is the one I needed to use since I’m on 2.5 still. But as of 2.7 you can use the @unittest.skip() decorator on any test methods you want to skip.

http://docs.python.org/library/unittest.html#skipping-tests-and-expected-failures

You’ll need to implement your own skipping decorator to check for the base type. Haven’t used this feature before, but off the top of my head you could use BaseTest as a marker type to condition the skip:

def skipBaseTest(obj):
    if type(obj) is BaseTest:
        return unittest.skip("BaseTest tests skipped")
    return lambda func: func

Question 55

A way I’ve thought of solving this is by hiding the test methods if the base class is used. This way the tests aren’t skipped, so the test results can be green instead of yellow in many test reporting tools.

Compared to the mixin method, ide’s like PyCharm won’t complain that unit test methods are missing from the base class.

If a base class inherits from this class, it will need to override the setUpClass and tearDownClass methods.

class BaseTest(unittest.TestCase):
    @classmethod
    def setUpClass(cls):
        cls._test_methods = []
        if cls is BaseTest:
            for name in dir(cls):
                if name.startswith('test') and callable(getattr(cls, name)):
                    cls._test_methods.append((name, getattr(cls, name)))
                    setattr(cls, name, lambda self: None)

    @classmethod
    def tearDownClass(cls):
        if cls is BaseTest:
            for name, method in cls._test_methods:
                setattr(cls, name, method)
            cls._test_methods = []

Question 56

You can add __test__ = False in BaseTest class, but if you add it, be aware that you must add __test__ = True in derived classes to be able to run tests.

import unittest

class BaseTest(unittest.TestCase):
    __test__ = False

    def testCommon(self):
        print 'Calling BaseTest:testCommon'
        value = 5
        self.assertEquals(value, 5)

class SubTest1(BaseTest):
    __test__ = True

    def testSub1(self):
        print 'Calling SubTest1:testSub1'
        sub = 3
        self.assertEquals(sub, 3)


class SubTest2(BaseTest):
    __test__ = True

    def testSub2(self):
        print 'Calling SubTest2:testSub2'
        sub = 4
        self.assertEquals(sub, 4)

if __name__ == '__main__':
    unittest.main()

Question 57

Another option is not to execute

unittest.main()

Instead of that you can use

suite = unittest.TestLoader().loadTestsFromTestCase(TestClass)
unittest.TextTestRunner(verbosity=2).run(suite)

So you only execute the tests in the class TestClass

Question 58

I made about the same than @Vladim P. (https://stackoverflow.com/a/25695512/2451329) but slightly modified:

import unittest2


from some_module import func1, func2


def make_base_class(func):

    class Base(unittest2.TestCase):

        def test_common1(self):
            print("in test_common1")
            self.assertTrue(func())

        def test_common2(self):
            print("in test_common1")
            self.assertFalse(func(42))

    return Base



class A(make_base_class(func1)):
    pass


class B(make_base_class(func2)):

    def test_func2_with_no_arg_return_bar(self):
        self.assertEqual("bar", func2())

and there we go.

Question 59

As of Python 3.2, you can add a test_loader function to a module to control which tests (if any) are found by the test discovery mechanism.

For example, the following will only load the original poster’s SubTest1 and SubTest2 Test Cases, ignoring Base:

def load_tests(loader, standard_tests, pattern):
    suite = TestSuite()
    suite.addTests([SubTest1, SubTest2])
    return suite

It ought to be possible to iterate over standard_tests (a TestSuite containing the tests the default loader found) and copy all but Base to suite instead, but the nested nature of TestSuite.__iter__ makes that a lot more complicated.

Question 60

Just rename the testCommon method to something else. Unittest (usually) skips anything that doesn’t have ‘test’ in it.

Quick and simple

  import unittest

  class BaseTest(unittest.TestCase):

   def methodCommon(self):
       print 'Calling BaseTest:testCommon'
       value = 5
       self.assertEquals(value, 5)

  class SubTest1(BaseTest):

      def testSub1(self):
          print 'Calling SubTest1:testSub1'
          sub = 3
          self.assertEquals(sub, 3)


  class SubTest2(BaseTest):

      def testSub2(self):
          print 'Calling SubTest2:testSub2'
          sub = 4
          self.assertEquals(sub, 4)

  if __name__ == '__main__':
      unittest.main()`

Question 61

So this is kind of an old thread but I came across this problem today and thought of my own hack for it. It uses a decorator that makes the values of the functions None when acessed through the base class. Don’t need to worry about setup and setupclass because if the baseclass has no tests they won’t run.

import types
import unittest


class FunctionValueOverride(object):
    def __init__(self, cls, default, override=None):
        self.cls = cls
        self.default = default
        self.override = override

    def __get__(self, obj, klass):
        if klass == self.cls:
            return self.override
        else:
            if obj:
                return types.MethodType(self.default, obj)
            else:
                return self.default


def fixture(cls):
    for t in vars(cls):
        if not callable(getattr(cls, t)) or t[:4] != "test":
            continue
        setattr(cls, t, FunctionValueOverride(cls, getattr(cls, t)))
    return cls


@fixture
class BaseTest(unittest.TestCase):
    def testCommon(self):
        print('Calling BaseTest:testCommon')
        value = 5
        self.assertEqual(value, 5)


class SubTest1(BaseTest):
    def testSub1(self):
        print('Calling SubTest1:testSub1')
        sub = 3
        self.assertEqual(sub, 3)


class SubTest2(BaseTest):

    def testSub2(self):
        print('Calling SubTest2:testSub2')
        sub = 4
        self.assertEqual(sub, 4)

if __name__ == '__main__':
    unittest.main()

Question 62

Change the BaseTest method name to setUp:

class BaseTest(unittest.TestCase):
    def setUp(self):
        print 'Calling BaseTest:testCommon'
        value = 5
        self.assertEquals(value, 5)


class SubTest1(BaseTest):
    def testSub1(self):
        print 'Calling SubTest1:testSub1'
        sub = 3
        self.assertEquals(sub, 3)


class SubTest2(BaseTest):
    def testSub2(self):
        print 'Calling SubTest2:testSub2'
        sub = 4
        self.assertEquals(sub, 4)

Output:

Ran 2 tests in 0.000s

Calling BaseTest:testCommon Calling
SubTest1:testSub1 Calling
BaseTest:testCommon Calling
SubTest2:testSub2

From the documentation:

TestCase.setUp()
Method called to prepare the test fixture. This is called immediately before calling the test method; any exception raised by this method will be considered an error rather than a test failure. The default implementation does nothing.

Question 63

I recently discovered pytest. It seems great. However, I feel the documentation could be better.

I’m trying to understand what conftest.py files are meant to be used for.

In my (currently small) test suite I have one conftest.py file at the project root. I use it to define the fixtures that I inject into my tests.

I have two questions:

Is this the correct use of conftest.py? Does it have other uses?
Can I have more than one conftest.py file? When would I want to do that? Examples will be appreciated.

More generally, how would you define the purpose and correct use of conftest.py file(s) in a py.test test suite?

Question 64

Is this the correct use of conftest.py?

Yes it is. Fixtures are a potential and common use of conftest.py. The fixtures that you will define will be shared among all tests in your test suite. However, defining fixtures in the root conftest.py might be useless and it would slow down testing if such fixtures are not used by all tests.

Does it have other uses?

Yes it does.

Fixtures: Define fixtures for static data used by tests. This data can be accessed by all tests in the suite unless specified otherwise. This could be data as well as helpers of modules which will be passed to all tests.
External plugin loading: conftest.py is used to import external plugins or modules. By defining the following global variable, pytest will load the module and make it available for its test. Plugins are generally files defined in your project or other modules which might be needed in your tests. You can also load a set of predefined plugins as explained here.

pytest_plugins = "someapp.someplugin"
Hooks: You can specify hooks such as setup and teardown methods and much more to improve your tests. For a set of available hooks, read here. Example:
```
def pytest_runtest_setup(item):
     """ called before ``pytest_runtest_call(item). """
     #do some stuff`
```
Test root path: This is a bit of a hidden feature. By defining conftest.py in your root path, you will have pytest recognizing your application modules without specifying PYTHONPATH. In the background, py.test modifies your sys.path by including all submodules which are found from the root path.

Can I have more than one conftest.py file?

Yes you can and it is strongly recommended if your test structure is somewhat complex. conftest.py files have directory scope. Therefore, creating targeted fixtures and helpers is good practice.

When would I want to do that? Examples will be appreciated.

Several cases could fit:

Creating a set of tools or hooks for a particular group of tests.

root/mod/conftest.py

def pytest_runtest_setup(item):
    print("I am mod")
    #do some stuff


test root/mod2/test.py will NOT produce "I am mod"

Loading a set of fixtures for some tests but not for others.

root/mod/conftest.py

@pytest.fixture()
def fixture():
    return "some stuff"

root/mod2/conftest.py

@pytest.fixture()
def fixture():
    return "some other stuff"

root/mod2/test.py

def test(fixture):
    print(fixture)

Will print “some other stuff”.

Overriding hooks inherited from the root conftest.py.

root/mod/conftest.py

def pytest_runtest_setup(item):
    print("I am mod")
    #do some stuff

root/conftest.py

def pytest_runtest_setup(item):
    print("I am root")
    #do some stuff

By running any test inside root/mod, only “I am mod” is printed.

You can read more about conftest.py here.

EDIT:

What if I need plain-old helper functions to be called from a number of tests in different modules – will they be available to me if I put them in a conftest.py? Or should I simply put them in a helpers.py module and import and use it in my test modules?

You can use conftest.py to define your helpers. However, you should follow common practice. Helpers can be used as fixtures at least in pytest. For example in my tests I have a mock redis helper which I inject into my tests this way.

root/helper/redis/redis.py

@pytest.fixture
def mock_redis():
    return MockRedis()

root/tests/stuff/conftest.py

pytest_plugin="helper.redis.redis"

root/tests/stuff/test.py

def test(mock_redis):
    print(mock_redis.get('stuff'))

This will be a test module that you can freely import in your tests. NOTE that you could potentially name redis.py as conftest.py if your module redis contains more tests. However, that practice is discouraged because of ambiguity.

If you want to use conftest.py, you can simply put that helper in your root conftest.py and inject it when needed.

root/tests/conftest.py

@pytest.fixture
def mock_redis():
    return MockRedis()

root/tests/stuff/test.py

def test(mock_redis):
    print(mock_redis.get(stuff))

Another thing you can do is to write an installable plugin. In that case your helper can be written anywhere but it needs to define an entry point to be installed in your and other potential test frameworks. See this.

If you don’t want to use fixtures, you could of course define a simple helper and just use the plain old import wherever it is needed.

root/tests/helper/redis.py

class MockRedis():
    # stuff

root/tests/stuff/test.py

from helper.redis import MockRedis

def test():
    print(MockRedis().get(stuff))

However, here you might have problems with the path since the module is not in a child folder of the test. You should be able to overcome this (not tested) by adding an __init__.py to your helper

root/tests/helper/__init__.py

from .redis import MockRedis

Or simply adding the helper module to your PYTHONPATH.

问题：用python练习BDD [关闭]

回答 0

回答 1

回答 2

回答 3

回答 4

回答 5

回答 6

回答 7

回答 8

测试代码：

控制台输出：

Test Code:

Console Output:

回答 9

问题：Python：如何使用TO，CC和BCC发送邮件？

回答 0

回答 1

回答 2

回答 3

回答 4

回答 5

回答 6

回答 7

httpbin(1)：HTTP请求和响应服务

正式部署在以下位置：

另请参阅

生成状态

问题：如何使用Python的timeit计时代码段以测试性能？

回答 0

回答 1

回答 2

回答 3

回答 4

回答 5

回答 6

问题：尝试模拟datetime.date.today（），但无法正常工作

回答 0

回答 1

回答 2

回答 3

回答 4

回答 5

回答 6

回答 7

回答 8

回答 9

回答 10

回答 11

回答 12

回答 13

回答 14

回答 15

回答 16

回答 17

问题：具有基类和子类的Python单元测试

回答 0

回答 1

回答 2

回答 3

回答 4

回答 5

回答 6

回答 7

回答 8

回答 9

回答 10

回答 11

回答 12

回答 13

问题：在pytest中，conftest.py文件有什么用？

回答 0

回答 1

回答 2

问题：如何在目录中运行所有Python单元测试？

回答 0

回答 1

回答 2

回答 3

回答 4