Python 实用宝典

Question 1

I am using Python 3.4 with IPython and have the following code. I’m unable to read a csv-file from the given URL:

import pandas as pd
import requests

url="https://github.com/cs109/2014_data/blob/master/countries.csv"
s=requests.get(url).content
c=pd.read_csv(s)

I have the following error

“Expected file path name or file-like object, got type”

How can I fix this?

Question 2

Update

From pandas 0.19.2 you can now just pass the url directly.

Just as the error suggests, pandas.read_csv needs a file-like object as the first argument.

If you want to read the csv from a string, you can use io.StringIO (Python 3.x) or StringIO.StringIO (Python 2.x) .

Also, for the URL – https://github.com/cs109/2014_data/blob/master/countries.csv – you are getting back html response , not raw csv, you should use the url given by the Raw link in the github page for getting raw csv response , which is – https://raw.githubusercontent.com/cs109/2014_data/master/countries.csv

Example –

import pandas as pd
import io
import requests
url="https://raw.githubusercontent.com/cs109/2014_data/master/countries.csv"
s=requests.get(url).content
c=pd.read_csv(io.StringIO(s.decode('utf-8')))

Question 3

In the latest version of pandas (0.19.2) you can directly pass the url

import pandas as pd

url="https://raw.githubusercontent.com/cs109/2014_data/master/countries.csv"
c=pd.read_csv(url)

Question 4

As I commented you need to use a StringIO object and decode i.e c=pd.read_csv(io.StringIO(s.decode("utf-8"))) if using requests, you need to decode as .content returns bytes if you used .text you would just need to pass s as is s = requests.get(url).text c = pd.read_csv(StringIO(s)).

A simpler approach is to pass the correct url of the raw data directly to read_csv, you don’t have to pass a file like object, you can pass a url so you don’t need requests at all:

c = pd.read_csv("https://raw.githubusercontent.com/cs109/2014_data/master/countries.csv")

print(c)

Output:

                              Country         Region
0                             Algeria         AFRICA
1                              Angola         AFRICA
2                               Benin         AFRICA
3                            Botswana         AFRICA
4                             Burkina         AFRICA
5                             Burundi         AFRICA
6                            Cameroon         AFRICA
..................................

From the docs:

filepath_or_buffer :

string or file handle / StringIO The string could be a URL. Valid URL schemes include http, ftp, s3, and file. For file URLs, a host is expected. For instance, a local file could be file ://localhost/path/to/table.csv

Question 5

The problem you’re having is that the output you get into the variable ‘s’ is not a csv, but a html file. In order to get the raw csv, you have to modify the url to:

‘https://raw.githubusercontent.com/cs109/2014_data/master/countries.csv‘

Your second problem is that read_csv expects a file name, we can solve this by using StringIO from io module. Third problem is that request.get(url).content delivers a byte stream, we can solve this using the request.get(url).text instead.

End result is this code:

from io import StringIO

import pandas as pd
import requests
url='https://raw.githubusercontent.com/cs109/2014_data/master/countries.csv'
s=requests.get(url).text

c=pd.read_csv(StringIO(s))

output:

>>> c.head()
    Country  Region
0   Algeria  AFRICA
1    Angola  AFRICA
2     Benin  AFRICA
3  Botswana  AFRICA
4   Burkina  AFRICA

Question 6

url = "https://github.com/cs109/2014_data/blob/master/countries.csv"
c = pd.read_csv(url, sep = "\t")

Question 7

To Import Data through URL in pandas just apply the simple below code it works actually better.

import pandas as pd
train = pd.read_table("https://urlandfile.com/dataset.csv")
train.head()

If you are having issues with a raw data then just put ‘r’ before URL

import pandas as pd
train = pd.read_table(r"https://urlandfile.com/dataset.csv")
train.head()

Question 8

I’m using python’s ftplib to write a small FTP client, but some of the functions in the package don’t return string output, but print to stdout. I want to redirect stdout to an object which I’ll be able to read the output from.

I know stdout can be redirected into any regular file with:

stdout = open("file", "a")

But I prefer a method that doesn’t uses the local drive.

I’m looking for something like the BufferedReader in Java that can be used to wrap a buffer into a stream.

Question 9

from cStringIO import StringIO # Python3 use: from io import StringIO
import sys

old_stdout = sys.stdout
sys.stdout = mystdout = StringIO()

# blah blah lots of code ...

sys.stdout = old_stdout

# examine mystdout.getvalue()

Question 10

There is contextlib.redirect_stdout() function in Python 3.4:

import io
from contextlib import redirect_stdout

with io.StringIO() as buf, redirect_stdout(buf):
    print('redirected')
    output = buf.getvalue()

Here’s code example that shows how to implement it on older Python versions.

Question 11

Just to add to Ned’s answer above: you can use this to redirect output to any object that implements a write(str) method.

This can be used to good effect to “catch” stdout output in a GUI application.

Here’s a silly example in PyQt:

import sys
from PyQt4 import QtGui

class OutputWindow(QtGui.QPlainTextEdit):
    def write(self, txt):
        self.appendPlainText(str(txt))

app = QtGui.QApplication(sys.argv)
out = OutputWindow()
sys.stdout=out
out.show()
print "hello world !"

Question 12

Starting with Python 2.6 you can use anything implementing the TextIOBase API from the io module as a replacement. This solution also enables you to use sys.stdout.buffer.write() in Python 3 to write (already) encoded byte strings to stdout (see stdout in Python 3). Using StringIO wouldn’t work then, because neither sys.stdout.encoding nor sys.stdout.buffer would be available.

A solution using TextIOWrapper:

import sys
from io import TextIOWrapper, BytesIO

# setup the environment
old_stdout = sys.stdout
sys.stdout = TextIOWrapper(BytesIO(), sys.stdout.encoding)

# do something that writes to stdout or stdout.buffer

# get output
sys.stdout.seek(0)      # jump to the start
out = sys.stdout.read() # read output

# restore stdout
sys.stdout.close()
sys.stdout = old_stdout

This solution works for Python 2 >= 2.6 and Python 3.

Please note that our new sys.stdout.write() only accepts unicode strings and sys.stdout.buffer.write() only accepts byte strings. This might not be the case for old code, but is often the case for code that is built to run on Python 2 and 3 without changes, which again often makes use of sys.stdout.buffer.

You can build a slight variation that accepts unicode and byte strings for write():

class StdoutBuffer(TextIOWrapper):
    def write(self, string):
        try:
            return super(StdoutBuffer, self).write(string)
        except TypeError:
            # redirect encoded byte strings directly to buffer
            return super(StdoutBuffer, self).buffer.write(string)

You don’t have to set the encoding of the buffer the sys.stdout.encoding, but this helps when using this method for testing/comparing script output.

Question 13

This method restores sys.stdout even if there’s an exception. It also gets any output before the exception.

import io
import sys

real_stdout = sys.stdout
fake_stdout = io.BytesIO()   # or perhaps io.StringIO()
try:
    sys.stdout = fake_stdout
    # do what you have to do to create some output
finally:
    sys.stdout = real_stdout
    output_string = fake_stdout.getvalue()
    fake_stdout.close()
    # do what you want with the output_string

Tested in Python 2.7.10 using io.BytesIO()

Tested in Python 3.6.4 using io.StringIO()

Bob, added for a case if you feel anything from the modified / extended code experimentation might get interesting in any sense, otherwise feel free to delete it

^{Ad informandum … a few remarks from extended experimentation during finding some viable mechanics to “grab” outputs, directed by numexpr.print_versions() directly to the <stdout> ( upon a need to clean GUI and collecting details into debugging-report )}

# THIS WORKS AS HELL: as Bob Stein proposed years ago:
#  py2 SURPRISEDaBIT:
#
import io
import sys
#
real_stdout = sys.stdout                        #           PUSH <stdout> ( store to REAL_ )
fake_stdout = io.BytesIO()                      #           .DEF FAKE_
try:                                            # FUSED .TRY:
    sys.stdout.flush()                          #           .flush() before
    sys.stdout = fake_stdout                    #           .SET <stdout> to use FAKE_
    # ----------------------------------------- #           +    do what you gotta do to create some output
    print 123456789                             #           + 
    import  numexpr                             #           + 
    QuantFX.numexpr.__version__                 #           + [3] via fake_stdout re-assignment, as was bufferred + "late" deferred .get_value()-read into print, to finally reach -> real_stdout
    QuantFX.numexpr.print_versions()            #           + [4] via fake_stdout re-assignment, as was bufferred + "late" deferred .get_value()-read into print, to finally reach -> real_stdout
    _ = os.system( 'echo os.system() redir-ed' )#           + [1] via real_stdout                                 + "late" deferred .get_value()-read into print, to finally reach -> real_stdout, if not ( _ = )-caught from RET-d "byteswritten" / avoided from being injected int fake_stdout
    _ = os.write(  sys.stderr.fileno(),         #           + [2] via      stderr                                 + "late" deferred .get_value()-read into print, to finally reach -> real_stdout, if not ( _ = )-caught from RET-d "byteswritten" / avoided from being injected int fake_stdout
                       b'os.write()  redir-ed' )#  *OTHERWISE, if via fake_stdout, EXC <_io.BytesIO object at 0x02C0BB10> Traceback (most recent call last):
    # ----------------------------------------- #           ?                              io.UnsupportedOperation: fileno
    #'''                                                    ? YET:        <_io.BytesIO object at 0x02C0BB10> has a .fileno() method listed
    #>>> 'fileno' in dir( sys.stdout )       -> True        ? HAS IT ADVERTISED,
    #>>> pass;            sys.stdout.fileno  -> <built-in method fileno of _io.BytesIO object at 0x02C0BB10>
    #>>> pass;            sys.stdout.fileno()-> Traceback (most recent call last):
    #                                             File "<stdin>", line 1, in <module>
    #                                           io.UnsupportedOperation: fileno
    #                                                       ? BUT REFUSES TO USE IT
    #'''
finally:                                        # == FINALLY:
    sys.stdout.flush()                          #           .flush() before ret'd back REAL_
    sys.stdout = real_stdout                    #           .SET <stdout> to use POP'd REAL_
    sys.stdout.flush()                          #           .flush() after  ret'd back REAL_
    out_string = fake_stdout.getvalue()         #           .GET string           from FAKE_
    fake_stdout.close()                         #                <FD>.close()
    # +++++++++++++++++++++++++++++++++++++     # do what you want with the out_string
    #
    print "\n{0:}\n{1:}{0:}".format( 60 * "/\\",# "LATE" deferred print the out_string at the very end reached -> real_stdout
                                     out_string #                   
                                     )
'''
PASS'd:::::
...
os.system() redir-ed
os.write()  redir-ed
/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
123456789
'2.5'
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
Numexpr version:   2.5
NumPy version:     1.10.4
Python version:    2.7.13 |Anaconda 4.0.0 (32-bit)| (default, May 11 2017, 14:07:41) [MSC v.1500 32 bit (Intel)]
AMD/Intel CPU?     True
VML available?     True
VML/MKL version:   Intel(R) Math Kernel Library Version 11.3.1 Product Build 20151021 for 32-bit applications
Number of threads used by default: 4 (out of 4 detected cores)
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
>>>

EXC'd :::::
...
os.system() redir-ed
/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
123456789
'2.5'
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
Numexpr version:   2.5
NumPy version:     1.10.4
Python version:    2.7.13 |Anaconda 4.0.0 (32-bit)| (default, May 11 2017, 14:07:41) [MSC v.1500 32 bit (Intel)]
AMD/Intel CPU?     True
VML available?     True
VML/MKL version:   Intel(R) Math Kernel Library Version 11.3.1 Product Build 20151021 for 32-bit applications
Number of threads used by default: 4 (out of 4 detected cores)
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\

Traceback (most recent call last):
  File "<stdin>", line 9, in <module>
io.UnsupportedOperation: fileno
'''

Question 14

A context manager for python3:

import sys
from io import StringIO


class RedirectedStdout:
    def __init__(self):
        self._stdout = None
        self._string_io = None

    def __enter__(self):
        self._stdout = sys.stdout
        sys.stdout = self._string_io = StringIO()
        return self

    def __exit__(self, type, value, traceback):
        sys.stdout = self._stdout

    def __str__(self):
        return self._string_io.getvalue()

use like this:

>>> with RedirectedStdout() as out:
>>>     print('asdf')
>>>     s = str(out)
>>>     print('bsdf')
>>> print(s, out)
'asdf\n' 'asdf\nbsdf\n'

Question 15

In Python3.6, the StringIO and cStringIO modules are gone, you should use io.StringIO instead.So you should do this like the first answer:

import sys
from io import StringIO

old_stdout = sys.stdout
old_stderr = sys.stderr
my_stdout = sys.stdout = StringIO()
my_stderr = sys.stderr = StringIO()

# blah blah lots of code ...

sys.stdout = self.old_stdout
sys.stderr = self.old_stderr

// if you want to see the value of redirect output, be sure the std output is turn back
print(my_stdout.getvalue())
print(my_stderr.getvalue())

my_stdout.close()
my_stderr.close()

Question 16

Use pipe() and write to the appropriate file descriptor.

https://docs.python.org/library/os.html#file-descriptor-operations

Question 17

Here’s another take on this. contextlib.redirect_stdout with io.StringIO() as documented is great, but it’s still a bit verbose for every day use. Here’s how to make it a one-liner by subclassing contextlib.redirect_stdout:

import sys
import io
from contextlib import redirect_stdout

class capture(redirect_stdout):

    def __init__(self):
        self.f = io.StringIO()
        self._new_target = self.f
        self._old_targets = []  # verbatim from parent class

    def __enter__(self):
        self._old_targets.append(getattr(sys, self._stream))  # verbatim from parent class
        setattr(sys, self._stream, self._new_target)  # verbatim from parent class
        return self  # instead of self._new_target in the parent class

    def __repr__(self):
        return self.f.getvalue()

Since __enter__ returns self, you have the context manager object available after the with block exits. Moreover, thanks to the __repr__ method, the string representation of the context manager object is, in fact, stdout. So now you have,

with capture() as message:
    print('Hello World!')
print(str(message)=='Hello World!\n')  # returns True

Question 18

How would I generate a random date that has to be between two other given dates?

The function’s signature should be something like this:

random_date("1/1/2008 1:30 PM", "1/1/2009 4:50 AM", 0.34)
                   ^                       ^          ^

            date generated has  date generated has  a random number
            to be after this    to be before this

and would return a date such as: 2/4/2008 7:20 PM

Question 19

Convert both strings to timestamps (in your chosen resolution, e.g. milliseconds, seconds, hours, days, whatever), subtract the earlier from the later, multiply your random number (assuming it is distributed in the range [0, 1]) with that difference, and add again to the earlier one. Convert the timestamp back to date string and you have a random time in that range.

Python example (output is almost in the format you specified, other than 0 padding – blame the American time format conventions):

import random
import time

def str_time_prop(start, end, format, prop):
    """Get a time at a proportion of a range of two formatted times.

    start and end should be strings specifying times formated in the
    given format (strftime-style), giving an interval [start, end].
    prop specifies how a proportion of the interval to be taken after
    start.  The returned time will be in the specified format.
    """

    stime = time.mktime(time.strptime(start, format))
    etime = time.mktime(time.strptime(end, format))

    ptime = stime + prop * (etime - stime)

    return time.strftime(format, time.localtime(ptime))


def random_date(start, end, prop):
    return str_time_prop(start, end, '%m/%d/%Y %I:%M %p', prop)

print(random_date("1/1/2008 1:30 PM", "1/1/2009 4:50 AM", random.random()))

Question 20

from random import randrange
from datetime import timedelta

def random_date(start, end):
    """
    This function will return a random datetime between two datetime 
    objects.
    """
    delta = end - start
    int_delta = (delta.days * 24 * 60 * 60) + delta.seconds
    random_second = randrange(int_delta)
    return start + timedelta(seconds=random_second)

The precision is seconds. You can increase precision up to microseconds, or decrease to, say, half-hours, if you want. For that just change the last line’s calculation.

example run:

from datetime import datetime

d1 = datetime.strptime('1/1/2008 1:30 PM', '%m/%d/%Y %I:%M %p')
d2 = datetime.strptime('1/1/2009 4:50 AM', '%m/%d/%Y %I:%M %p')

print(random_date(d1, d2))

output:

2008-12-04 01:50:17

Question 21

A tiny version.

import datetime
import random


def random_date(start, end):
    """Generate a random datetime between `start` and `end`"""
    return start + datetime.timedelta(
        # Get a random amount of seconds between `start` and `end`
        seconds=random.randint(0, int((end - start).total_seconds())),
    )

Note that both start and end arguments should be datetime objects. If you’ve got strings instead, it’s fairly easy to convert. The other answers point to some ways to do so.

Question 22

Updated answer

It’s even more simple using Faker.

Installation

pip install faker

Usage:

from faker import Faker
fake = Faker()

fake.date_between(start_date='today', end_date='+30y')
# datetime.date(2025, 3, 12)

fake.date_time_between(start_date='-30y', end_date='now')
# datetime.datetime(2007, 2, 28, 11, 28, 16)

# Or if you need a more specific date boundaries, provide the start 
# and end dates explicitly.
import datetime
start_date = datetime.date(year=2015, month=1, day=1)
fake.date_between(start_date=start_date, end_date='+30y')

Old answer

It’s very simple using radar

Installation

pip install radar

Usage

import datetime

import radar 

# Generate random datetime (parsing dates from str values)
radar.random_datetime(start='2000-05-24', stop='2013-05-24T23:59:59')

# Generate random datetime from datetime.datetime values
radar.random_datetime(
    start = datetime.datetime(year=2000, month=5, day=24),
    stop = datetime.datetime(year=2013, month=5, day=24)
)

# Just render some random datetime. If no range is given, start defaults to 
# 1970-01-01 and stop defaults to datetime.datetime.now()
radar.random_datetime()

Question 23

This is a different approach – that sort of works..

from random import randint
import datetime

date=datetime.date(randint(2005,2025), randint(1,12),randint(1,28))

BETTER APPROACH

startdate=datetime.date(YYYY,MM,DD)
date=startdate+datetime.timedelta(randint(1,365))

Question 24

Since Python 3 timedelta supports multiplication with floats, so now you can do:

import random
random_date = start + (end - start) * random.random()

given that start and end are of the type datetime.datetime. For example, to generate a random datetime within the next day:

import random
from datetime import datetime, timedelta

start = datetime.now()
end = start + timedelta(days=1)
random_date = start + (end - start) * random.random()

Question 25

To chip in a pandas-based solution I use:

import pandas as pd
import numpy as np

def random_date(start, end, position=None):
    start, end = pd.Timestamp(start), pd.Timestamp(end)
    delta = (end - start).total_seconds()
    if position is None:
        offset = np.random.uniform(0., delta)
    else:
        offset = position * delta
    offset = pd.offsets.Second(offset)
    t = start + offset
    return t

I like it, because of the nice pd.Timestamp features that allow me to throw different stuff and formats at it. Consider the following few examples…

Your signature.

>>> random_date(start="1/1/2008 1:30 PM", end="1/1/2009 4:50 AM", position=0.34)
Timestamp('2008-05-04 21:06:48', tz=None)

Random position.

>>> random_date(start="1/1/2008 1:30 PM", end="1/1/2009 4:50 AM")
Timestamp('2008-10-21 05:30:10', tz=None)

Different format.

>>> random_date('2008-01-01 13:30', '2009-01-01 4:50')
Timestamp('2008-11-18 17:20:19', tz=None)

Passing pandas/datetime objects directly.

>>> random_date(pd.datetime.now(), pd.datetime.now() + pd.offsets.Hour(3))
Timestamp('2014-03-06 14:51:16.035965', tz=None)

Question 26

Here is an answer to the literal meaning of the title rather than the body of this question:

import time
import datetime
import random

def date_to_timestamp(d) :
  return int(time.mktime(d.timetuple()))

def randomDate(start, end):
  """Get a random date between two dates"""

  stime = date_to_timestamp(start)
  etime = date_to_timestamp(end)

  ptime = stime + random.random() * (etime - stime)

  return datetime.date.fromtimestamp(ptime)

This code is based loosely on the accepted answer.

Question 27

You can Use Mixer,

pip install mixer

and,

from mixer import generators as gen
print gen.get_datetime(min_datetime=(1900, 1, 1, 0, 0, 0), max_datetime=(2020, 12, 31, 23, 59, 59))

Question 28

#!/usr/bin/env python
# -*- coding: utf-8 -*-

"""Create random datetime object."""

from datetime import datetime
import random


def create_random_datetime(from_date, to_date, rand_type='uniform'):
    """
    Create random date within timeframe.

    Parameters
    ----------
    from_date : datetime object
    to_date : datetime object
    rand_type : {'uniform'}

    Examples
    --------
    >>> random.seed(28041990)
    >>> create_random_datetime(datetime(1990, 4, 28), datetime(2000, 12, 31))
    datetime.datetime(1998, 12, 13, 23, 38, 0, 121628)
    >>> create_random_datetime(datetime(1990, 4, 28), datetime(2000, 12, 31))
    datetime.datetime(2000, 3, 19, 19, 24, 31, 193940)
    """
    delta = to_date - from_date
    if rand_type == 'uniform':
        rand = random.random()
    else:
        raise NotImplementedError('Unknown random mode \'{}\''
                                  .format(rand_type))
    return from_date + rand * delta


if __name__ == '__main__':
    import doctest
    doctest.testmod()

Question 29

Convert your dates into timestamps and call random.randint with the timestamps, then convert the randomly generated timestamp back into a date:

from datetime import datetime
import random

def random_date(first_date, second_date):
    first_timestamp = int(first_date.timestamp())
    second_timestamp = int(second_date.timestamp())
    random_timestamp = random.randint(first_timestamp, second_timestamp)
    return datetime.fromtimestamp(random_timestamp)

Then you can use it like this

from datetime import datetime

d1 = datetime.strptime("1/1/2018 1:30 PM", "%m/%d/%Y %I:%M %p")
d2 = datetime.strptime("1/1/2019 4:50 AM", "%m/%d/%Y %I:%M %p")

random_date(d1, d2)

random_date(d2, d1)  # ValueError because the first date comes after the second date

If you care about timezones you should just use date_time_between_dates from the Faker library, where I stole this code from, as a different answer already suggests.

Question 30

Convert your input dates to numbers (int, float, whatever is best for your usage)
Choose a number between your two date numbers.
Convert this number back to a date.

Many algorithms for converting date to and from numbers are already available in many operating systems.

Question 31

What do you need the random number for? Usually (depending on the language) you can get the number of seconds/milliseconds from the Epoch from a date. So for a randomd date between startDate and endDate you could do:

compute the time in ms between startDate and endDate (endDate.toMilliseconds() – startDate.toMilliseconds())
generate a number between 0 and the number you obtained in 1
generate a new Date with time offset = startDate.toMilliseconds() + number obtained in 2

Question 32

The easiest way of doing this is to convert both numbers to timestamps, then set these as the minimum and maximum bounds on a random number generator.

A quick PHP example would be:

// Find a randomDate between $start_date and $end_date
function randomDate($start_date, $end_date)
{
    // Convert to timetamps
    $min = strtotime($start_date);
    $max = strtotime($end_date);

    // Generate random number using above bounds
    $val = rand($min, $max);

    // Convert back to desired date format
    return date('Y-m-d H:i:s', $val);
}

This function makes use of strtotime() to convert a datetime description into a Unix timestamp, and date() to make a valid date out of the random timestamp which has been generated.

Question 33

Just to add another one:

datestring = datetime.datetime.strftime(datetime.datetime( \
    random.randint(2000, 2015), \
    random.randint(1, 12), \
    random.randint(1, 28), \
    random.randrange(23), \
    random.randrange(59), \
    random.randrange(59), \
    random.randrange(1000000)), '%Y-%m-%d %H:%M:%S')

The day handling needs some considerations. With 28 you are on the secure site.

Question 34

Here’s a solution modified from emyller’s approach which returns an array of random dates at any resolution

import numpy as np

def random_dates(start, end, size=1, resolution='s'):
    """
    Returns an array of random dates in the interval [start, end]. Valid 
    resolution arguments are numpy date/time units, as documented at: 
        https://docs.scipy.org/doc/numpy-dev/reference/arrays.datetime.html
    """
    start, end = np.datetime64(start), np.datetime64(end)
    delta = (end-start).astype('timedelta64[{}]'.format(resolution))
    delta_mat = np.random.randint(0, delta.astype('int'), size)
    return start + delta_mat.astype('timedelta64[{}]'.format(resolution))

Part of what’s nice about this approach is that np.datetime64 is really good at coercing things to dates, so you can specify your start/end dates as strings, datetimes, pandas timestamps… pretty much anything will work.

Question 35

Conceptually it’s quite simple. Depending on which language you’re using you will be able to convert those dates into some reference 32 or 64 bit integer, typically representing seconds since epoch (1 January 1970) otherwise known as “Unix time” or milliseconds since some other arbitrary date. Simply generate a random 32 or 64 bit integer between those two values. This should be a one liner in any language.

On some platforms you can generate a time as a double (date is the integer part, time is the fractional part is one implementation). The same principle applies except you’re dealing with single or double precision floating point numbers (“floats” or “doubles” in C, Java and other languages). Subtract the difference, multiply by random number (0 <= r <= 1), add to start time and done.

Question 36

In python:

>>> from dateutil.rrule import rrule, DAILY
>>> import datetime, random
>>> random.choice(
                 list(
                     rrule(DAILY, 
                           dtstart=datetime.date(2009,8,21), 
                           until=datetime.date(2010,10,12))
                     )
                 )
datetime.datetime(2010, 2, 1, 0, 0)

(need python dateutil library – pip install python-dateutil)

Question 37

Use ApacheCommonUtils to generate a random long within a given range, and then create Date out of that long.

Example:

import org.apache.commons.math.random.RandomData;

import org.apache.commons.math.random.RandomDataImpl;

public Date nextDate(Date min, Date max) {

RandomData randomData = new RandomDataImpl();

return new Date(randomData.nextLong(min.getTime(), max.getTime()));

}

Question 38

I made this for another project using random and time. I used a general format from time you can view the documentation here for the first argument in strftime(). The second part is a random.randrange function. It returns an integer between the arguments. Change it to the ranges that match the strings you would like. You must have nice arguments in the tuple of the second arugment.

import time
import random


def get_random_date():
    return strftime("%Y-%m-%d %H:%M:%S",(random.randrange(2000,2016),random.randrange(1,12),
    random.randrange(1,28),random.randrange(1,24),random.randrange(1,60),random.randrange(1,60),random.randrange(1,7),random.randrange(0,366),1))

Question 39

Pandas + numpy solution

import pandas as pd
import numpy as np

def RandomTimestamp(start, end):
    dts = (end - start).total_seconds()
    return start + pd.Timedelta(np.random.uniform(0, dts), 's')

dts is the difference between timestamps in seconds (float). It is then used to create a pandas timedelta between 0 and dts, that is added to the start timestamp.

Question 40

Based on the answer by mouviciel, here is a vectorized solution using numpy. Convert the start and end dates to ints, generate an array of random numbers between them, and convert the whole array back to dates.

import time
import datetime
import numpy as np

n_rows = 10

start_time = "01/12/2011"
end_time = "05/08/2017"

date2int = lambda s: time.mktime(datetime.datetime.strptime(s,"%d/%m/%Y").timetuple())
int2date = lambda s: datetime.datetime.fromtimestamp(s).strftime('%Y-%m-%d %H:%M:%S')

start_time = date2int(start_time)
end_time = date2int(end_time)

random_ints = np.random.randint(low=start_time, high=end_time, size=(n_rows,1))
random_dates = np.apply_along_axis(int2date, 1, random_ints).reshape(n_rows,1)

print random_dates

Question 41

It’s modified method of @(Tom Alsberg). I modified it to get date with milliseconds.

import random
import time
import datetime

def random_date(start_time_string, end_time_string, format_string, random_number):
    """
    Get a time at a proportion of a range of two formatted times.
    start and end should be strings specifying times formated in the
    given format (strftime-style), giving an interval [start, end].
    prop specifies how a proportion of the interval to be taken after
    start.  The returned time will be in the specified format.
    """
    dt_start = datetime.datetime.strptime(start_time_string, format_string)
    dt_end = datetime.datetime.strptime(end_time_string, format_string)

    start_time = time.mktime(dt_start.timetuple()) + dt_start.microsecond / 1000000.0
    end_time = time.mktime(dt_end.timetuple()) + dt_end.microsecond / 1000000.0

    random_time = start_time + random_number * (end_time - start_time)

    return datetime.datetime.fromtimestamp(random_time).strftime(format_string)

Example:

print TestData.TestData.random_date("2000/01/01 00:00:00.000000", "2049/12/31 23:59:59.999999", '%Y/%m/%d %H:%M:%S.%f', random.random())

Output: 2028/07/08 12:34:49.977963

Question 42

start_timestamp = time.mktime(time.strptime('Jun 1 2010  01:33:00', '%b %d %Y %I:%M:%S'))
end_timestamp = time.mktime(time.strptime('Jun 1 2017  12:33:00', '%b %d %Y %I:%M:%S'))
time.strftime('%b %d %Y %I:%M:%S',time.localtime(randrange(start_timestamp,end_timestamp)))

refer

Question 43

    # needed to create data for 1000 fictitious employees for testing code 
    # code relating to randomly assigning forenames, surnames, and genders
    # has been removed as not germaine to the question asked above but FYI
    # genders were randomly assigned, forenames/surnames were web scrapped,
    # there is no accounting for leap years, and the data stored in mySQL

    import random 
    from datetime import datetime
    from datetime import timedelta

    for employee in range(1000):
        # assign a random date of birth (employees are aged between sixteen and sixty five)
        dlt = random.randint(365*16, 365*65)
        dob = datetime.today() - timedelta(days=dlt)
        # assign a random date of hire sometime between sixteenth birthday and yesterday
        doh = datetime.today() - timedelta(days=random.randint(1, dlt-365*16))
        print("born {} hired {}".format(dob.strftime("%d-%m-%y"), doh.strftime("%d-%m-%y")))

Question 44

Alternative way to create random dates between two dates using np.random.randint(), pd.Timestamp().value and pd.to_datetime() with for loop:

# Import libraries
import pandas as pd

# Initialize
start = '2020-01-01' # Specify start date
end = '2020-03-10' # Specify end date
n = 10 # Specify number of dates needed

# Get random dates
x = np.random.randint(pd.Timestamp(start).value, pd.Timestamp(end).value,n)
random_dates = [pd.to_datetime((i/10**9)/(60*60)/24, unit='D').strftime('%Y-%m-%d')  for i in x]

print(random_dates)

Output

['2020-01-06',
 '2020-03-08',
 '2020-01-23',
 '2020-02-03',
 '2020-01-30',
 '2020-01-05',
 '2020-02-16',
 '2020-03-08',
 '2020-02-09',
 '2020-01-04']

Question 45

I don’t understand what %s and %d do and how they work.

Question 46

They are used for formatting strings. %s acts a placeholder for a string while %d acts as a placeholder for a number. Their associated values are passed in via a tuple using the % operator.

name = 'marcog'
number = 42
print '%s %d' % (name, number)

will print marcog 42. Note that name is a string (%s) and number is an integer (%d for decimal).

See https://docs.python.org/3/library/stdtypes.html#printf-style-string-formatting for details.

In Python 3 the example would be:

print('%s %d' % (name, number))

Question 47

from python 3 doc

%d is for decimal integer

%s is for generic string or object and in case of object, it will be converted to string

Consider the following code

name ='giacomo'
number = 4.3
print('%s %s %d %f %g' % (name, number, number, number, number))

the out put will be

giacomo 4.3 4 4.300000 4.3

as you can see %d will truncate to integer, %s will maintain formatting, %f will print as float and %g is used for generic number

obviously

print('%d' % (name))

will generate an exception; you cannot convert string to number

Question 48

%s is used as a placeholder for string values you want to inject into a formatted string.

%d is used as a placeholder for numeric or decimal values.

For example (for python 3)

print ('%s is %d years old' % ('Joe', 42))

Would output

Joe is 42 years old

Question 49

These are placeholders:

For example: 'Hi %s I have %d donuts' %('Alice', 42)

This line of code will substitute %s with Alice (str) and %d with 42.

Output: 'Hi Alice I have 42 donuts'

This could be achieved with a “+” most of the time. To gain a deeper understanding to your question, you may want to check {} / .format() as well. Here is one example: Python string formatting: % vs. .format

also see here a google python tutorial video @ 40′, it has some explanations https://www.youtube.com/watch?v=tKTZoB2Vjuk

Question 50

The %d and %s string formatting “commands” are used to format strings. The %d is for numbers, and %s is for strings.

For an example:

print("%s" % "hi")

and

print("%d" % 34.6)

To pass multiple arguments:

print("%s %s %s%d" % ("hi", "there", "user", 123456)) will return hi there user123456

Question 51

These are all informative answers, but none are quite getting at the core of what the difference is between %s and %d.

%s tells the formatter to call the str() function on the argument and since we are coercing to a string by definition, %s is essentially just performing str(arg).

%d on the other hand, is calling int() on the argument before calling str(), like str(int(arg)), This will cause int coercion as well as str coercion.

For example, I can convert a hex value to decimal,

>>> '%d' % 0x15
'21'

or truncate a float.

>>> '%d' % 34.5
'34'

But the operation will raise an exception if the argument isn’t a number.

>>> '%d' % 'thirteen'
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: %d format: a number is required, not str

So if the intent is just to call str(arg), then %s is sufficient, but if you need extra formatting (like formatting float decimal places) or other coercion, then the other format symbols are needed.

With the f-string notation, when you leave the formatter out, the default is str.

>>> a = 1
>>> f'{a}'
'1'
>>> f'{a:d}'
'1'
>>> a = '1'
>>> f'{a:d}'
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ValueError: Unknown format code 'd' for object of type 'str'

The same is true with string.format; the default is str.

>>> a = 1
>>> '{}'.format(a)
'1'
>>> '{!s}'.format(a)
'1'
>>> '{:d}'.format(a)
'1'

Question 52

%d and %s are placeholders, they work as a replaceable variable. For example, if you create 2 variables

variable_one = "Stackoverflow"
variable_two = 45

you can assign those variables to a sentence in a string using a tuple of the variables.

variable_3 = "I was searching for an answer in %s and found more than %d answers to my question"

Note that %s works for String and %d work for numerical or decimal variables.

if you print variable_3 it would look like this

print(variable_3 % (variable_one, variable_two))

I was searching for an answer in StackOverflow and found more than 45 answers to my question.

Question 53

They are format specifiers. They are used when you want to include the value of your Python expressions into strings, with a specific format enforced.

See Dive into Python for a relatively detailed introduction.

Question 54

As per latest standards, this is how it should be done.

print("My name is {!s} and my number is{:d}".format("Agnel Vishal",100))

Do check python3.6 docs and sample program

Question 55

In case you would like to avoid %s or %d then..

name = 'marcog'
number = 42
print ('my name is',name,'and my age is:', number)

Output:

my name is marcog and my name is 42

Question 56

%s is used to hold space for string %d is used to hold space for number

name = "Moses";
age = 23
print("My name is %s am CEO at MoTech Computers " %name)
print("Current am %d years old" %age)
print("So Am %s and am %d years old" %(name,age))

Program output

this video goes deep about that tip https://www.youtube.com/watch?v=4zN5YsuiqMA

Question 57

speaking of which …
python3.6 comes with f-strings which makes things much easier in formatting!
now if your python version is greater than 3.6 you can format your strings with these available methods:

name = "python"

print ("i code with %s" %name)          # with help of older method
print ("i code with {0}".format(name))  # with help of format
print (f"i code with {name}")           # with help of f-strings

Question 58

My understanding is that MagicMock is a superset of Mock that automatically does “magic methods” thus seamlessly providing support for lists, iterations and so on… Then what is the reason for plain Mock existing? Isn’t that just a stripped down version of MagicMock that can be practically ignored? Does Mock class know any tricks that are not available in MagicMock?

Question 59

What is the reason for plain Mock existing?

Mock’s author, Michael Foord, addressed a very similar question at Pycon 2011 (31:00):

Q: Why was MagicMock made a separate thing rather than just folding the ability into the default mock object?

A: One reasonable answer is that the way MagicMock works is that it preconfigures all these protocol methods by creating new Mocks and setting them, so if every new mock created a bunch of new mocks and set those as protocol methods and then all of those protocol methods created a bunch more mocks and set them on their protocol methods, you’ve got infinite recursion…

What if you want accessing your mock as a container object to be an error — you don’t want that to work? If every mock has automatically got every protocol method, then it becomes much more difficult to do that. And also, MagicMock does some of this preconfiguring for you, setting return values that might not be appropriate, so I thought it would be better to have this convenience one that has everything preconfigured and available for you, but you can also take a ordinary mock object and just configure the magic methods you want to exist…

The simple answer is: just use MagicMock everywhere if that’s the behavior you want.

Question 60

With Mock you can mock magic methods but you have to define them. MagicMock has “default implementations of most of the magic methods.”.

If you don’t need to test any magic methods, Mock is adequate and doesn’t bring a lot of extraneous things into your tests. If you need to test a lot of magic methods MagicMock will save you some time.

Question 61

To begin with, MagicMock is a subclass of Mock.

class MagicMock(MagicMixin, Mock)

As a result, MagicMock provides everything that Mock provides and more. Rather than thinking of Mock as being a stripped down version of MagicMock, think of MagicMock as an extended version of Mock. This should address your questions about why Mock exists and what does Mock provide on top of MagicMock.

Secondly, MagicMock provides default implementations of many/most magic methods, whereas Mock doesn’t. See here for more information on the magic methods provided.

Some examples of provided magic methods:

>>> int(Mock())
TypeError: int() argument must be a string or a number, not 'Mock'
>>> int(MagicMock())
1
>>> len(Mock())
TypeError: object of type 'Mock' has no len()
>>> len(MagicMock())
0

And these which may not be as intuitive (at least not intuitive to me):

>>> with MagicMock():
...     print 'hello world'
...
hello world
>>> MagicMock()[1]
<MagicMock name='mock.__getitem__()' id='4385349968'>

You can “see” the methods added to MagicMock as those methods are invoked for the first time:

>>> magic1 = MagicMock()
>>> dir(magic1)
['assert_any_call', 'assert_called_once_with', ...]
>>> int(magic1)
1
>>> dir(magic1)
['__int__', 'assert_any_call', 'assert_called_once_with', ...]
>>> len(magic1)
0
>>> dir(magic1)
['__int__', '__len__', 'assert_any_call', 'assert_called_once_with', ...]

So, why not use MagicMock all the time?

The question back to you is: Are you okay with the default magic method implementations? For example, is it okay for mocked_object[1] to not error? Are you okay with any unintended consequences due to the magic method implementations being already there?

If the answer to these questions is a yes, then go ahead and use MagicMock. Otherwise, stick to Mock.

Question 62

This is what python’s official documentation says:

In most of these examples the Mock and MagicMock classes are interchangeable. As the MagicMock is the more capable class it makes a sensible one to use by default.

Question 63

I’ve found another particular case where simple Mock may turn more useful than MagicMock:

In [1]: from unittest.mock import Mock, MagicMock, ANY
In [2]: mock = Mock()
In [3]: magic = MagicMock()
In [4]: mock.foo == ANY
Out[4]: True
In [5]: magic.foo == ANY
Out[5]: False

Comparing against ANY can be useful, for example, comparing almost every key between two dictionaries where some value is calculated using a mock.

This will be valid if you’re using Mock:


self.assertDictEqual(my_dict, {
  'hello': 'world',
  'another': ANY
})

while it will raise an AssertionError if you’ve used MagicMock

Question 64

On Ubuntu 10.04 by default Python 2.6 is installed, then I have installed Python 2.7. How can I use pip install to install packages for Python 2.7.

For example:

pip install beautifulsoup4

by default installs BeautifulSoup for Python 2.6

When I do:

import bs4

in Python 2.6 it works, but in Python 2.7 it says:

No module named bs4

问题：网址中的熊猫read_csv

回答 0

更新资料

Update

回答 1

回答 2

回答 3

回答 4

回答 5

要通过熊猫中的URL导入数据，只需应用下面的简单代码即可，实际上效果更好。

如果您对原始数据有疑问，则只需在网址前添加“ r”

To Import Data through URL in pandas just apply the simple below code it works actually better.

If you are having issues with a raw data then just put ‘r’ before URL

问题：我可以将python中的stdout重定向到某种字符串缓冲区吗？

回答 0

回答 1

回答 2

回答 3

回答 4

回答 5

回答 6

回答 7

回答 8

问题：在其他两个日期之间生成一个随机日期

回答 0

回答 1

回答 2

回答 3

更新的答案

安装

用法：

旧答案

安装

用法

Updated answer

Installation

Usage:

Old answer

Installation

Usage

回答 4

回答 5

回答 6

回答 7

回答 8

回答 9

回答 10

回答 11

回答 12

回答 13

回答 14

回答 15

回答 16

回答 17

回答 18

回答 19

回答 20

回答 21

回答 22

回答 23

回答 24

回答 25

问题：Python字符串格式中的％s和％d有什么区别？

回答 0

回答 1

回答 2

回答 3

回答 4

回答 5

回答 6

回答 7

回答 8

回答 9

回答 10

回答 11

问题：模拟与魔术模拟

回答 0

纯模拟存在的原因是什么？

What is the reason for plain Mock existing?

回答 1