Python 实用宝典

Question 1

Let’s say df is a pandas DataFrame. I would like to find all columns of numeric type. Something like:

isNumeric = is_numeric(df)

Question 2

You could use select_dtypes method of DataFrame. It includes two parameters include and exclude. So isNumeric would look like:

numerics = ['int16', 'int32', 'int64', 'float16', 'float32', 'float64']

newdf = df.select_dtypes(include=numerics)

Question 3

You can use the undocumented function _get_numeric_data() to filter only numeric columns:

df._get_numeric_data()

Example:

In [32]: data
Out[32]:
   A  B
0  1  s
1  2  s
2  3  s
3  4  s

In [33]: data._get_numeric_data()
Out[33]:
   A
0  1
1  2
2  3
3  4

Note that this is a “private method” (i.e., an implementation detail) and is subject to change or total removal in the future. Use with caution.

Question 4

Simple one-line answer to create a new dataframe with only numeric columns:

df.select_dtypes(include=np.number)

If you want the names of numeric columns:

df.select_dtypes(include=np.number).columns.tolist()

Complete code:

import pandas as pd
import numpy as np

df = pd.DataFrame({'A': range(7, 10),
                   'B': np.random.rand(3),
                   'C': ['foo','bar','baz'],
                   'D': ['who','what','when']})
df
#    A         B    C     D
# 0  7  0.704021  foo   who
# 1  8  0.264025  bar  what
# 2  9  0.230671  baz  when

df_numerics_only = df.select_dtypes(include=np.number)
df_numerics_only
#    A         B
# 0  7  0.704021
# 1  8  0.264025
# 2  9  0.230671

colnames_numerics_only = df.select_dtypes(include=np.number).columns.tolist()
colnames_numerics_only
# ['A', 'B']

Question 5

df.select_dtypes(exclude = ['object'])

Update

df.select_dtypes(inlcude = np.number)
#or with new version of panda
df.select_dtypes('number')

Question 6

Simple one-liner:

df.select_dtypes('number').columns

Question 7

Following codes will return list of names of the numeric columns of a data set.

cnames=list(marketing_train.select_dtypes(exclude=['object']).columns)

here marketing_train is my data set and select_dtypes() is function to select data types using exclude and include arguments and columns is used to fetch the column name of data set output of above code will be following:

['custAge',
     'campaign',
     'pdays',
     'previous',
     'emp.var.rate',
     'cons.price.idx',
     'cons.conf.idx',
     'euribor3m',
     'nr.employed',
     'pmonths',
     'pastEmail']

Thanks

Question 8

This is another simple code for finding numeric column in pandas data frame,

numeric_clmns = df.dtypes[df.dtypes != "object"].index

Question 9

def is_type(df, baseType):
    import numpy as np
    import pandas as pd
    test = [issubclass(np.dtype(d).type, baseType) for d in df.dtypes]
    return pd.DataFrame(data = test, index = df.columns, columns = ["test"])
def is_float(df):
    import numpy as np
    return is_type(df, np.float)
def is_number(df):
    import numpy as np
    return is_type(df, np.number)
def is_integer(df):
    import numpy as np
    return is_type(df, np.integer)

Question 10

Adapting this answer, you could do

df.ix[:,df.applymap(np.isreal).all(axis=0)]

Here, np.applymap(np.isreal) shows whether every cell in the data frame is numeric, and .axis(all=0) checks if all values in a column are True and returns a series of Booleans that can be used to index the desired columns.

Question 11

Please see the below code:

if(dataset.select_dtypes(include=[np.number]).shape[1] > 0):
display(dataset.select_dtypes(include=[np.number]).describe())
if(dataset.select_dtypes(include=[np.object]).shape[1] > 0):
display(dataset.select_dtypes(include=[np.object]).describe())

This way you can check whether the value are numeric such as float and int or the srting values. the second if statement is used for checking the string values which is referred by the object.

Question 12

We can include and exclude data types as per the requirement as below:

train.select_dtypes(include=None, exclude=None)
train.select_dtypes(include='number') #will include all the numeric types

Referred from Jupyter Notebook.

To select all numeric types, use np.number or 'number'

To select strings you must use the object dtype but note that this will return all object dtype columns
See the NumPy dtype hierarchy <http://docs.scipy.org/doc/numpy/reference/arrays.scalars.html>__
To select datetimes, use np.datetime64, 'datetime' or 'datetime64'
To select timedeltas, use np.timedelta64, 'timedelta' or 'timedelta64'
To select Pandas categorical dtypes, use 'category'
To select Pandas datetimetz dtypes, use 'datetimetz' (new in 0.20.0) or “’datetime64[ns, tz]’

Question 13

How can I update a row’s information?

For example I’d like to alter the name column of the row that has the id 5.

Question 14

Retrieve an object using the tutorial shown in the Flask-SQLAlchemy documentation. Once you have the entity that you want to change, change the entity itself. Then, db.session.commit().

For example:

admin = User.query.filter_by(username='admin').first()
admin.email = 'my_new_email@example.com'
db.session.commit()

user = User.query.get(5)
user.name = 'New Name'
db.session.commit()

Flask-SQLAlchemy is based on SQLAlchemy, so be sure to check out the SQLAlchemy Docs as well.

Question 15

There is a method update on BaseQuery object in SQLAlchemy, which is returned by filter_by.

admin = User.query.filter_by(username='admin').update(dict(email='my_new_email@example.com')))
db.session.commit()

The advantage of using update over changing the entity comes when there are many objects to be updated.

If you want to give add_user permission to all the admins,

rows_changed = User.query.filter_by(role='admin').update(dict(permission='add_user'))
db.session.commit()

Notice that filter_by takes keyword arguments (use only one =) as opposed to filter which takes an expression.

Question 16

This does not work if you modify a pickled attribute of the model. Pickled attributes should be replaced in order to trigger updates:

from flask import Flask
from flask.ext.sqlalchemy import SQLAlchemy
from pprint import pprint

app = Flask(__name__)
app.config['SQLALCHEMY_DATABASE_URI'] = 'sqllite:////tmp/users.db'
db = SQLAlchemy(app)


class User(db.Model):
    id = db.Column(db.Integer, primary_key=True)
    name = db.Column(db.String(80), unique=True)
    data = db.Column(db.PickleType())

    def __init__(self, name, data):
        self.name = name
        self.data = data

    def __repr__(self):
        return '<User %r>' % self.username

db.create_all()

# Create a user.
bob = User('Bob', {})
db.session.add(bob)
db.session.commit()

# Retrieve the row by its name.
bob = User.query.filter_by(name='Bob').first()
pprint(bob.data)  # {}

# Modifying data is ignored.
bob.data['foo'] = 123
db.session.commit()
bob = User.query.filter_by(name='Bob').first()
pprint(bob.data)  # {}

# Replacing data is respected.
bob.data = {'bar': 321}
db.session.commit()
bob = User.query.filter_by(name='Bob').first()
pprint(bob.data)  # {'bar': 321}

# Modifying data is ignored.
bob.data['moo'] = 789
db.session.commit()
bob = User.query.filter_by(name='Bob').first()
pprint(bob.data)  # {'bar': 321}

Question 17

Just assigning the value and committing them will work for all the data types but JSON and Pickled attributes. Since pickled type is explained above I’ll note down a slightly different but easy way to update JSONs.

class User(db.Model):
    id = db.Column(db.Integer, primary_key=True)
    name = db.Column(db.String(80), unique=True)
    data = db.Column(db.JSON)

def __init__(self, name, data):
    self.name = name
    self.data = data

Let’s say the model is like above.

user = User("Jon Dove", {"country":"Sri Lanka"})
db.session.add(user)
db.session.flush()
db.session.commit()

This will add the user into the MySQL database with data {“country”:”Sri Lanka”}

Modifying data will be ignored. My code that didn’t work is as follows.

user = User.query().filter(User.name=='Jon Dove')
data = user.data
data["province"] = "south"
user.data = data
db.session.merge(user)
db.session.flush()
db.session.commit()

Instead of going through the painful work of copying the JSON to a new dict (not assigning it to a new variable as above), which should have worked I found a simple way to do that. There is a way to flag the system that JSONs have changed.

Following is the working code.

from sqlalchemy.orm.attributes import flag_modified
user = User.query().filter(User.name=='Jon Dove')
data = user.data
data["province"] = "south"
user.data = data
flag_modified(user, "data")
db.session.merge(user)
db.session.flush()
db.session.commit()

This worked like a charm. There is another method proposed along with this method here Hope I’ve helped some one.

Question 18

I’m trying to find out why the use of global is considered to be bad practice in python (and in programming in general). Can somebody explain? Links with more info would also be appreciated.

Question 19

This has nothing to do with Python; global variables are bad in any programming language.

However, global constants are not conceptually the same as global variables; global constants are perfectly harmless. In Python the distinction between the two is purely by convention: CONSTANTS_ARE_CAPITALIZED and globals_are_not.

The reason global variables are bad is that they enable functions to have hidden (non-obvious, surprising, hard to detect, hard to diagnose) side effects, leading to an increase in complexity, potentially leading to Spaghetti code.

However, sane use of global state is acceptable (as is local state and mutability) even in functional programming, either for algorithm optimization, reduced complexity, caching and memoization, or the practicality of porting structures originating in a predominantly imperative codebase.

All in all, your question can be answered in many ways, so your best bet is to just google “why are global variables bad”. Some examples:

If you want to go deeper and find out why side effects are all about, and many other enlightening things, you should learn Functional Programming:

Question 20

Yes, in theory, globals (and “state” in general) are evil. In practice, if you look into your python’s packages directory you’ll find that most modules there start with a bunch of global declarations. Obviously, people have no problem with them.

Specifically to python, globals’ visibility is limited to a module, therefore there are no “true” globals that affect the whole program – that makes them a way less harmful. Another point: there are no const, so when you need a constant you have to use a global.

In my practice, if I happen to modify a global in a function, I always declare it with global, even if there technically no need for that, as in:

cache = {}

def foo(args):
    global cache

    cache[args] = ...

This makes globals’ manipulations easier to track down.

Question 21

A personal opinion on the topic is that having global variables being used in a function logic means that some other code can alter the logic and the expected output of that function which will make debugging very hard (especially in big projects) and will make testing harder as well.

Furthermore, if you consider other people reading your code (open-source community, colleagues etc) they will have a hard time trying to understand where the global variable is being set, where has been changed and what to expect from this global variable as opposed to an isolated function that its functionality can be determined by reading the function definition itself.

(Probably) Violating Pure Function definition

I believe that a clean and (nearly) bug-free code should have functions that are as pure as possible (see pure functions). A pure function is the one that has the following conditions:

The function always evaluates the same result value given the same argument value(s). The function result value cannot depend on any hidden information or state that may change while program execution proceeds or between different executions of the program, nor can it depend on any external input from I/O devices (usually—see below).
Evaluation of the result does not cause any semantically observable side effect or output, such as mutation of mutable objects or output to I/O devices.

Having global variables is violating at least one of the above if not both as an external code can probably cause unexpected results.

Another clear definition of pure functions: “Pure function is a function that takes all of its inputs as explicit arguments and produces all of its outputs as explicit results.” [1]. Having global variables violates the idea of pure functions since an input and maybe one of the outputs (the global variable) is not explicitly being given or returned.

(Probably) Violating Unit testing F.I.R.S.T principle

Further on that, if you consider unit-testing and the F.I.R.S.T principle (Fast tests, Independent tests, Repeatable, Self-Validating and Timely) will probably violate the Independent tests principle (which means that tests don’t depend on each other).

Having a global variable (not always) but in most of the cases (at least of what I have seen so far) is to prepare and pass results to other functions. This violates this principle as well. If the global variable has been used in that way (i.e the global variable used in function X has to be set in a function Y first) it means that to unit test function X you have to run test/run function Y first.

Globals as constants

On the other hand and as other people have already mentioned, if the global variable is used as a “constant” variable can be slightly better since the language does not support constants. However, I always prefer working with classes and having the “constants” as a class member and not use a global variable at all. If you have a code that two different classes require to share a global variable then you probably need to refactor your solution and make your classes independent.

I don’t believe that globals shouldn’t be used. But if they are used the authors should consider some principles (the ones mentioned above perhaps and other software engineering principles and good practices) for a cleaner and nearly bug-free code.

Question 22

They are essential, the screen being a good example. However, in a multithreaded environment or with many developers involved, in practice often the question arises: who did (erraneously) set or clear it? Depending on the architecture, analysis can be costly and be required often. While reading the global var can be ok, writing to it must be controlled, for example by a single thread or threadsafe class. Hence, global vars arise the fear of high development costs possible by the consequences for which themselves are considered evil. Therefore in general, it’s good practice to keep the number of global vars low.

Question 23

I have the following shell script for a very simple HTTP server:

#!/bin/sh

echo "Serving at http://localhost:3000"
python -m SimpleHTTPServer 3000

I was wondering how I can enable or add a CORS header like Access-Control-Allow-Origin: * to this server?

Question 24

Unfortunately, the simple HTTP server is really that simple that it does not allow any customization, especially not for the headers it sends. You can however create a simple HTTP server yourself, using most of SimpleHTTPRequestHandler, and just add that desired header.

For that, simply create a file simple-cors-http-server.py (or whatever) and, depending on the Python version you are using, put one of the following codes inside.

Then you can do python simple-cors-http-server.py and it will launch your modified server which will set the CORS header for every response.

With the shebang at the top, make the file executable and put it into your PATH, and you can just run it using simple-cors-http-server.py too.

Python 3 solution

Python 3 uses SimpleHTTPRequestHandler and HTTPServer from the http.server module to run the server:

#!/usr/bin/env python3
from http.server import HTTPServer, SimpleHTTPRequestHandler, test
import sys

class CORSRequestHandler (SimpleHTTPRequestHandler):
    def end_headers (self):
        self.send_header('Access-Control-Allow-Origin', '*')
        SimpleHTTPRequestHandler.end_headers(self)

if __name__ == '__main__':
    test(CORSRequestHandler, HTTPServer, port=int(sys.argv[1]) if len(sys.argv) > 1 else 8000)

Python 2 solution

Python 2 uses SimpleHTTPServer.SimpleHTTPRequestHandler and the BaseHTTPServer module to run the server.

#!/usr/bin/env python2
from SimpleHTTPServer import SimpleHTTPRequestHandler
import BaseHTTPServer

class CORSRequestHandler (SimpleHTTPRequestHandler):
    def end_headers (self):
        self.send_header('Access-Control-Allow-Origin', '*')
        SimpleHTTPRequestHandler.end_headers(self)

if __name__ == '__main__':
    BaseHTTPServer.test(CORSRequestHandler, BaseHTTPServer.HTTPServer)

Python 2 & 3 solution

If you need compatibility for both Python 3 and Python 2, you could use this polyglot script that works in both versions. It first tries to import from the Python 3 locations, and otherwise falls back to Python 2:

#!/usr/bin/env python
try:
    # Python 3
    from http.server import HTTPServer, SimpleHTTPRequestHandler, test as test_orig
    import sys
    def test (*args):
        test_orig(*args, port=int(sys.argv[1]) if len(sys.argv) > 1 else 8000)
except ImportError: # Python 2
    from BaseHTTPServer import HTTPServer, test
    from SimpleHTTPServer import SimpleHTTPRequestHandler

class CORSRequestHandler (SimpleHTTPRequestHandler):
    def end_headers (self):
        self.send_header('Access-Control-Allow-Origin', '*')
        SimpleHTTPRequestHandler.end_headers(self)

if __name__ == '__main__':
    test(CORSRequestHandler, HTTPServer)

Question 25

Try an alternative like http-server

As SimpleHTTPServer is not really the kind of server you deploy to production, I’m assuming here that you don’t care that much about which tool you use as long as it does the job of exposing your files at http://localhost:3000 with CORS headers in a simple command line

# install (it requires nodejs/npm)
npm install http-server -g

#run
http-server -p 3000 --cors

Need HTTPS?

If you need https in local you can also try caddy or certbot

Some related tools you might find useful

ngrok: when running ngrok http 3000, it creates an url https://$random.ngrok.com that permits anyone to access your http://localhost:3000 server. It can expose to the world what runs locally on your computer (including local backends/apis)
localtunnel: almost the same as ngrok
now: when running now, it uploads your static assets online and deploy them to https://$random.now.sh. They remain online forever unless you decide otherwise. Deployment is fast (except the first one) thanks to diffing. Now is suitable for production frontend/SPA code deployment It can also deploy Docker and NodeJS apps. It is not really free, but they have a free plan.

Question 26

I had the same problem and came to this solution:

class Handler(SimpleHTTPRequestHandler):
    def send_response(self, *args, **kwargs):
        SimpleHTTPRequestHandler.send_response(self, *args, **kwargs)
        self.send_header('Access-Control-Allow-Origin', '*')

I simply created a new class inheriting from SimpleHTTPRequestHandler that only changes the send_response method.

Question 27

You’ll need to provide your own instances of do_GET() (and do_HEAD() if choose to support HEAD operations). something like this:

class MyHTTPServer(SimpleHTTPServer):

    allowed_hosts = (('127.0.0.1', 80),)

    def do_GET(self):
        if self.client_address not in allowed_hosts:
            self.send_response(401, 'request not allowed')
        else:
            super(MyHTTPServer, self).do_Get()

Question 28

I know about islower and isupper, but can you check whether or not that character is a letter? For Example:

>>> s = 'abcdefg'
>>> s2 = '123abcd'
>>> s3 = 'abcDEFG'
>>> s[0].islower()
True

>>> s2[0].islower()
False

>>> s3[0].islower()
True

Is there any way to just ask if it is a character besides doing .islower() or .isupper()?

Question 29

You can use str.isalpha().

For example:

s = 'a123b'

for char in s:
    print(char, char.isalpha())

Output:

a True
1 False
2 False
3 False
b True

Question 30

str.isalpha()

Return true if all characters in the string are alphabetic and there is at least one character, false otherwise. Alphabetic characters are those characters defined in the Unicode character database as “Letter”, i.e., those with general category property being one of “Lm”, “Lt”, “Lu”, “Ll”, or “Lo”. Note that this is different from the “Alphabetic” property defined in the Unicode Standard.

In python2.x:

>>> s = u'a1中文'
>>> for char in s: print char, char.isalpha()
...
a True
1 False
中 True
文 True
>>> s = 'a1中文'
>>> for char in s: print char, char.isalpha()
...
a True
1 False
� False
� False
� False
� False
� False
� False
>>>

In python3.x:

>>> s = 'a1中文'
>>> for char in s: print(char, char.isalpha())
...
a True
1 False
中 True
文 True
>>>

This code work:

>>> def is_alpha(word):
...     try:
...         return word.encode('ascii').isalpha()
...     except:
...         return False
...
>>> is_alpha('中国')
False
>>> is_alpha(u'中国')
False
>>>

>>> a = 'ａ'
>>> b = 'a'
>>> ord(a), ord(b)
(65345, 97)
>>> a.isalpha(), b.isalpha()
(True, True)
>>> is_alpha(a), is_alpha(b)
(False, True)
>>>

Question 31

I found a good way to do this with using a function and basic code. This is a code that accepts a string and counts the number of capital letters, lowercase letters and also ‘other’. Other is classed as a space, punctuation mark or even Japanese and Chinese characters.

def check(count):

    lowercase = 0
    uppercase = 0
    other = 0

    low = 'a','b','c','d','e','f','g','h','i','j','k','l','m','n','o','p','q','r','s','t','u','v','w','x','y','z'
    upper = 'A','B','C','D','E','F','G','H','I','J','K','L','M','N','O','P','Q','R','S','T','U','V','W','X','Y','Z'



    for n in count:
        if n in low:
            lowercase += 1
        elif n in upper:
            uppercase += 1
        else:
            other += 1

    print("There are " + str(lowercase) + " lowercase letters.")
    print("There are " + str(uppercase) + " uppercase letters.")
    print("There are " + str(other) + " other elements to this sentence.")

Question 32

data = "abcdefg hi j 12345"

digits_count = 0
letters_count = 0
others_count = 0

for i in userinput:

    if i.isdigit():
        digits_count += 1 
    elif i.isalpha():
        letters_count += 1
    else:
        others_count += 1

print("Result:")        
print("Letters=", letters_count)
print("Digits=", digits_count)

Output:

Please Enter Letters with Numbers:
abcdefg hi j 12345
Result:
Letters = 10
Digits = 5

By using str.isalpha() you can check if it is a letter.

Question 33

This works:

any(c.isalpha() for c in 'string')

Question 34

This works:

word = str(input("Enter string:"))
notChar = 0
isChar = 0
for char in word:
    if not char.isalpha():
        notChar += 1
    else:
        isChar += 1
print(isChar, " were letters; ", notChar, " were not letters.")

Question 35

I am surprised this specific question hasn’t been asked before, but I really didn’t find it on SO nor on the documentation of np.sort.

Say I have a random numpy array holding integers, e.g:

> temp = np.random.randint(1,10, 10)    
> temp
array([2, 4, 7, 4, 2, 2, 7, 6, 4, 4])

If I sort it, I get ascending order by default:

> np.sort(temp)
array([2, 2, 2, 4, 4, 4, 4, 6, 7, 7])

but I want the solution to be sorted in descending order.

Now, I know I can always do:

reverse_order = np.sort(temp)[::-1]

but is this last statement efficient? Doesn’t it create a copy in ascending order, and then reverses this copy to get the result in reversed order? If this is indeed the case, is there an efficient alternative? It doesn’t look like np.sort accepts parameters to change the sign of the comparisons in the sort operation to get things in reverse order.

Question 36

temp[::-1].sort() sorts the array in place, whereas np.sort(temp)[::-1] creates a new array.

In [25]: temp = np.random.randint(1,10, 10)

In [26]: temp
Out[26]: array([5, 2, 7, 4, 4, 2, 8, 6, 4, 4])

In [27]: id(temp)
Out[27]: 139962713524944

In [28]: temp[::-1].sort()

In [29]: temp
Out[29]: array([8, 7, 6, 5, 4, 4, 4, 4, 2, 2])

In [30]: id(temp)
Out[30]: 139962713524944

Question 37

>>> a=np.array([5, 2, 7, 4, 4, 2, 8, 6, 4, 4])

>>> np.sort(a)
array([2, 2, 4, 4, 4, 4, 5, 6, 7, 8])

>>> -np.sort(-a)
array([8, 7, 6, 5, 4, 4, 4, 4, 2, 2])

Question 38

For short arrays I suggest using np.argsort() by finding the indices of the sorted negatived array, which is slightly faster than reversing the sorted array:

In [37]: temp = np.random.randint(1,10, 10)

In [38]: %timeit np.sort(temp)[::-1]
100000 loops, best of 3: 4.65 µs per loop

In [39]: %timeit temp[np.argsort(-temp)]
100000 loops, best of 3: 3.91 µs per loop

Question 39

Unfortunately when you have a complex array, only np.sort(temp)[::-1] works properly. The two other methods mentioned here are not effective.

Question 40

Be careful with dimensions.

Let

x  # initial numpy array
I = np.argsort(x) or I = x.argsort() 
y = np.sort(x)    or y = x.sort()
z  # reverse sorted array

Full Reverse

z = x[I[::-1]]
z = -np.sort(-x)
z = np.flip(y)

flip changed in 1.15, previous versions 1.14 required axis. Solution: pip install --upgrade numpy.

First Dimension Reversed

z = y[::-1]
z = np.flipud(y)
z = np.flip(y, axis=0)

Second Dimension Reversed

z = y[::-1, :]
z = np.fliplr(y)
z = np.flip(y, axis=1)

Testing

Testing on a 100×10×10 array 1000 times.

Method       | Time (ms)
-------------+----------
y[::-1]      | 0.126659  # only in first dimension
-np.sort(-x) | 0.133152
np.flip(y)   | 0.121711
x[I[::-1]]   | 4.611778

x.sort()     | 0.024961
x.argsort()  | 0.041830
np.flip(x)   | 0.002026

This is mainly due to reindexing rather than argsort.

# Timing code
import time
import numpy as np


def timeit(fun, xs):
    t = time.time()
    for i in range(len(xs)):  # inline and map gave much worse results for x[-I], 5*t
        fun(xs[i])
    t = time.time() - t
    print(np.round(t,6))

I, N = 1000, (100, 10, 10)
xs = np.random.rand(I,*N)
timeit(lambda x: np.sort(x)[::-1], xs)
timeit(lambda x: -np.sort(-x), xs)
timeit(lambda x: np.flip(x.sort()), xs)
timeit(lambda x: x[x.argsort()[::-1]], xs)
timeit(lambda x: x.sort(), xs)
timeit(lambda x: x.argsort(), xs)
timeit(lambda x: np.flip(x), xs)

Question 41

Hello I was searching for a solution to reverse sorting a two dimensional numpy array, and I couldn’t find anything that worked, but I think I have stumbled on a solution which I am uploading just in case anyone is in the same boat.

x=np.sort(array)
y=np.fliplr(x)

np.sort sorts ascending which is not what you want, but the command fliplr flips the rows left to right! Seems to work!

Hope it helps you out!

I guess it’s similar to the suggest about -np.sort(-a) above but I was put off going for that by comment that it doesn’t always work. Perhaps my solution won’t always work either however I have tested it with a few arrays and seems to be OK.

Question 42

You could sort the array first (Ascending by default) and then apply np.flip() (https://docs.scipy.org/doc/numpy/reference/generated/numpy.flip.html)

FYI It works with datetime objects as well.

Example:

    x = np.array([2,3,1,0]) 
    x_sort_asc=np.sort(x) 
    print(x_sort_asc)

    >>> array([0, 1, 2, 3])

    x_sort_desc=np.flip(x_sort_asc) 
    print(x_sort_desc)

    >>> array([3,2,1,0])

Question 43

Here is a quick trick

In[3]: import numpy as np
In[4]: temp = np.random.randint(1,10, 10)
In[5]: temp
Out[5]: array([5, 4, 2, 9, 2, 3, 4, 7, 5, 8])

In[6]: sorted = np.sort(temp)
In[7]: rsorted = list(reversed(sorted))
In[8]: sorted
Out[8]: array([2, 2, 3, 4, 4, 5, 5, 7, 8, 9])

In[9]: rsorted
Out[9]: [9, 8, 7, 5, 5, 4, 4, 3, 2, 2]

Question 44

i suggest using this …

np.arange(start_index, end_index, intervals)[::-1]

for example:

np.arange(10, 20, 0.5)
np.arange(10, 20, 0.5)[::-1]

Then your resault:

[ 19.5,  19. ,  18.5,  18. ,  17.5,  17. ,  16.5,  16. ,  15.5,
    15. ,  14.5,  14. ,  13.5,  13. ,  12.5,  12. ,  11.5,  11. ,
    10.5,  10. ]

Question 45

I am trying my very first formal python program using Threading and Multiprocessing on a windows machine. I am unable to launch the processes though, with python giving the following message. The thing is, I am not launching my threads in the main module. The threads are handled in a separate module inside a class.

EDIT: By the way this code runs fine on ubuntu. Not quite on windows

RuntimeError: 
            Attempt to start a new process before the current process
            has finished its bootstrapping phase.
            This probably means that you are on Windows and you have
            forgotten to use the proper idiom in the main module:
                if __name__ == '__main__':
                    freeze_support()
                    ...
            The "freeze_support()" line can be omitted if the program
            is not going to be frozen to produce a Windows executable.

My original code is pretty long, but I was able to reproduce the error in an abridged version of the code. It is split in two files, the first is the main module and does very little other than import the module which handles processes/threads and calls a method. The second module is where the meat of the code is.

testMain.py:

import parallelTestModule

extractor = parallelTestModule.ParallelExtractor()
extractor.runInParallel(numProcesses=2, numThreads=4)

parallelTestModule.py:

import multiprocessing
from multiprocessing import Process
import threading

class ThreadRunner(threading.Thread):
    """ This class represents a single instance of a running thread"""
    def __init__(self, name):
        threading.Thread.__init__(self)
        self.name = name
    def run(self):
        print self.name,'\n'

class ProcessRunner:
    """ This class represents a single instance of a running process """
    def runp(self, pid, numThreads):
        mythreads = []
        for tid in range(numThreads):
            name = "Proc-"+str(pid)+"-Thread-"+str(tid)
            th = ThreadRunner(name)
            mythreads.append(th) 
        for i in mythreads:
            i.start()
        for i in mythreads:
            i.join()

class ParallelExtractor:    
    def runInParallel(self, numProcesses, numThreads):
        myprocs = []
        prunner = ProcessRunner()
        for pid in range(numProcesses):
            pr = Process(target=prunner.runp, args=(pid, numThreads)) 
            myprocs.append(pr) 
#        if __name__ == 'parallelTestModule':    #This didnt work
#        if __name__ == '__main__':              #This obviously doesnt work
#        multiprocessing.freeze_support()        #added after seeing error to no avail
        for i in myprocs:
            i.start()

        for i in myprocs:
            i.join()

Question 46

On Windows the subprocesses will import (i.e. execute) the main module at start. You need to insert an if __name__ == '__main__': guard in the main module to avoid creating subprocesses recursively.

Modified testMain.py:

import parallelTestModule

if __name__ == '__main__':    
    extractor = parallelTestModule.ParallelExtractor()
    extractor.runInParallel(numProcesses=2, numThreads=4)

Question 47

Try putting your code inside a main function in testMain.py

import parallelTestModule

if __name__ ==  '__main__':
  extractor = parallelTestModule.ParallelExtractor()
  extractor.runInParallel(numProcesses=2, numThreads=4)

See the docs:

"For an explanation of why (on Windows) the if __name__ == '__main__' 
part is necessary, see Programming guidelines."

which say

“Make sure that the main module can be safely imported by a new Python interpreter without causing unintended side effects (such a starting a new process).”

… by using if __name__ == '__main__'

Question 48

Though the earlier answers are correct, there’s a small complication it would help to remark on.

In case your main module imports another module in which global variables or class member variables are defined and initialized to (or using) some new objects, you may have to condition that import in the same way:

if __name__ ==  '__main__':
  import my_module

Question 49

As @Ofer said, when you are using another libraries or modules, you should import all of them inside the if __name__ == '__main__':

So, in my case, ended like this:

if __name__ == '__main__':       
    import librosa
    import os
    import pandas as pd
    run_my_program()

Question 50

In my case it was a simple bug in the code, using a variable before it was created. Worth checking that out before trying the above solutions. Why I got this particular error message, Lord knows.

Question 51

How to get the code of the headers through urllib?

Question 52

The getcode() method (Added in python2.6) returns the HTTP status code that was sent with the response, or None if the URL is no HTTP URL.

>>> a=urllib.urlopen('http://www.google.com/asdfsf')
>>> a.getcode()
404
>>> a=urllib.urlopen('http://www.google.com/')
>>> a.getcode()
200

Question 53

You can use urllib2 as well:

import urllib2

req = urllib2.Request('http://www.python.org/fish.html')
try:
    resp = urllib2.urlopen(req)
except urllib2.HTTPError as e:
    if e.code == 404:
        # do something...
    else:
        # ...
except urllib2.URLError as e:
    # Not an HTTP-specific error (e.g. connection refused)
    # ...
else:
    # 200
    body = resp.read()

Note that HTTPError is a subclass of URLError which stores the HTTP status code.

Question 54

For Python 3:

import urllib.request, urllib.error

url = 'http://www.google.com/asdfsf'
try:
    conn = urllib.request.urlopen(url)
except urllib.error.HTTPError as e:
    # Return code error (e.g. 404, 501, ...)
    # ...
    print('HTTPError: {}'.format(e.code))
except urllib.error.URLError as e:
    # Not an HTTP-specific error (e.g. connection refused)
    # ...
    print('URLError: {}'.format(e.reason))
else:
    # 200
    # ...
    print('good')

Question 55

import urllib2

try:
    fileHandle = urllib2.urlopen('http://www.python.org/fish.html')
    data = fileHandle.read()
    fileHandle.close()
except urllib2.URLError, e:
    print 'you got an error with the code', e

Question 56

I’m trying learn Python (3 to be more specific) and I’m getting this error:

ValueError: zero length field name in format

I googled it and I found out you need to specify the numbers:

a, b = 0, 1
if a < b:
     print('a ({0}) is less than b ({1})'.format(a, b))
else:
     print('a ({0}) is not less than b ({1})'.format(a, b))

And not like the tutorial (from lynda.com) actually says to do:

a, b = 0, 1
if a < b:
     print('a ({}) is less than b ({})'.format(a, b))
else:
     print('a ({}) is not less than b ({})'.format(a, b))

The tutorial im following has Python 3.1, and im using 3.2 and what i read about this error is that this only happens in <3.1 (3.0). Did they undo this in 3.2, or am i doing something wrong?

Also, speak slowly ;) this is literally my first night learning Python and only the 2nd “script” i’ve written in Python.

Question 57

I’m gonna guess that you are running python 2.6 by accident somehow.

This feature is only available for at least 3.1 if you are using python 3, or 2.7 if you are using python 2.

Question 58

Python 2.6 and 3.0 require the field numbers. In Python 2.7 and later and 3.1 and later, they can be omitted.

Changed in version 2.7: The positional argument specifiers can be omitted, so ‘{} {}’ is equivalent to ‘{0} {1}’.

python2.6.4>>> print '|{0:^12}|{1:^12}|'.format(3,4)
|     3      |     4     |

Question 59

If you’re using Eclipse you should look into Window -> Preferences -> PyDev -> Interpreter – Python. There you have a list of interpreters (with name and location). If for your current project you’re using interpreter which is located for example in /usr/bin/python then probably executing /usr/bin/python -V whill give you something like “Python 2.6.6”. And there is your answer like Winston Ewert wrote.

(you can add new interperter by simply clicking “New…” button and giving /usr/bin/python3 as “location”. Then you have probably to change your project settings (Preferences -> PyDev – Interpreter/Grammar).

Question 60

I have Python classes, of which I need only one instance at runtime, so it would be sufficient to have the attributes only once per class and not per instance. If there would be more than one instance (which won’t happen), all instance should have the same configuration. I wonder which of the following options would be better or more “idiomatic” Python.

Class variables:

class MyController(Controller):

  path = "something/"
  children = [AController, BController]

  def action(self, request):
    pass

Instance variables:

class MyController(Controller):

  def __init__(self):
    self.path = "something/"
    self.children = [AController, BController]

  def action(self, request):
    pass

Question 61

If you have only one instance anyway, it’s best to make all variables per-instance, simply because they will be accessed (a little bit) faster (one less level of “lookup” due to the “inheritance” from class to instance), and there are no downsides to weigh against this small advantage.

Question 62

Further echoing Mike’s and Alex’s advice and adding my own color…

Using instance attributes are the typical… the more idiomatic Python. Class attributes are not used used as much, since their use cases are specific. The same is true for static and class methods vs. “normal” methods. They’re special constructs addressing specific use cases, else it’s code created by an aberrant programmer wanting to show off they know some obscure corner of Python programming.

Alex mentions in his reply that access will be (a little bit) faster due to one less level of lookup… let me further clarify for those who don’t know about how this works yet. It is very similar to variable access — the search order of which is:

locals
nonlocals
globals
built-ins

For attribute access, the order is:

instance
class
base classes as determined by the MRO (method resolution order)

Both techniques work in an “inside-out” manner, meaning the most local objects are checked first, then outer layers are checked in succession.

In your example above, let’s say you’re looking up the path attribute. When it encounters a reference like “self.path“, Python will look at the instance attributes first for a match. When that fails, it checks the class from which the object was instantiated from. Finally, it will search the base classes. As Alex stated, if your attribute is found in the instance, it doesn’t need to look elsewhere, hence your little bit of time savings.

However, if you insist on class attributes, you need that extra lookup. Or, your other alternative is to refer to the object via the class instead of the instance, e.g., MyController.path instead of self.path. That’s a direct lookup which will get around the deferred lookup, but as alex mentions below, it’s a global variable, so you lose that bit that you thought you were going to save (unless you create a local reference to the [global] class name).

The bottom-line is that you should use instance attributes most of the time. However, there will be occasions where a class attribute is the right tool for the job. Code using both at the same time will require the most diligence, because using self will only get you the instance attribute object and shadows access to the class attribute of the same name. In this case, you must use access the attribute by the class name in order to reference it.

Question 63

When in doubt, you probably want an instance attribute.

Class attributes are best reserved for special cases where they make sense. The only very-common use case is methods. It isn’t uncommon to use class attributes for read-only constants that instances need to know (though the only benefit to this is if you also want access from outside the class), but you should certainly be cautious about storing any state in them, which is seldom what you want. Even if you will only have one instance, you should write the class like you would any other, which usually means using instance attributes.

Question 64

Same question at Performance of accessing class variables in Python – the code here adapted from @Edward Loper

Local Variables are the fastest to access, pretty much tied with Module Variables, followed by Class Variables, followed by Instance Variables.

There are 4 scopes you can access variables from:

Instance Variables (self.varname)
Class Variables (Classname.varname)
Module Variables (VARNAME)
Local Variables (varname)

The test:

import timeit

setup='''
XGLOBAL= 5
class A:
    xclass = 5
    def __init__(self):
        self.xinstance = 5
    def f1(self):
        xlocal = 5
        x = self.xinstance
    def f2(self):
        xlocal = 5
        x = A.xclass
    def f3(self):
        xlocal = 5
        x = XGLOBAL
    def f4(self):
        xlocal = 5
        x = xlocal
a = A()
'''
print('access via instance variable: %.3f' % timeit.timeit('a.f1()', setup=setup, number=300000000) )
print('access via class variable: %.3f' % timeit.timeit('a.f2()', setup=setup, number=300000000) )
print('access via module variable: %.3f' % timeit.timeit('a.f3()', setup=setup, number=300000000) )
print('access via local variable: %.3f' % timeit.timeit('a.f4()', setup=setup, number=300000000) )

The result:

access via instance variable: 93.456
access via class variable: 82.169
access via module variable: 72.634
access via local variable: 72.199

问题：如何在Pandas中找到数字列？

回答 0

回答 1

回答 2

回答 3

回答 4

回答 5

回答 6

回答 7

回答 8

回答 9

回答 10

问题：Flask-SQLalchemy更新行的信息

回答 0

回答 1

回答 2

回答 3

问题：为什么全局变量是邪恶的？[关闭]

回答 0

回答 1

回答 2

（可能）违反纯函数定义

（可能）违反单元测试FIRST原则

全局常量

(Probably) Violating Pure Function definition

(Probably) Violating Unit testing F.I.R.S.T principle

Globals as constants

回答 3

问题：在简单的HTTP服务器上启用访问控制

回答 0

Python 3解决方案

Python 2解决方案

Python 2和3解决方案

Python 3 solution

Python 2 solution

Python 2 & 3 solution

回答 1

回答 2

回答 3

问题：如何检查字符串中的字符是否为字母？（Python）

回答 0

回答 1

回答 2

回答 3

回答 4

回答 5

问题：有效地对numpy数组进行降序排序？

回答 0

回答 1

回答 2

回答 3

回答 4

注意尺寸。

让

全反转

第一维反转

逆向二维

测试中

Be careful with dimensions.

Let

Full Reverse

First Dimension Reversed

Second Dimension Reversed

Testing

回答 5

回答 6

回答 7

回答 8

然后您的恢复：

Then your resault:

问题：Windows上的RuntimeError尝试Python多处理

回答 0

回答 1

回答 2

回答 3

回答 4

问题：在Python中，如何使用urllib查看网站是404还是200？

回答 0

回答 1

回答 2