I have a Python codebase, built for Python 3, which uses Python 3 style open() with encoding parameter:

https://github.com/miohtama/vvv/blob/master/vvv/textlineplugin.py#L47

    with open(fname, "rt", encoding="utf-8") as f:

Now I’d like to backport this code to Python 2.x, so that I would have a codebase which works with Python 2 and Python 3.

What’s the recommended strategy to work around open() differences and lack of encoding parameter?

Could I have a Python 3 open() style file handler which streams bytestrings, so it would act like Python 2 open()?

1. To get an encoding parameter in Python 2:

If you only need to support Python 2.6 and 2.7 you can use io.open instead of open. io is the new io subsystem for Python 3, and it exists in Python 2,6 ans 2.7 as well. Please be aware that in Python 2.6 (as well as 3.0) it’s implemented purely in python and very slow, so if you need speed in reading files, it’s not a good option.

If you need speed, and you need to support Python 2.6 or earlier, you can use codecs.open instead. It also has an encoding parameter, and is quite similar to io.open except it handles line-endings differently.

2. To get a Python 3 open() style file handler which streams bytestrings:

open(filename, 'rb')

Note the ‘b’, meaning ‘binary’.

I think

from io import open

should do.

Here’s one way:

with open("filename.txt", "rb") as f:
    contents = f.read().decode("UTF-8")

This may do the trick:

import sys
if sys.version_info[0] > 2:
    # py3k
    pass
else:
    # py2
    import codecs
    import warnings
    def open(file, mode='r', buffering=-1, encoding=None,
             errors=None, newline=None, closefd=True, opener=None):
        if newline is not None:
            warnings.warn('newline is not supported in py2')
        if not closefd:
            warnings.warn('closefd is not supported in py2')
        if opener is not None:
            warnings.warn('opener is not supported in py2')
        return codecs.open(filename=file, mode=mode, encoding=encoding,
                    errors=errors, buffering=buffering)

Then you can keep you code in the python3 way.

Note that some APIs like newline, closefd, opener do not work

If you are using six, you can try this, by which utilizing the latest Python 3 API and can run in both Python 2/3:

import six

if six.PY2:
    # FileNotFoundError is only available since Python 3.3
    FileNotFoundError = IOError
    from io import open

fname = 'index.rst'
try:
    with open(fname, "rt", encoding="utf-8") as f:
        pass
        # do_something_with_f ...
except FileNotFoundError:
    print('Oops.')

And, Python 2 support abandon is just deleting everything related to six.

I’d like to point to a function that does nothing:

def identity(*args)
    return args

my use case is something like this

try:
    gettext.find(...)
    ...
    _ = gettext.gettext
else:
    _ = identity

Of course, I could use the identity defined above, but a built-in would certainly run faster (and avoid bugs introduced by my own).

Apparently, map and filter use None for the identity, but this is specific to their implementations.

>>> _=None
>>> _("hello")
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: 'NoneType' object is not callable

Doing some more research, there is none, a feature was asked in issue 1673203 And from Raymond Hettinger said there won’t be:

Better to let people write their own trivial pass-throughs and think about the signature and time costs.

So a better way to do it is actually (a lambda avoids naming the function):

_ = lambda *args: args

• advantage: takes any number of parameters
• disadvantage: the result is a boxed version of the parameters

OR

_ = lambda x: x

• advantage: doesn’t change the type of the parameter
• disadvantage: takes exactly 1 positional parameter

An identity function, as defined in https://en.wikipedia.org/wiki/Identity_function, takes a single argument and returns it unchanged:

def identity(x):
    return x

What you are asking for when you say you want the signature def identity(*args) is not strictly an identity function, as you want it to take multiple arguments. That’s fine, but then you hit a problem as Python functions don’t return multiple results, so you have to find a way of cramming all of those arguments into one return value.

The usual way of returning “multiple values” in Python is to return a tuple of the values – technically that’s one return value but it can be used in most contexts as if it were multiple values. But doing that here means you get

>>> def mv_identity(*args):
...     return args
...
>>> mv_identity(1,2,3)
(1, 2, 3)
>>> # So far, so good. But what happens now with single arguments?
>>> mv_identity(1)
(1,)

And fixing that problem quickly gives other issues, as the various answers here have shown.

So, in summary, there’s no identity function defined in Python because:

1. The formal definition (a single argument function) isn’t that useful, and is trivial to write.
2. Extending the definition to multiple arguments is not well-defined in general, and you’re far better off defining your own version that works the way you need it to for your particular situation.

For your precise case,

def dummy_gettext(message):
    return message

is almost certainly what you want – a function that has the same calling convention and return as gettext.gettext, which returns its argument unchanged, and is clearly named to describe what it does and where it’s intended to be used. I’d be pretty shocked if performance were a crucial consideration here.

yours will work fine. When the number of parameters is fix you can use an anonymous function like this:

lambda x: x

There is no a built-in identity function in Python. An imitation of the Haskell’s id function would be:

identity = lambda x, *args: (x,) + args if args else x

Example usage:

identity(1)
1
identity(1,2)
(1, 2)

Since identity does nothing except returning the given arguments, I do not think that it is slower than a native implementation would be.

No, there isn’t.

Note that your identity:

1. is equivalent to lambda *args: args

2. Will box its args – i.e.

   In [6]: id = lambda *args: args
   
   In [7]: id(3)
   Out[7]: (3,)
   

So, you may want to use lambda arg: arg if you want a true identity function.

NB: This example will shadow the built-in id function (which you will probably never use).

If the speed does not matter, this should handle all cases:

def identity(*args, **kwargs):
    if not args:
        if not kwargs:
            return None
        elif len(kwargs) == 1:
            return  next(iter(kwargs.values()))
        else:
            return (*kwargs.values(),)
    elif not kwargs:
        if len(args) == 1:
            return args[0]
        else:
            return args
    else:
        return (*args, *kwargs.values())

Examples of usage:

print(identity())
None
$identity(1)
1
$ identity(1, 2)
(1, 2)
$ identity(1, b=2)
(1, 2)
$ identity(a=1, b=2)
(1, 2)
$ identity(1, 2, c=3)
(1, 2, 3)

Stub of a single-argument function

gettext.gettext (the OP’s example use case) accepts a single argument, message. If one needs a stub for it, there’s no reason to return [message] instead of message (def identity(*args): return args). Thus both

_ = lambda message: message

def _(message):
    return message

fit perfectly.

…but a built-in would certainly run faster (and avoid bugs introduced by my own).

Bugs in such a trivial case are barely relevant. For an argument of predefined type, say str, we can use str() itself as an identity function (because of string interning it even retains object identity, see id note below) and compare its performance with the lambda solution:

$ python3 -m timeit -s "f = lambda m: m" "f('foo')"
10000000 loops, best of 3: 0.0852 usec per loop
$ python3 -m timeit "str('foo')"
10000000 loops, best of 3: 0.107 usec per loop

A micro-optimisation is possible. For example, the following Cython code:

test.pyx

cpdef str f(str message):
    return message

Then:

$ pip install runcython3
$ makecython3 test.pyx
$ python3 -m timeit -s "from test import f" "f('foo')"
10000000 loops, best of 3: 0.0317 usec per loop

Build-in object identity function

Don’t confuse an identity function with the id built-in function which returns the ‘identity’ of an object (meaning a unique identifier for that particular object rather than that object’s value, as compared with == operator), its memory address in CPython.

The thread is pretty old. But still wanted to post this.

It is possible to build an identity method for both arguments and objects. In the example below, ObjOut is an identity for ObjIn. All other examples above haven’t dealt with dict **kwargs.

class test(object):
    def __init__(self,*args,**kwargs):
        self.args = args
        self.kwargs = kwargs
    def identity (self):
        return self

objIn=test('arg-1','arg-2','arg-3','arg-n',key1=1,key2=2,key3=3,keyn='n')
objOut=objIn.identity()
print('args=',objOut.args,'kwargs=',objOut.kwargs)

#If you want just the arguments to be printed...
print(test('arg-1','arg-2','arg-3','arg-n',key1=1,key2=2,key3=3,keyn='n').identity().args)
print(test('arg-1','arg-2','arg-3','arg-n',key1=1,key2=2,key3=3,keyn='n').identity().kwargs)

$ py test.py
args= ('arg-1', 'arg-2', 'arg-3', 'arg-n') kwargs= {'key1': 1, 'keyn': 'n', 'key2': 2, 'key3': 3}
('arg-1', 'arg-2', 'arg-3', 'arg-n')
{'key1': 1, 'keyn': 'n', 'key2': 2, 'key3': 3}