Lambda Expressions

Aug 22, 2018

Python’s Lambda Expressions, Map and Filter

enter image description here

Python’s Lambda Expressions, using the lambda operator, are quite controversial. Some call them clunky, others call them elegant.

As a person who likes the functional programming paradigm, I like to sprinkle my code with lambda expressions from time to time, as long as it’s in small, contained doses. However I do see how putting a kinda long keyword in the middle of a line, and defining a function anonymously under the assumption that it will not be used in the rest of the program, may end up being a problem.

But without getting ahead of ourselves, let us start by defining the relevant terms here:

Lambdas are Anonymous. They never forgive, they never forget. Expect them. Source: Pixabay

What are Lambda Expressions?

A lambda expression is an anonymous, inline declaration of a function, usually passed as an argument. It can do anything a regular function can, except it can’t be called outside of the line where it was defined, since it is anonymous: it has no name.

 regular, imperative declaration.
def twice(x):
  return 2*x
twice(5) # 10

# declaration assigning a lambda 
#(bad practice, don't try this at home!)
twice = lambda x: 2*x
twice(x) #10

#inline declaration: what is sanity?
(lambda x: 2*x )(5) # 10. Really.

The snippet above shows a comparison between using a function defined regularly (or imperatively, for those functional people in a corner who feel discriminated by my imperative-privileged talk), one defined by a lambda and assigned to a variable (which can be done, but is usually seen as a bad practice) and an inline call to a lambda function (I don’t even need to tell you why this shouldn’t be done too much).

Those three pieces of code end up returning the same value, but each of them has different effects in scope and memory: two define a callable object for the rest of the program, while the last one creates an anonymous function that won’t be called in any other line.

In case you are not fluent in List Comprehensions yet, here’s a regular, imperative, for-loop example:

def f(x):
  return 2*x
a = list( map(f,[1,2,3]) )

b = []
for i in [1,2,3]:
  b.append(f(i))

a == b #True

You will probably agree that map is less clumsy and takes less space than regular for-loops, but can be a bit less intuitive than List Comprehensions.

We’ve talked enough about the criticisms (harder to read code, a clumsy keyword), let’s talk about the practical side.

Map and filter: making use of lambda Expressions

Let us introduce two more of Python’s native keywords: map and filter.

The map function, for those unfamiliar with functional programming, is a higher-order function — that is, a function that takes at least one function as an input, or produces one as its output.

In its formal definition, its parameters are a function, and a sequence (any iterable object), and its output is another sequence, containing the result of applying the given function to each element in the supplied sequence, in the same order.

Sound familiar? That’s because calling a map of a function over a list is almost equivalent to using a list comprehension! There are some caveats to that statement though, and I will address them promptly.

The following equivalence almost holds true:

def triple(a):
  return 3*a

thrice = lambda x: 3*x

these = [triple(i) for i in range(5) ]
are = [(lambda x: 3*x)(i) for i in range(5) ]
all = [thrice(i) for i in range(5) ]
the = map(thrice, range(5))
same = map(triple, range(5))

Note that in the last two lines, I passed the triple and thrice functions as arguments, thanks to map being a higher order function.

If you were to iterate each of those five variables, they would all yield the same values. However in Python 3+, on printing them, you’d see some are lists, and others are map objects. That’s right, applying map on a list will return a Generator! That basically means it will generate a sequence that’s lazily evaluated, can be iterated on and must be cast into a list in order to be sliced or indexed. On the other hand, map returns a normal list in Python 2.7.

So that’s where lambdas and maps make a sort of synergy: As fluid as writing a line using map can be, it can become even more fluid if you can invent your small function on the fly. Once you get the hang of it, you’ll start thinking in terms of mapping, and appreciate this feature. But beware, unreadable code can fester very quickly if maps are left unguarded.

For completion, I will also introduce you to filter. Calling filter in a sequence is the same as adding an if at the end of a List Comprehension, except it leaves a functional aftertaste. The following snippets are equivalent:

some_list = list(range(5))

only_evens = filter(lambda x: x%2 == 0, some_list)
only_evens_list = [x for x in some_list if x%2==0]

# list(only_evens) == only_evens_list

As with map, filter returns a Generator (in this case a filter object), but casting it to a list reveals it is equivalent to using an if in a List Comprehension.

Is map faster than list comprehensions? What about filter?

I ran a few benchmarks comparing normal for-loops, List Comprehensions and map. I wasn’t sure what the results would be, but have a few theories about them.

Here are a few things I defined before running the experiment:

import time 

BIG = 20000000

def f(k):
    return 2*k

def benchmark(function, function_name):
    start = time.time()
    function()
    end = time.time()
    print("{0} seconds for {1}".format((end - start), function_name))

And this is the code for the benchmarks I ran:

def list_a():
    list_a = []
    for i in range(BIG):
        list_a.append(f(i))
        
def list_b():
    list_b = [f(i) for i in range(BIG)]

def list_c():
    list_c = map(f, range(BIG))

benchmark(list_a, "list a")
benchmark(list_b, "list b")
benchmark(list_c, "list c")

I compared a regular, imperative list declaration using append calls, one using List Comprehensions, and a last one calling map.

In Python 2.7, my results were the following:

• 6.00 seconds for list a

• 4.12 seconds for list b

• 3.53 seconds for list c (the one using map)

I actually don’t know or have any theories on what optimization map uses in Python 2.7, but it came out faster than a List Comprehension! So there’s a trade off to be made when choosing between the clarity of the latter and the speed of the former. It should be noted that I ran this experiment many times and had consistent results.

In Python 3, I had surprising results: the map test ran in less than a millisecond, while the list comprehension one took 5 seconds! That’s when I remembered to cast the result to a list to make the playground fair, as the Generator will of course load a lot faster, thanks to not having to initialize its values. These are the results:

• 7.08 seconds for list a

• 5.1 seconds for list b

• 5.1 seconds for list c

I had to run this one many times to check, but they basically take the same time. It’s apparent they’re both doing very similar things under the hood, and the choice between them then becomes only one of clarity, and whether laziness will be useful.

So to sum up, lambdas can be clunky, but they also add a lot of expression power to your code. Filter and map can be elegant to some, but don’t add a lot to the table in terms of performance (and are seen as less Pythonic than List Comprehensions by some people).

My personal opinion is I like that Python has these features, because they make some lines of code more beautiful, but generally don’t like them as much as List Comprehensions. Without tail call elimination and other optimizations, it can be a bit challenging to cite good reasons to use functional programming in Python (whereas we can actually gain a lot from using it in, say, Haskell) other than style and laziness (the good kind!).

What is your take? Is there some other functional topic you’d like me to discuss? Are you angry about me not mentioning reduce or fold? What other functional things would you like to see brought over to Python? Which ones do you dislike? I am sure you have an opinion on this, and I would love to learn more about it. I’d also love to know if there are non-trivial cases where map is actually faster than List Comprehensions, and I am just not seeing them.

That was my introduction to the functional programming features in Python, and I hope you found it useful or at least had some fun reading it.

For more Python tutorials, tips and tricks, try visiting my Data Science website!

If you want to become a Data Scientist, check out my recommended Machine Learning books.

Reference : towardsdatascience.com