Fun with Python Decorators on February 29

Happy leap year, everyone. Despite having been writing this blog for a decade now, it turns out I didn't think to write a post on February 29 either of the previous two leap years it's been around for. So I'm rectifying that oversight this year! Of course, there's a purpose to this post beyond merely taking the opportunity of having one up on February 29 (though that was a motivating factor). This week I wrote some rather interesting Python code, and thought I'd share it.

I was working on some code to generate synthetic data sets to test various line-fitting code, and wanted a way to add some random noise to the output, but potentially using different functions to generate the noise, which might take their own variable number of parameters. I turned to the concept of decorators in Python, which are, essentially, functions which operate on other functions. For a mathematical analogy, consider the following situation: \[f(x)=g(h(x)).\] Here we have a function, g, which operates on another function \(h(x)\), and we define this resulting function as \(f(x)\). In Python, the function g would be a decorator, as it takes another function and returns a modified form of it.

Of course, we can extend this idea further; what if, in addition to a function, g also takes additional parameters? Perhaps something like \[f(x)=g(h(x), a, b, c).\] It turns out we can do this in Python as well, though it's somewhat abstract and I don't fully understand it. I read some tutorials on the subject, I guessed at how to extend what they said into code which works, but I'd be lying if I said I truly understood it at this point. Though I'll still take a stab at explaining it. The process involves a triply-nested function to handle passing arbitrary functions and arguments to the decorator. I've embedded the entire decorator function below:

 def add_noise(noise_func, *noise_args, **noise_kwargs):  
   """Add noise from a given function to the output of the decorated function.  
   Parameters  
   ----------  
   noise_func : callable  
       A function to add noise to a data set. Should take as input a 1-D array  
       (and optionally additional parameters) and return the same.  
   args, kwargs  
       Additional arguments passed to this decorator will be passed on through  
       to `noise_func`.  
   Returns  
   -------  
   callable  
       A decorated function which adds noise from the given `noise_func` to  
       its own output.  
   """  
   def decorator_noise(func):  
       @functools.wraps(func)  
       def wrapper_noise(*args, **kwargs):  
           # Create the values from the function wrapped:  
           y = func(*args, **kwargs)  
           # Now generate noise to add to those values from the noise function  
           # provided to the decorator:  
           noise = noise_func(y, *noise_args, **noise_kwargs)  
           return y + noise  
       return wrapper_noise  
   return decorator_noise  

The main action happens in the third function where the decorated function is used to create a set of data points, y, the given noise function is used to create a variable noise, and then y and noise are added together to give the output result. You can then use this decorator on a function at definition time like so (assuming you already have a function called gaussian which takes a single parameter sigma and does the appropriate calculations:

 @add_noise(gaussian, sigma=10)  
 def generate_line_1d(x, m, b):  
   ... code ...  

This would mean any time you called the generate_line_1d function its output would be modified by the addition of Gaussian noise drawn from a distribution with a standard deviation of 10. If you instead wanted to define multiple instances of generate_line_1d with, say, different values of the sigma parameter, you could do the following:

 gaussian_10 = add_noise(gaussian, sigma=10)(generate_line_1d)  
 gaussian_20 = add_noise(gaussian, sigma=20)(generate_line_1d)  
 ...  

And so on and so forth. These various returned functions would be analogous to the \(f(x)\) defined above. You could also switch out the gaussian function for another function, which could itself take an arbitrary number of arguments.

Looking at it now, it feels less useful in the specific context I'm using it in than it seemed when I was writing it, but I'm still proud of it—it's basically more flexible and abstract than I really need, but it's still a pretty neat trick of abstraction. Come the middle of this year I'll have been using Python for a decade now, and I'm still learning new tricks and features. And hey, I might not necessarily need it now, but you never know when it might come in handy down the road! A hui hou!