Sometimes it is important to map function arguments with their parameter values. Python offers basic - which will be discussed below - but if arguments are passed either positionally, or are handled by "catch-alls", such as or , then it quickly can become more difficult to deterministically map parameter names to their associated values in a function call. keyword arguments *args **kwargs Some situations where this mapping may be desired include: - Seeing what functions were called with what method helps to determine how efficiently a program is running, or can help debug it when things go wrong. Observability Tools and Call Stack Tracing - If your system is a product used by organizations and users, it is important to have an easy, low-overhead way to determine what resources are being affected by a function call and who requested the change. Determining Function/task Ownership For Audit Purposes - Sometimes a function's arguments need to be validated or modified before being used, and creating a decorator that can handle any arbitrary parameter definitions may be useful. Argument Modification In this post, we will dive into some more magical Python libraries and utilities that allow us to wrap and perform certain operations against arbitrary function parameters. I will start with some Python basics - feel free to skip those sections as needed - before diving into some of the magic and its real-world use cases. Signature Functions Review Python, like most other programming languages, has the ability to define . These functions usually define input and output return values. values can then be provided to the function, which will perform some work and return a value. functions parameters Argument Note that I will try to use Python annotations where possible to make parameter types more clear. def add(num_1: int, num_2: int) -> int:
    """Adds and returns the two arguments
    """
    return num_1 + num_2

add(5, 7)  # returns 12 Parameters are the definitions of what variables a function defines, and arguments are the actual runtime values that satisfy those parameters. In the example above, and are the parameter definitions, and values and are the arguments passed to the function. Quick note: num_1 num_2 5 7 We can define arguments in Python Functions in two common ways: as or as . positional arguments keyword arguments Positional Arguments Positional arguments are defined at the beginning of the parameter list, they are matched with their parameters , and they have no default values. A positional argument be provided to the function when called. based on the order they are provided must In the example above, since the function defines , it means that the first argument passed ( ) will be saved to and the second argument passed ( ) will be saved to . num_1 and then num_2 5 num_1 7 num_2 Keyword Arguments Keyword arguments are defined after positional arguments, and they can provide default values in the parameter definition, meaning that a matching argument does not need to be provided when the function is called. These arguments are provided using the parameter , which is the variable name given to the parameter. If they are given positionally, without the identifier, then they act in the same way as positional arguments. identifier We could call using keyword arguments: add add(num_1=10, num_2=8)  # returns 18 Notice how we have . We are now using keyword arguments to set and . <identifier>=<value> num_1 num_2 Default Keyword Arguments doesn't require keyword arguments by default, so lets consider the following function: add def exponential(num, power=2):
    return num ** power The parameter above is defined as a keyword argument by default, where if it’s not provided, it will be given a default argument value of . power 2 exponential(2)           # returns 4
exponential(2, 3)        # returns 8
exponential(2, power=3)  # returns 8 as well Problem Statement Now that we know how arguments and parameters work in Python, what if we wanted to write a program that prints out what arguments are given to a function, and lines up positional and keyword arguments to a dictionary map. We will create a to accomplish this. decorator We are going to write the decorator function that will print out the arguments and return value of any function wrapped by the decorator. log_function_call @log_function_call
def pow(num, power=2):
    return num ** power

pow(5)
# -> "pow was called with { num: 5, power: 2 }"
# -> 25

pow(5, 3)
# -> "pow was called with { num: 5, power: 3 }"
# -> 125

pow(2, power=4)
# -> "pow was called with { num: 2, power: 4 }"
# -> 16 To create this, we will need to solve the problem of determining which arguments given line up with the parameter definition in the function, so that we can print it out. In a more general sense: How can we intercept and determine what values are passed to a Python callable? In this case, a is anything that can take arguments and return a value, such as a function, method, or even a function. callable lambda Python's Signature Class Python has a built-in module that provides many different utility functions and classes which allow the caller to get information and metadata about runtime variables during a program's execution. We can leverage this module and its class to introspect a function's arguments to create a mapping of parameter names back to their values, ! inspect Signature even if they are passed as position arguments Without the class, only can be easily converted into a map, which limits the amount of introspection that can be performed. By using the class, a generic decorator, such as , can convert arguments given to a function into a map of their parameter names and values. This is powerful as it allows us to create a more powerful decorator that can perform extra actions using a function's arguments, whether passed positionally or as keywords. Signature keyword arguments dict Signature log_function_call any Quick Look at Decorators Before diving into how the class can help us with the decorator, let’s start setting up the decorator itself so that we can understand code before going further. While I assume some basic knowledge of Python decorators, I will briefly explain what the code in the decorator block would look like. Signature log_function_call that def log_function_call(func)

   def wrapper(*args, **kwargs):
       """
	   Wrap the original function call to print the arguments before
	   calling the intended function
	   """
       print("TODO: Print the arguments here!")
       func(*args, **kwargs)

    return wrapper By wrapping a function with this above, we will emit a print line before the function actually gets called. The only issue is that we are printing an unhelpful line instead of our actual argument values. log_function_call Signatures and Binding So we have a decorator that is intercepting the function call with access to the arbitrary positional argument - as a list represented as - and keyword arguments - represented as the map. By generating a of the arbitrarily wrapped function - in the example - we can merge these two generic argument objects into a single, declarative map of *args **kwargs signature func <parameter name>: <arg value> which can then be acted upon easily to perform different operations, such as printing out the function call and arguments for auditing! In the following block of code, we are continuing on from the previous decorator block with the addition of our introspection code; we are generating a signature of the wrapped function and then the given arguments to the function’s parameters, which returns a dictionary of parameter names to their passed values. binding from inspect import signature


def log_function_call(func)

    def wrapper(*args, **kwargs):
        """
	    Wrap the original function call to print the arguments before
	    calling the intended function
	    """
	    func_sig = signature(func)
	   
        # Create the argument binding so we can determine what
		# parameters are given what values
		argument_binding = sig.bind(*args, **kwargs)
		argument_map = argument_binding.arguments
	   
	    # Perform the print so that it shows the function name
		# and arguments as a dictionary
        print(f"{func.__name__} was called with {argument_map}")
        func(*args, **kwargs)

    return wrapper Real-World Example So when can this actually be used? Well, for our project, we have a project that implements . We can schedule these tasks to be run by a background worker, but we require an audit log to determine started a task and the task is tied back to. Django web application Celery Asynchronous Tasks who what organization For instance, if we have a task such as that ensures the right members have the proper permissions, it may take in a parameter of the organization to resolve membership for. resolve_membership organization_id from celery import task


@task(bind=True)  # <-- Fanciness that just denotes the function is an async task
def resolve_membership(task: TaskResult, organization_id: int):
    org = Originization.objects.get(id=organization_id)
    do_something_with_the_org(org)  # arbituary code being run While this task is created, scheduled, started, and completed, its current state and result ( , , , ...) will be stored in the database by Celery automatically. This gives us a queriable audit log of all tasks that have been run, but it doesn't easily actually allow us to see which tasks correspond to which organizations. As a part of task scheduling, we wanted to create an audit log of these tasks and draw a relationship to what organizations they were affecting, and what - if any - user scheduled the tasks. SUCCESS FAILED QUEUED resolve_membership We could have performed this audit log creation explicitly in every task, like the following code example, but it would have gotten more difficult to manage as more tasks were created, or if any changes were made to the custom class. This would quickly become too cumbersome and would not result in a . TaskResult DRY program structure from celery import task


@task(bind=True)  # <-- Fanciness that just denotes the function as an async task
def resolve_membership(task: TaskResult, organization_id: int):
    org = Originization.objects.get(id=organization_id)

	# Assign the organization to the task for audit logging purposes
	#   NOTE: this would need to be done on EVERY task
	task.organization = org

	# Perform the actual task needed
    do_something_with_the_org(org)  # arbituary code being run Using a wrapping decorator like the above, we were able to create a single decorator that would wrap each task execution. This decorator would inspect the task arguments and provide some static arguments to the decorator itself, it would be able to create the same ownership relationship without needing to copy anything more than just the decorator name itself. This made it much easier to create the task audit log without having to worry about how the ownership was actually being created. It reduced all of our individual implementations across tasks to a single function block that was much easier to manage. log_function_call assign_ownership from celery import task

@task(bind=True)
@audit_ownership(org_param='organization_id')
def resolve_membership(task: TaskResult, organization_id: int):
    org = Originization.objects.get(id=organization_id)

	# Perform the actual task needed
    do_something_with_the_org(org)  # arbituary code being run The following code block is a paraphrased version of the decorator. Notice how we are using the class to load the value of a given parameter no matter how it was passed as a function; whether as a positional or keyword argument. Note that the following code is a bit advanced and does require some knowledge of how to define decorators in Python. Signature from inspect import signature


def audit_ownership(org_param: str, task_param: str ='task'):
    """
	Wrapping a celery task, this will attempt to create an audit log
	tying the task execution back to an organization using passed arguments
	
	: org_param:  str  : function parameter that denotes the organization ID
	: task_param: str  : function parameter that denotes the task instance
	"""
    
	def decorator(func):
	    """
		This is the actual task decorator, but nested to allow for parameters
		to be passed on the decorator definition
		"""
	
	    def wrapper(*args, **kwargs):
		    """
			The wrapper replaces the actual function call and performs the
			needed extra auditing work before calling the original function
			"""
		
		    # create a function signature to introspect the call
		    sig = signature(func)     
			# Create the argument binding so we can determine what
			# parameters are given what values
			argument_binding = sig.bind(*args, **kwargs)
			argument_map = argument_binding.arguments
			
			# Using the argument binding and decorator arguments, we can
			# fetch the task and organization, no matter how the function
			# was invoked. The power of argument introspection!!
			task = argument_map[task_param]
			organization_id = argument_map[org_param]
			organization = Organization.objects.get(id=organization.id)
		
			# The actual logic is abstracted out to be more readable
			# Assume this function creates the relationship between the
			# task result instance and the organization instance
		    assign_ownership_to_task(task, organization)
			
		    func(*args, **kwargs)            # this calls the original function
			                                 # as it was original intended	
	    return wrapper As an extra bonus, we made the decorator parameter also be a , instead of being a , so that we could dynamically fetch the associated value. This was useful in situations where organization-owned resources were passed to the task instead of the organization itself, so we could have the callable load the child object and return a reference to its parent's organization. org_param callable str Python decorators that modify and perform operations using the provided arguments can rely on function signatures and argument bindings to determine parameter values as a part of their operations. This allows programs to utilize generalized decorators that can be reused across an application to perform operations such as debugging or audit-logging. First Published here

Fetch

Switching Jobs in Tech Industry: 6 Key Lessons I've Learned

Follow me on dev.to

Argument Binding and Introspection In Python

About Author

Comments

TOPICS

THIS ARTICLE WAS FEATURED IN

Related Stories

Implement a Dynamic Breadcrumb in React/NextJS

Asynchronous Tasks with Celery + Redis in Django

Creating a Distributed Task Queue in Python With Celery + RabbitMQ + Redis

Django, Celery, Redis and Flower Implementation

How to schedule ‘the Boring Stuff’ with Django and Celery Beat

Scaling Effectively: when Kubernetes met Celery

Implement a Dynamic Breadcrumb in React/NextJS

Asynchronous Tasks with Celery + Redis in Django

Creating a Distributed Task Queue in Python With Celery + RabbitMQ + Redis

Django, Celery, Redis and Flower Implementation

How to schedule ‘the Boring Stuff’ with Django and Celery Beat

Scaling Effectively: when Kubernetes met Celery

Light-Mode

Classic

Newspaper

Dark-Mode

Neon Noir

Minty

HN StartUps