The details mentioned below are just my thoughts on an approach towards software correctness. I consider myself not to be an expert in Python, Rust, software verification, or in general software development. Disclaimer: Please use your own judgement when designing/implementing software. That said, I am interested in practices to make software safer, and would highly appreciate any constructive criticism/feedback. What is TypeState Pattern? The basic idea behind TypeState pattern is to encode the state information inside types. While TypeState pattern is a generic pattern that could be used in multiple languages, I encountered this pattern first while developing software using . Rust There are multitude of resources (see below) that discuss TypeState pattern inside Rust in further detail, so I will not delve into details of TypeState pattern. References Example of TypeState pattern in Python Let's explore an example API: say we have a connection API over which we could establish connections and communicate with web servers using HTTP. The connection can exist in two states: the connection to remote end has not been established, and the connection to remote end has been established. There can be more than two states for the connection, but for this example, we will consider only these two states. One approach to represent this API using TypeState pattern would be as follows: socket typing Optional self._host = host self._port = port self._connected_socket: Optional[HTTPConnection._HTTPConnectionConnectedState] = self._socket = socket_ self._host = host request = ( ).format(url, self._host) self._socket.sendall(request.encode( )) result = : data = self._socket.recv( ) data: result += data : result s = socket.socket() s.connect((self._host, self._port)) self._connected_socket = self._HTTPConnectionConnectedState( s, self._host) self._connected_socket self._connected_socket: self._connected_socket._socket: self._connected_socket._socket.close() self._connected_socket = print(http_conn.get_request( )) # Approach 1 import from import : class HTTPConnection () """HTTP Connection without the connection established.""" -> : def __init__ (self, host: bytes, port: int) None """Initialize HTTP connection instance object with connection details.""" None : class _HTTPConnectionConnectedState () """HTTP connection with connection to remote end already established""" -> : def __init__ (self, socket_: socket.socket, host: bytes) None """Initialize HTTPConnectionConnectedState instance object.""" -> bytes: def get_request (self, url: bytes) """Send GET request to a URL over an HTTP connection.""" "GET {!r} HTTP/1.1\r\nHost: {!r}\r\nConnection: close\r\n" "\r\n" 'utf-8' b"" while True 4096 if else break return -> "HTTPConnection._HTTPConnectionConnectedState": def __enter__ (self) """Manage starting of an HTTP connection.""" return -> : def __exit__ (self, _exc_type, _exc_value, _traceback) None """Manage closing of an HTTP connection.""" if if # The following operation will not delete the instance object if the # user of this API stored reference to the instance object inside a # variable outside the scope of context manager None -> : def print_example_homepage ( http_conn: HTTPConnection._HTTPConnectionConnectedState ) None """Fetch and print response data over an HTTP connection.""" b'/' The following shows an approach to implement a similar logic without using TypeState pattern, by maintaining the state information inside an : attribute # Approach socket typing Optional = host self._port = port self._socket: Optional[socket.socket] = None self._is_connected = False def get_request(self, : bytes) -> bytes: not self._socket or not self._is_connected: raise Exception( ) request = ( .format(url, self._host)) self._socket.sendall(request.encode( )) result = b True: data = self._socket.recv( ) data: result += data : result def __enter__(self) -> : self._socket = socket.socket() self._socket.connect((self._host, self._port)) self._is_connected = True self def __exit__(self, _exc_type, _exc_value, _traceback) -> None: self._socket and self._is_connected: self._socket.close() self._socket = None self._is_connected = False def print_example_homepage(http_conn: HTTPConnection) -> None: print(http_conn.get_request(b )) 2 import from import (): """ .""" ( , : , : ) -> : """ .""" . class HTTPConnection HTTP connection to connect to a remote endpoint def __init__ self host bytes port int None Initialize the HTTP connection details self _host url "" "Send GET request to a URL over an HTTP connection." "" if "HTTP connection not established" "GET {!r} HTTP/1.1\r\nHost: {!r}\r\nConnection: close\r\n\r\n" 'utf-8' "" while 4096 if else break return 'HTTPConnection' "" "Manage starting of an HTTP connection." "" return "" "Manage closing of an HTTP connection." "" if "" "Fetch and print response data over an HTTP connection." "" '/' What benefit does Approach 1 provide over Approach 2? Let's consider two usage examples for the above-mentioned API: # Usage http_conn = HTTPConnection(b , ) print_example_homepage(http_conn) 1 "www.example.com" 80 # Usage HTTPConnection(b , ) http_conn: print_example_homepage(http_conn) 2 with "www.example.com" 80 as For this simple example, it is obvious that the second usage example is correct, but this might not be the case in large codebases. The following table shows the results for execution of both usage examples with each approach: | | Approach 1 | Approach 2 | |-------------|------------------------------------------------------------------------|--------------------------------------------| | Usage 1 | AttributeError: 'HTTPConnection' object has no attribute 'get_request' | Exception: HTTP connection not established | | Usage 2 | Prints the HTTP response as expected | Prints the HTTP response as expected | Both the approaches run into an exception when used incorrectly, and print the expected response when used correctly. So what advantage does the first approach provides over the second one? Even the exception message is more user-friendly in the second case. Let's see the advantage of using the first approach along with static type checking. Static type checking in Python Newer versions of Python have added support for providing inside code. These hints could be used by static type checking tools such as in order to provide guarantees of static typing (similar to that in compiled languages such as C, C++, Rust, etc.) along with the benefits of duck typing that Python provides. In addition, typing hints can be . type hints mypy added gradually inside the codebase While proper unit testing can help catch many issues with API usage, it can sometimes be hard to cover all the scenarios because of the dynamic nature of this approach. In comparison, static analyses are usually more conservative and tend to cover all flow scenarios. In my opinion, using the best of both worlds to establish correctness of code is the way to go forward (unless becomes feasible for generic programming, in which case I would prefer using formal verification procedures). Let's do static type checking using mypy on both the usage examples with each approach for API design: Formal Verification | | Approach 1 | Approach 2 | |---------|-------------------------------------------------------------------------------------------------------------------------|-----------------| | Usage 1 | Argument 1 to "print homepage" has incompatible type "HTTPConnection"; expected "_HTTPConnectionConnectedState" | No issues found | | Usage 2 | No issues found | No issues found | _example_ Conclusion As observed above, the approach of using TypeState pattern allows us to find a certain class of errors without even executing the code. NOTE that the implementation above does not prevent all the issues (such as while using the socket), since the remote end can close the connection anytime after the connection has been established. socket.error The only guarantee this approach tries to provide is that the user of an API is using it as intended by the API developer (assuming that the user is following other generic best practices, such as not using protected members directly), where the intention is encoded in the defined types and type signatures. In my opinion, the decision of whether to use the TypeState pattern or not depends on the particular use case. The example implementation using TypeState pattern discussed here has an extra cost of memory. Each state has been encoded using a Python , which requires memory to be allocated inside the Python process. class Using a single would take less memory than using a new to store the type information. (I am not a Python expert, so maybe there is another way to implement the TypeState pattern more efficiently). Python provides to optimize memory allocation for classes, which could help in the examples above. attribute class __slots__ Personally, I would consider not using TypeState pattern in Python when developing for platforms with tight memory constraints (that is if I have to develop in Python, otherwise I would preferably use a compiled language like C/C++/Rust). But if performance is not as major a concern as correctness of code, I would consider using TypeState patterns along with static type checking at least for the core APIs used in 80% of the codebase. References 1. TypeState pattern https://www.cliffle.com/blog/rust-typestate/ https://gist.github.com/barafael/0cbb0d89e7a9c0ada5744848cb3aa2fe https://stanford-cs242.github.io/f19/lectures/08-2-typestate Python context manager https://docs.python.org/3/library/contextlib.html https://docs.python.org/3/library/stdtypes.html#typecontextmanager https://docs.python.org/3/reference/datamodel.html#context-manager Python socket module https://docs.python.org/3/library/socket.html
