Clean Architecture in Python
Principle of Inversion in Python
Breaking down the Dependency Inversion Principle.
The Dependency Inversion Principle is often explained in the OOP paradigm. I want to share another angle on this concept. In this short essay, I will walk through a path from abstraction to understanding what is actually inverted.
Introduction
Since the first programming languages had appeared, their usage had been growing continuously. Programs were becoming more large and complicated. Software development turned to industry. Even pioneers of programming began to consider a size of the programs as a difficulty.
My basic problem is that precisely this difference in scale is one of the major sources of our difficulties in programming!
- E. W. Dijkstra. Notes on structured programming. 1970
Along with the size, the cost of software also continued to grow up. Developing software became a long-running process. Once developed, software continues to evolve with time and requirements. It turns out that the main cost of software can be not in its initial development but in maintenance.
The engineers and scientists continued to develop theories and approaches to handle cost and complexity.
Simplicity is the primary measurement recommended for evaluating alternative designs relative to reduced debugging and modification time.
- W.P. Steven, G.J. Myers, L.L. Constantine. Structured Design. 1974
There are plenty of programming paradigms have appeared in the 1960s and 1970s. And the dominating models have become Structured Design and Object-oriented programming. Break down the program on modules and objects which communicate with each other that a brilliant idea!
But how exactly a developer should organize parts of the system to achieve good qualities. What should be the rules to follow?
In 1994 Robert Martin published a paper where he researched metrics for measuring the quality of object-oriented design.
This paper presents the case that simply using objects to model an application is insufficient to gain robust, maintainable and reusable designs.
- R. Martin. OO Design Quality Metrics. 1994
There is much more important how parts of the systems organized and communicate. Interdependence of the subsystems within a design makes it rigid and fragile.
In the next paper, R. Martin formulated the principle.
Principle
- High-level modules should not depend on low-level modules. Both should depend on abstractions (e.g., interfaces).
- Abstractions should not depend on details. Details (concrete implementations) should depend on abstractions. - R. Martin. The Dependency Inversion Principle. 1996
Showcase
For example, let’s come up with a simple use case. We have a script that receives some data for input, processes it, and prints a result.
For simplicity, I write it down in the following way.
if __name__ == "__main__":
data = [1, 2, 3]
result = [each * each for each in data]
print(results)
For our experiment, we will think abstract how data is received and how we process it. I wrote it as data = [1, 2, 3], but instead could write requests.get(“https://jsonplaceholder.typicode.com/users”) or something more complicated. The same is with the result = [each * each for each in data]. These details we don’t care about.
There we introduce abstractions.
Abstraction
Thinking in abstractions is considered to be one of the key attributes of the human mind. This ability is likely to have been closely connected with the development of human language.
In software engineering and computer science, abstraction is the process of removing temporal details to focus attention on details of greater importance.
The technical form of abstraction can be different, to name a few:
- the usage of data types;
- the concept of procedures and functions;
- the process of reorganizing common behavior from non-abstract classes into “abstract classes”.
We can rewrite code using functions as abstractions.
def fetch():
return [1, 2, 3]
def process(data):
return [each * each for each in data]
def run():
data = fetch()
return process(data)
if __name__ == "__main__":
result = run()
print(result)
First what we can notice that the code is getting bigger. It is a cost for using abstractions.
But we added structure to our code. We separated the use case to the run() function, and now it is reading like a story. It is our high-level policy.
Fetch data and process data.
We hid low-level details behind abstractions fetch() and process().
As we can see, our high-level module depends on abstractions in our case, they are functions. But our low-level abstractions still depend on details.
Our dependencies do not feel right.
Dependency
When we introduce a new component, it does not exist in a vacuum. It communicates and connects with other components. There is a special term for such connection — a coupling.
Not all dependencies are created equal. The degree of coupling can be different.
High coupling — component uses internal details from another component.
Low coupling — component relies on the external name or interface of another component.
External and internal relations are closely associated with the name scopes.
Back to our example, the names fetch() and process() are the global names and are used directly in the internal namespace of the run() function. This usage creates a rigid connection between components.
If we want low coupling, we can add arguments to the run() function and use these names as references to the original functions.
def fetch():
return [1, 2, 3]
def process(data):
return [each * each for each in data]
def run(fetch, process):
data = fetch()
return process(data)
if __name__ == "__main__":
result = run(fetch, process)
print(result)
With this simple change, we make our run() more flexible and testable. We can use other functions with similar interfaces. We can use mocks for testing or implement other fetching or processing procedures.
What are exactly these functions’ interfaces?
Polymorphism
The arguments, fetch and process, are not the same things as the fetch() and process() functions. They are just the names for something that is expected to have certain behavior or interface that can be used in the run() function.
On the language level, this possibility is known as polymorphism.
Now our dependency would be not a function, but a polymorphic interface. As Python has a duck typing polymorphism this interface is defined implicitly.
We could use type hints to describe expected interfaces. It could be something like this.
def script(
fetch: Callable[[], Iterable], process: Callable[[Iterable], Iterable]
) -> Iterable:
data = fetch()
return process(data)
And now we shifted our dependencies from functions to polymorphic interfaces.
That is where inversion happens.
More examples of polymorphism in Python I described in previous story.
Inversion
We will draw diagrams to show what is exactly being inverted.
First, a schema for the “Functions as Abstractions” variant. Where high-level function run() has low-level functions as dependencies. And we are considering functions as abstractions.
But the coupling between abstractions was too rigid and our low-level abstractions have been dependent on details.
We added arguments to the run() function. By doing this, we introduced the new abstraction as the polymorphic interface.
Solid lines show the runtime flow when code is executed. Dashed lines are source code flow.
The runtime doesn’t care about all these inversions and abstractions. They are helpful only for a developer. It is becoming easier to understand design and easier to maintain.
We can see the inversion as the difference in direction between low-level function and interfaces that this function implements.
Some more examples of layering with DIP can be found in my GitHub repo py-dependency-inversion-study.
Final thoughts
As being a highly dynamic language Python has all possibilities to create expressive and well-speaking applications.
The usage of the DIP is the fundamental mechanism for the creation of clean architecture within a system. Its proper application will help to make a well-layered design with separated business logic from the other details.
More reading
If you like this article you can be interested in the following.
Thank you for reading! Share you thoughts with me on LinkedIn and Twitter.