{"id":1342,"date":"2016-06-28T00:00:00","date_gmt":"2016-06-28T00:00:00","guid":{"rendered":"http:\/\/www-staging.carta.com\/sg\/blog\/supercharging-django-productivity\/"},"modified":"2021-03-05T06:58:52","modified_gmt":"2021-03-05T06:58:52","slug":"supercharging-django-productivity","status":"publish","type":"post","link":"https:\/\/www-staging.carta.com\/sg\/blog\/supercharging-django-productivity\/","title":{"rendered":"Supercharging Django Productivity"},"content":{"rendered":"\n<p>A good framework can be measured by what functionality you get out of the box and by how much friction it removes when customizing it. <mark><a href=\"https:\/\/www-staging.carta.com\/sg\/\" target=\"_blank\" rel=\"noopener noreferrer\">Carta<\/a> is the source of truth for financial agreements backing billions of dollars. Code clarity, consistency, and safety are critical. For us, Django is more than boilerplates of models, views, and templates; it is a cohesive easy-to-extend abstraction of how data interacts with the world. <\/mark>Using Python principles and inheriting Django classes, we create sophisticated services with minimal code.<\/p>\n\n\n\n<h3 id=\"anappsfirststeps\">An App\u2019s First Steps<\/h3>\n\n\n\n<p>All of our models inherit from one source. We are able to centralize global functionality in one place. Making an application inherit from one model is powerful. The same is true for views. When you first start a project, the purpose of these classes might seem unclear. If you build your project from the beginning with the methodology that it will become a big app, it makes scaling your code much easier when the time comes.<\/p>\n\n\n\n<script src=\"https:\/\/gist.github.com\/kyle-eshares\/b99455dc27abe5a350a6f94f372eeb30.js\"><\/script>\n\n\n\n<p>Simple base classes should not be underestimated. While they don\u2019t enable functionality, they give you the mindset from the start that your models and views are really <em>your<\/em> models and views. Managing site-wide behavior becomes trivial.<\/p>\n\n\n\n<h3 id=\"logicaldeletion\">Logical Deletion<\/h3>\n\n\n\n<p>One of the features we wanted from the beginning was a default policy against deleting anything from the database. It helps with reversing mistakes and separates active data from stuff we want to keep but don\u2019t want to see. The challenge in such a system is how to reliably keep all the objects that were soft-deleted out of our queries.<\/p>\n\n\n\n<p>A boolean <code>is_void<\/code> flag on our base model makes the magic happen. All of our querysets exclude this flag by default, but they can be accessed by using <code>Model.objects.all_objects_including_void()<\/code> (a purposefully long name). On the model and queryset we override the <code>delete()<\/code> method to instead set the <code>is_void<\/code> flag to true and void any dependents. Hard deleting behavior lives at <code>purge_from_db()<\/code>, making it obvious to developers what is happening.<\/p>\n\n\n\n<script src=\"https:\/\/gist.github.com\/kyle-eshares\/726d001cd33ced51fb86e1612f1ff72d.js\"><\/script>\n\n\n\n<p>We only need logical deletion on a few tables, but inheritance makes implementing it for one model the same amount of effort as standardizing it across the application. Storage is cheap. It\u2019s better to keep it and never need it than to ditch it and regret it.<\/p>\n\n\n\n<p>A more comprehensive implementation with cascading delete and exceptions and can be found on <a href=\"https:\/\/gist.github.com\/kyle-eshares\/fbecc71022b62515a484a4a33d13014d\" target=\"_blank\" rel=\"noopener\">github<\/a>.<\/p>\n\n\n\n<h3 id=\"viewpermissions\">View Permissions<\/h3>\n\n\n\n<p>Inheriting from one view, we enable uniform site wide permissions. A common class creates composable permission sets. Our permissions are specialized around what legal entity the page is showing. They inspect the URL and POST body parameters to ensure all object parameters are consistent with the legal entity and if the user has access to those items. Broadly, a permission check is anything that returns a boolean.<\/p>\n\n\n\n<script src=\"https:\/\/gist.github.com\/kyle-eshares\/3175313d423fd6066e809b705e4c8c93.js\"><\/script>\n\n\n\n<p>The <code>SuperUserGuard<\/code> permission makes our views secure by default. Any more relaxed permissions must be set explicitly. It gives us a safety net so we don\u2019t accidentally publish a view.<\/p>\n\n\n\n<h3 id=\"eventaudit\">Event Audit<\/h3>\n\n\n\n<p>A historical record needs to exist over the lifetime of a financial security. We need to track every page view, every change to a table, and every action on the site.<\/p>\n\n\n\n<p>Django opens the possibility to hook into every part of an application by extending a few base classes. Hooking into models and views creates a unified system for linking the contexts of web requests with actions.<\/p>\n\n\n\n<p>The event system is implemented using a dedicated insert-only single-table Postgres instance. The table contains fixed columns (such as username, user id, url path, time, browser, etc.) and a JSON field for arbitrary context data. By having a central event stream, we can create arbitrary event types and then have celery post-process the events. We use this to post messages to slack, send emails, interact with analytics services, and more.<\/p>\n\n\n\n<p>To track the model instance changes, we extended the queryset\u2019s <code>update()<\/code>, <code>delete()<\/code>, <code>create()<\/code> methods as well as the model\u2019s <code>save()<\/code> method. Anytime fields are changed through <code>update()<\/code> or <code>save()<\/code>, we generate a diff of what has changed and push it into the event database. Creation and deletion are also tracked. While it doesn\u2019t track changes made directly in psql, it can alert us to data inconsistencies and point us to look at other logs.<\/p>\n\n\n\n<h3 id=\"conclusions\">Conclusions<\/h3>\n\n\n\n<p>Django\u2019s virtue is how easy it is to quickly implement complex behavior. All of the above projects were originally done in one day and have stood up over the years with minimal maintenance. Many more projects like querysets that recreate past history, object-oriented caching, and customer centric labeling add critical functionality without much maintenance or upfront development. A good chunk of the productivity can be attributed to Python itself.<\/p>\n\n\n\n<p>This was important for a start-up with limited resources that needed to maintain a secure financial application. The low cost of implementation meant we were able to include these features from the beginning.<\/p>\n\n\n\n<p>These design patterns also solidified a mentality within our team that there is an internal standard of handling models and views. We can extend our base classes to tackle any business requirement. Our codebase is constantly evolving, both in new features and how we model the world. Django has proven to be a valuable tool that lets us aim big and iterate often.<\/p>\n\n\n\n<p><small><em>Thanks to Eric Hurkman, Alexandre Laplante, and Brett Hardin<\/em>.<\/small><\/p>\n","protected":false},"excerpt":{"rendered":"<p>For us, Django is more than boilerplates of models, views, and templates; it is a cohesive easy-to-extend abstraction of how data interacts with the world.<\/p>\n","protected":false},"author":15,"featured_media":2948,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":[],"categories":[14,15],"tags":[],"acf":[],"_links":{"self":[{"href":"https:\/\/www-staging.carta.com\/sg\/wp-json\/wp\/v2\/posts\/1342"}],"collection":[{"href":"https:\/\/www-staging.carta.com\/sg\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www-staging.carta.com\/sg\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www-staging.carta.com\/sg\/wp-json\/wp\/v2\/users\/15"}],"replies":[{"embeddable":true,"href":"https:\/\/www-staging.carta.com\/sg\/wp-json\/wp\/v2\/comments?post=1342"}],"version-history":[{"count":0,"href":"https:\/\/www-staging.carta.com\/sg\/wp-json\/wp\/v2\/posts\/1342\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www-staging.carta.com\/sg\/wp-json\/wp\/v2\/media\/2948"}],"wp:attachment":[{"href":"https:\/\/www-staging.carta.com\/sg\/wp-json\/wp\/v2\/media?parent=1342"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www-staging.carta.com\/sg\/wp-json\/wp\/v2\/categories?post=1342"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www-staging.carta.com\/sg\/wp-json\/wp\/v2\/tags?post=1342"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}