CTOs, Here's How You Can Build The “Ship Of Theseus”


Ship of Theseus, alludes to a thought experiment that questions the state of a object in use, if all the components were to be changed completely while in use. Is the object the same or has it gained a new identity? Closer home, a relatable allegory is from Bollywood potboilers of yore, where the main protagonist grows from a toddler to an angry young man, all while running to escape the cops.

CTOs, Application architects and engineers in a young scaling technology start-up face a similar task, although not so much as the “what” conundrum, but more about “how” - how to replace all the wood of the proverbial ship while it’s still sailing.

So you are a young tech setup, have made a few technology choices. Perhaps yours is a consumer internet company, has a web-based ordering engine at its heart, a slick UI / Mobile App, a bunch of databases for your structured and unstructured data storage needs and some caching/queuing/scheduling infrastructure thrown in to speed up things. You could also be an enterprise class technology provider with a SAS offering with a similar technology stack.

Now you enter the cycle of change management and doing that at a rapid pace while keeping the plant running. What could you do upfront so that each change or modification you attempt, small or large, doesn’t amount to changing the very foundations of the architecture? Listed below are a few tools, a few mental constructs and markers, that may help you think and incorporate such elements into your architecture early on.

1. Trust the cloud

While this is more about system architecture than application, it would be extremely difficult to think of experimenting with application components, if you have to worry about hosting, database management, data backups, monitoring, logging, data sharing etc. In fact, horizontal scaling that cloud hosting platforms provide form critical base for the application architecture. So think of platforms like AWS and hosted services and DevOps tools as early in the cycle as you can.

2. User is difficult

It isn’t difficult to agree with this. The user of your app/application is the most difficult to please and least under your control. For example, if your interface is an app, it would be incorrect to assume high speed data connection, user keeping up with app updates or proactively giving you feedback. She is also the most demanding. Your tech choices should adjust for the same. Rigidity in that would affect your ability to make frequent changes. New technologies like React Native framework and progressive web apps web views in native containers can provide you that flexibility.

3. Build flexible joints

Build flexible joints – pipes can be replaced. While implementing workflows, think of implementing rules to stitch events and actions together. Many rule engines, like Drools-JBoss, help create these decision making nodes, which can them be used to create design patterns that can change over a period of time. You can move from a hub-spoke model to a more supply-chain model, if your deployment/system architecture supports the same.

4. Always design asynchronous

Codify this as a design principle. You should always tilt in favour of dispatch/call-back architecture. This gives you positive workflow gaps, that you can then exploit when needed, of course subject to behavior constraints. Sometime though, particularly when there are external interfaces, you may have no choice.

5. Give relationships a try

In modern software architectures, we have moved away from the primarity of objects and entity relationship modelling. But that doesn’t reduce the importance of carefully considering the same. UX and workflow determine the direction the overall architecture takes, but it should still be rooted in a sound and flexible data/object modelling. With No-SQL and big data modelling, we now have tools to not give “junk-drawer” treatment to datasets difficult to model in a conventional relationship model.

So codify this one too. After each workflow is spec’d out, spend time in getting data modelling and relationships defined. Keep them flexible, but do not sacrifice cogency. Lest you will end up with an unusable model soon enough and that is exactly what this article is trying to prevent in the first place.

6. State machine ‘em

Where there are “entities”, there are “states”. It would help to view your pipes as state machines that affect the workflow by effecting state changes. This is particularly true for transactional systems like payment workflows or order management systems. State engines are inherently flexible as in, you expand of the meaningful states that entities can be transitioned.

7. You should sniff, but stop others

Framework to gather data should be one of the initial tenets of design. Deciding on the right tools (analytics frameworks like Mix Panel, GA, CleverTap etc.) and data capture methodology isn’t straightforward. But your ability to “change the wood” will be severely affected if you do not know which “one” to change.

Likewise, information security and data auditing has to think through initially. This is more of “difficult to do later” rather than “will increase flexibility”, but important nonetheless.

There certainly are more ways of dissecting this and more tools that can be added to the list, but this should be a good initial set. It still doesn’t answer the paradox though. If you succeed, is that the same ship you’ve been sailing?

Good luck Sailor.