Brian Richardson's Blog

I think I can speak for many developers when I say that there comes a point in every project when things just seem to get stuck. And from that day forward, you never look at the project the same way again. So you move on, hoping that you won’t get stuck again. And, perhaps you won’t, but why keep moving around only to get disappointed again? I’d propose an alternative to moving on: making development fun again.

Development is a job that is really dependent on the work itself for fulfillment. Working on a great project with great people is often worth more than making more money doing something boring or tedious; or worse, downright painful. If you find yourself in the latter category, it might be time to look for bigger picture solutions than keeping running into the same issues over and over. So, as a developer, sometimes the best thing you can do for yourself is find a different perspective. To that end, starting or joining an architecture practice at your organization can provide you both with new opportunities, and the chance to really “build it right”.

I started in architecture with a value proposition to my CIO. I didn’t claim that I was going to save tons of money, or that I was going to revolutionize anything. But I did have solutions to problems the organization was currently facing. It was his buy-in that made everything possible. Executive support is a critical step in establishing the architecture practice, so take your time with your value proposition. Really think about the challenges your organization faces, and how you can provide not only technological solutions, but business solutions as well. IT Architecture is about bridging the gap between business and IT – to understand business problems in technical terms, and to present technical solutions in business terms.

The next step I took was to build a roadmap. In the case where you are just establishing an architecture practice, there’s probably a lot of chaos. It’s important to have a vision of where you want to get to. Research is important here. Finding out about best practices, modern designs, and new techniques is the bread and butter of being an architect. Once you have an idea, however nebulous it might be right now, it’s time to figure out how to get there. Current State – Future State – Gap Analysis. That’s the essence of a roadmap. Here’s where you communicate your plan to the executive and present clear steps and timelines to achieve goals. Don’t worry about solving everything at once – establish some key principles and focus on those. Architecture is an iterative process, and you’ll hopefully get a chance to build another roadmap.

5 years ago I wrote my first roadmap. 6 months ago I wrote my second. Now I am in charge of making it happen. It is far better for me to have taken a different path, and it has been far better for the organization. This all started because I thought: “there must be a better way”. If you think there’s a better way, don’t be afraid to show its value and find a champion for your ideas. You just might get to implement them.

Frogwares brings back their successful line of Sherlock Holmes games as an open world adventure. Many familiar gameplay elements are combined into an enjoyable and immersive adventure. Cases abound, from the very small, helping the inept local police with some crime scene reconstructions – to the very large: how did Violet Holmes die?

Holmes arrives on the small island of Cordona seeking closure as he visits his mother’s grave. This is as early as we’ve ever seen Holmes in an adventure, coming well before Holmes meets Watson. But, Sherlock’s childhood friend Jon is there to fill the void. Mystery never follows far behind, and Holmes finds that closure is not so simple.

Jon will put his own comments about your performance into his diary for you to peruse. Acting like a dullard or behaving in unsavory ways is sure to evoke something pithy. But, he is equally generous with his praise when you dazzle as Holmes is known to.

Your mind palace will still allow you to combine clues to form deductions and find new avenues of investigation. As with the previous incarnation of the mind palace, some deductions can have multiple implications, and choosing the wrong one will lead to the wrong conclusion.

I am fond of a map function that requires you to actually read the map and the clues. While most games hold your hand and give you modern conveniences such as mini-maps, pathfinding and immediate knowledge of all locations, this game will have you read the clues and scour the map for the correct location. We see the same sort of map function in The Sinking City as well, also by Frogwares. I think it helps add to the immersiveness.

Your casebook is very well-organized. Holmes is nothing if not methodical, and you will find all the evidence laid out neatly, with icons suggesting the necessary actions to be taken. Actions include: crime scene reconstructions, asking people about specific evidence, disguising yourself, following footprints and other traces. While the game guides you toward new clues, there is still plenty of opportunity to use your brain and come to satifying conclusions worth of Holmes himself.

Holmes needs to learn how to defend himself in this episode. While armed with a revolver and a box of pepper snuff, Holmes still has a revulsion for murder, and combat is geared toward arresting and subduing your opponents, not simply killing them. Frogwares helpfully provides you with a number of bandit lairs to hone your combat skills. I probably spent 2-3 hours in one of these lairs alone (and did not complete it, alas). My combat skills improved significantly though by pursuing this side activity.

Overall, this episode presents very well as an open world adventure, giving a lot of freedom to pursue your own agenda, but having a number of cases and adventures to pursue at your leisure. The stories of Holmes’ early life are enjoyable, and a plausible picture of Holmes’ youth emerges. If you’ve enjoyed Frogwares’ previous episodes, you will love this one. It boasts excellent writing, thoughtful puzzles and a beautiful world to explore through the eyes of the ever-curious Sherlock Holmes.

I’ve spent the last few months studying Alexey Zimarev’s excellent book, Hands-On Domain Driven Design with ASP.NET Core (2017). While 5 years old now, it was clearly cutting edge at the time, since the Event Sourced system he demonstrates is now becoming more popular. I’ve found several reliable sources who refer to Event Sourcing by name, but very few actual implementations even today. Zimarev’s remains by far the most complete.

The Event Sourced system precipitates some properties I haven’t seen before. First, it comes with a natural audit log. Every action taken on the domain is stored in the event store along with our choice of metadata, most obviously the user taking the action. The event store is not queryable, necessitating the use of a projection into a readable form. While I’ve designed software using logical CQRS, this system is the first implementation I’ve seen that has different capabilities on the command and query sides, and allows many choices to fulfill these capabilities. Zimarev uses RavenDB + Event Store DB, a good choice today as they can both be deployed in cloud clusters to the AWS, GCP or Azure region of your choice. The specific choices for command and query side are quite involved themselves, and the subject for another article.

The next thing that jumps out at me is that the system is truly asynchronous. I had to move away from the idea that I present an object to the server for processing, and wait for some kind of result back (e.g. a database ID). Instead, the client does all of these id generations in advance and simply issues commands to the server to keep server state in sync. (Side note – look into ULIDs instead of GUIDs for universal IDs). This has its own challenges that must be understood. For example, the queryable database is now the “query” (“read”) side. However, there is latency between when an event enters the store, and when the read side is updated. This only matters if we insist on re-reading the object from the database to get something back. Universal id generation is important here, and the relationship between client and server is flipped on its head. The client holds the master state, and issues commands to the server to persist it.

Eventual consistency is a new challenge to developers. We should not avoid systems that are not immediately consistent, however. We simply need to learn how to build them such that we can truly fire and forget our commands, and be notified only if something goes wrong. The performance potential here is enormous: how much database effort is spent on joins and indexes to split object data into multiple tables, when we could simply store the data the way it is actually used? While some data (accounting, e.g.) seems to be better suited to tabular data, other data never needs to be anything but an object, and it is easiest to store it as such as a JSON document in a document database.

The biggest challenge that I face right now is the migration of the legacy system. It is simply too big to flip the switch on day 1. It needs to incrementally migrate to a new system, which means that the existing version must be able to write to the event store, and the new version must project back to the original database in addition to the document database. Of course, this is a lot of work that will be immediately obsolete, but we see this technique in bridge maintenance as well. If we wish to upgrade a bridge while allowing people to continue to cross the river, we need to build a temporary bridge first before closing the bridge for upgrades.

I don’t know what my performance expectations are at this point. Intuitively, it feels like a better design than using a relational database for absolutely everything. I know I no longer have to write error-prone mapping code to shove the data into an ORM. The more I work with document databases, the more I feel that relational databases are not appropriate persistence for object-based data. An interesting thought is that the databases will do less work. Writing to event streams is not compute-intensive work, nor is reading a selection of documents from a document database. The costs are already lower for the two databases, but I haven’t seen the resource requirements yet. The cost is simply the latency between write and read sides, which is handled by building the system asynchronously and making the application client-centric rather than server-centric. The projection, of course, is handled by the application server, so the latency can be managed to an extent by increasing the resources available to the application server.

Those are my initial thoughts about an Event Sourced system. With any luck, I’ll have a chance to design one soon as our company updates their applications to be cloud-native. I hope to see some tangible results this year.