Sharing Static Data Within Spend
Wise helps people who send money abroad save £1 billion a year compared to using a bank. We also help people spend their money without getting ripped off with our bright green debit card, and in the last few months of 2021 we’ve opened the door to help another 280 million people by launching our multi-currency accounts and debit card in 🇧🇷, 🇨🇦, and 🇲🇾.
Doing this at scale is not an easy task. To make sure our customers continue to have a smooth and stress-free experience using our cards, our Spend team — which handles everything to do with cards, from their colour to how we pay your money out — has dedicated a considerable amount of engineering effort to transition away from a monolith to a microservices architecture, something our Engineering Director, Erik, talked about at DevClub.lv.
The microservices architecture we adopted allows us to develop new features much faster (such as Digital Cards), and expanding our functionality into new regions takes much less engineering effort than before. It does, however, come with a new problem: how do we centralise and share this data between our services as much as possible?
Sam Newman has written about the problem of sharing data between services in Building Microservices and presents three options:
Option 1: Put the data where it’s needed
Data required by a service is stored within that service. This causes problems when the same data is required in multiple services, leading to duplication. Updating can cause issues if it’s not updated in all places!
Option 2: Static data library
Data is centralised in a repository. This can be a static data library or property file deployed with each service. Each service will (hopefully!) eventually use the most up to date version. The problem here is there is now an engineering effort necessary to keep data consistent between components, but at least the data is centralised.
Option 3: A new service provides the data
Data is centralised in a new service that responds with the data. The implementation of this can take many different forms depending on the requirements of the consumers.
Our requirements, and what we did
When centralising this data then the frequency of updates is useful to consider. Normally we make updates between once and five times per week but much more frequently as we approach the release of a new product or region.
With this update frequency the third option seems ideal, as new changes are automatically picked up by consumers. Unfortunately, the data is required by an important service allowing people to pay with their card — which has an availability SLA of 99.99%. To keep this data up to date and always available, we would need something much more sophisticated which can provide the data when offline. When dealing with high SLAs we prioritise minimising the amount of logic to reduce complexity and reduce the number of places that can eat into the error budget.
Option two is more appealing if we accept the risks of having our data become eventually consistent, and the option we decided to use. While we sometimes feel the pain of running services with different versions of the data library, it is usually only when testing internally. We have also developed an open-source java framework — tw-tasks — which we can use to rerun tasks that have issues because of inconsistent versions.
In the future, a combination of option two and three would be the best of both worlds. A service can provide the most recent copy of the data, but also provides a static version in case the service is unavailable. It has been tough to prioritise building this when we rarely feel the pain of the current solution.
Depending on data
Now that we’ve centralised this static data, we go a step further to make adding new information easier. At Wise, we scale by writing code that depends on behaviour instead of the data itself — modelling as a domain driven design (DDD) Aggregate offers a nice way to do this. Here’s an example where a set of card data is represented as a card program::
switch (cardProgram) case eea_business_card_program: case usa_business_card_program: case apac_business_card_program: activateCardWithChipAndPin(); case usa_business_card_program: case apac_business_card_program: activateCardWithEmbossedCode();
To something like this:
if (cardProgram.activateCardWithChipAndPin()) activateCardWithChipAndPin(); else if (cardProgram.activateCardWithEmbossedCode() activateCardWithEmbossedCode();
We depend on the behaviour and not the data, which makes it very easy to add new regions and products that generally behave in the same way but have small differences.
By using eventually-consistent data sharing and DDD Aggregates together, we have made it easier on ourselves to build an application layer which deals with new regions and product types, and does so in a relatively painless way.