Migrating to Microservice Databases

Foreword

To say that data is important is an understatement. Does your code outlive your data, or vice versa? QED. The most recent example of this adage involves Artificial Intelligence (AI). Algorithms are important. Computational power is important. But the key to AI is collecting a massive amount of data. Regardless of your algorithm, no data means no hope. That is why you see such a race to collect data by the tech giants in very diverse fields—automotive, voice, writing, behavior, and so on.

And despite the critical importance of data, this subject is often barely touched or even ignored when discussing microservices. In microservices style, you should write stateless applications. But useful applications are not without state, so what you end up doing is moving the state out of your app and into data services. You’ve just shifted the problem. I can’t blame anyone; properly implementing the full elasticity of a data service is so much more difficult than doing this for stateless code. Most of the patterns and platforms supporting the microservices architecture style have left the data problem for later. The good news is that this is changing. Some platforms, like Kubernetes, are now addressing this issue head on.

After you tackle the elasticity problem, you reach a second and more pernicious one: the evolution of your data. Like code, data structure evolves, whether for new business needs, or to reshape the actual structure to cope better with performance or address more ...