There was some interesting news out of Twitter which talked about the challenges of scaling databases for the massive social network. Despite the seemingly tiny 140 character tweet sizes, the millions of users create enough to make it a challenge for the social media juggernaut to keep their systems running and scaling at the pace of adoption.
When operating an application environment at massive scale, the products of today often don’t measure up. Working at the scale of a Twitter, or Facebook, requires an entirely different level of scalability. What we are seeing happen is the move of innovation of products out to the application vendors. The great thing about companies like Twitter and many others is that they see the value of that innovation and they choose to share it via open source.
This spawned a new exciting open source project proposal called Mysos.
MySQL is one of the most popular relational database platforms in the industry today. Twitter has benefitted greatly by using this popular RDBMS, and as a show of their love of the community platform, they have also contributed back to the MySQL code base.
As we are seeing more and more, open sourcing of application and infrastructure platforms creates a powerful way to grow and develop for greater consumption. As more of these tools are being innovated with open source development, it creates opportunities for the community to truly drive the direction of the applications. Having a company like Twitter kickstart the project can certainly be a big help.
Distributed RDBMS – Not everything is meant for sharding
As people are looking into Big Data and NoSQL platforms, it is often overlooked just how much is happening in the RDBMS world. There is a reason that many people know about the challenges of Microsoft SQL and Oracle, but they keep fighting the fight because RDBMS is a real requirement for organizations everywhere. With the move to open up a new strain of MySQL on Mesos we are seeing the best of both worlds. Easily scalable versions of an existing and popular database platform.
It will be interesting to see what people do with Mysos, and whether it does take off. Open source can be a profoundly innovative way to launch a new product development, but it can also lead to fragmentation and difficulty. Even when many agree on a direction, it will require some gatekeeping at some point to choose a path for the development. We have seen this in containers, OpenStack, and even the original open source powerhouse Linux.
Exciting times ahead no matter which we we look at it!