Over the last several years, Netflix has put a lot of work into building a cloud-based architecture off of Amazon Web Services (AWS) to run its video streaming and DVD rental services. Then the company announced that it was going to open source those same tools and make them available to other developers. Ever since, Netflix has been slowly making other cloud-management tools available for others to build off of. Now it’s hoping to make it easier for others to implement not just one or two of those tools, but all of them.
At a meetup for developers who use — or might be interested in using — the open source cloud management tools that Netflix has already released, the company showed off some new technology that it’s working on, and provided a roadmap for future open source releases. The big message for them is that Netflix is looking to make its open source cloud architecture more portable for those who wish to adopt it, and to make it easier to apply its tools to more use cases.
“Many of you who are using [our open source tools] are probably just using one or two,” Netflix Director Cloud Platform Engineering Ruslan Meshenberg said to an audience of developers. “But these are more powerful when you use them together… If there is one takeaway, it’s that the whole is much greater than the sum of its parts.”
Several years ago, Netflix standardized its cloud architecture on AWS, which Cloud Architect Adrian Cockcroft says the company sees as providing “undifferentiated heavy lifting” for the services that it wishes to run on top of the cloud. In addition, Netflix has built a series of tools to do smart light lifting, and has over the years been adding new tools as new AWS features appear.
But what works for Netflix doesn’t always work perfectly for other companies. The company is trying to change that, as it attempts to make its open source tools easier to use and more applicable to other services that might not be built around the personalized recommendations or the distribution of media online.
How will it do that? For one thing, the company is introducing a series of “recipes” that will help others launch multiple versions of its open source applications seamlessly together. It’s also hoping to roll out a simplified launcher that will make deploying those tools a piece of cake. One specific tool that the company is looking to release soon to make its architecture more portable is called “Denominator,” and it is being built by Adrian Cole, the jClouds guy.
For Netflix, making these tools open source is one way to ensure that its own development won’t eventually go to waste in the future if some other tools are standardized. Cockcroft compared its cloud philosophy to those caravans traveling west during the 19th century. Netflix may have blazed the Oregon Trail for cloud services, but now it’s looking to lay down railroad tracks to make it easier for others to standardize on the cloud.
“We started off a few years ahead of the industry,” Cockcroft said. But he believes it’s better to leverage that architecture as a shared pattern. The thinking goes that if Netflix makes it easier for others to adopt, those tools will become standards for the way cloud services are built. “We want to establish our solutions as best practices or standards,” he said.
For the industry, that’s probably a good thing. Offering reliable, scalable code that can be reused among multiple companies can make adoption of cloud services a lot more efficient over the coming years.