Curbing Connection Churn in Zuul
One of the longest-lived problems with Zuul is how many connections it opened to back-end servers. We recently invested in modifying our connection pooling logic to make use of subsetting and fix these problems. It’s worth reading the blog post if for nothing other than the phenomenal improvement graphs. Here’s a sneak peek of the total connection counts per region falling off a cliff:
Another highlight is the algorithm we used to build the distribution ring for the servers.
The algorithm relies on the idea of low-discrepancy numeric sequences to create a naturally balanced distribution ring that is more consistent than one built on a randomness-based consistent hash. The particular sequence used is a binary variant of the Van der Corput sequence. As long as the sequence of added servers is monotonically incrementing, for each additional server, the distribution will be evenly balanced between 0–1. Below is an example of what the binary Van der Corput sequence looks like.
Another big benefit of this distribution is that it provides a consistent expansion of the ring as servers are removed and added over time, evenly spreading new nodes among the subsets. This results in the stability of subsets and no cascading churn based on origin changes over time. Each node added or removed will only affect one subset, and new nodes will be added to a different subset every time.
For the full deep dive, check out the blog post here.
Securing Netflix Studios At Scale
Over the years, Netflix has evolved from a streaming company to a full-fledged studio. Being a tech company, we wanted to leverage that expertise to build a studio of the future. In this future world, there is an application for every part of the process – from creative, to production, to marketing, and eventually all the way to playback. We call it the “Studio in the Cloud.”
As you can imagine, one of the biggest concerns for these applications is security because of all the sensitive information related to pre-release content. So how do we get these applications on the internet as fast as possible and as securely as possible?
Well, we already have a secure gateway for streaming customers, so why don’t we also leverage it for Studio applications? That’s exactly what we did, and it has streamlined deployment while also increasing security. It has been an unmitigated success story.
