Bruce Lowekamp
Bruce Lowekamp
Principal Software Engineer
Bruce is architect for Skype’s cloud infrastructure. He enjoys the challenge of building highly available, low-latency global systems using commodity cloud services. At Skype he has worked on systems including client transports, experimentation, notifications, and registration. Prior to Skype he was cofounder of a communications startup and did research in high-performance computing including network performance measurement and adaptive applications. Bruce is co-author of several IETF RFCs in the area of P2PSIP and NAT traversal.
Seatbelt: A modular approach to reliable service-to-service communication

Skype builds its new services as microservices on the Microsoft Azure platform. Our services are deployed and managed independently, and while there is a requirement that they survive failure of a deployment, the HA solution is determined by the service team. While we’ve realized benefits from microservice architecture, one of the challenges we’ve faced is managing service-to-service (S2S) communication. In particular, the range of service requirements and variability in deployment patterns creates complexities that have contributed to numerous incidents as S2S calls fail.

For our solution we built Seatbelt, a library for S2S communication based on modular building blocks of basic functionality. The underlying functionality of Seatbelt is implemented in blocks providing concepts such as Circuit Breaker, Retry, and Resource Pools. The library provides a standard factory that supports typical patterns with a few tuning options—a single call can create the set of blocks required for both real-time and latency-tolerant services. But unusual requirements can be met by working directly with the building blocks to define behavior required for the particular S2S relationship.

In this talk I will describe the experiences at Skype that led to designing and building seatbelt, the way it is used for the "90%" case, and unusual use cases that use the building blocks directly.