Making The Sans I/O Ideal A Reality For The Websockets Library

Summary Working with network protocols is a common need for software projects, particularly in the current age of the internet. As a result, there are a multitude of libraries that provide interfaces to the various protocols. The problem is that implementing a network protocol properly and handling all of the edge cases is hard, and most of the available libraries are bound to a particular I/O paradigm which prevents them from being widely reused. To address this shortcoming there has been a movement towards "sans I/O" implementations that provide the business logic for a given protocol while remaining agnostic to whether you are using async I/O, Twisted, threads, etc. In this episode Aymeric Augustin shares his experience of refactoring his popular websockets library to be I/O agnostic, including the challenges involved in how to design the interfaces, the benefits it provides in simplifying the tests, and the work needed to add back support for async I/O and other runtimes. This is a great conversation about what is involved in making an ideal a reality. Announcements Hello and welcome to Podcast.__init__, the podcast about Python and the people who make it great. When you’re ready to launch your next app or want to try a project you hear about on the show, you’ll need somewhere to deploy it, so take a look at our friends over at Linode. With the launch of their managed Kubernetes platform it’s easy to get started with the next generation of deployment and scaling, powered by the battle tested Linode platform, including simple pricing, node balancers, 40Gbit networking, dedicated CPU and GPU instances, and worldwide data centers. Go to pythonpodcast.com/linode and get a $100 credit to try out a Kubernetes cluster of your own. And don’t forget to thank them for their continued support of this show! Your host as usual is Tobias Macey and today I’m interviewing Aymeric Augustin about his work on the websockets library and the work involved in making it sans I/O Interview Introductions How did you get introduced to Python? Can you start by giving an overview of your work on the websockets library and how the project got started? What does "sans I/O" mean and what are the goals associated with it? Can you share the history of your work on the websockets project? What was your motivation for starting down the path of rearchitecting a project that is already production ready? Can you talk through how the websockets library is architected currently? How has the design of the project evolved since you first began working on it? At a high level, what were the changes required to make it functionally sans i/o? What do you see as the primary challenges associated with making network related libraries sans i/o? In your experience of porting websockets to be purely protocol oriented, what are the technical and design challenges that you faced? One of the goals of the Sans I/O approach is to support reusability and composability of network protocol implementations. What has your experience been as to the viability of those goals in practice? What is your current perspective on the cost/benefit of the sans i/o conversion? Who are the primary consumers of the websockets library? How do you foresee the target audience changing once you have completed extracting the protocol logic? What are some of the most interesting, innovative, or unexpected ways that you have seen the websockets project used? What are the most interesting, unexpected, or challenging lessons that you have learned while working on the websockets project and sans i/o conversion? What do you have planned for the future of the project? Keep In Touch LinkedIn @aymericaugustin on Twitter Website Picks Tobias Jigsaw Puzzles Aymeric Inside Qonto interview Closing Announcements Thank you for listening! Don’t forget to check out our other show, the Data Engineering Podcast for the latest on modern data management. Visit the site to subscribe to the show, sign up for the mailing list, and read the show notes. If you’ve learned something or tried out a project from the show then tell us about it! Email [email protected]) with your story. To help other people find the show please leave a review on iTunes and tell your friends and co-workers Join the community in the new Zulip chat workspace at pythonpodcast.com/chat Links Sans I/O: When The Rubber Meets The Road Websockets library Websockets Protocol Qonto Tulip Asyncio CERN Particle Accelerator Sans I/O Cory Benfield HTTP/2 Twisted Curio Trio Inversion of Control ohneio helper library for implementing sans I/O network protocols SOCKS Proxy Sanic The intro and outro music is from Requiem for a Fish The Freak Fandango Orchestra / CC BY-SA