Real-world distributed systems are rarely built as a monolithic system. Instead, they are composed of multiple interacting components that together ensure the desired system specification. Programming these systems is challenging as one must deal with both concurrency and failures. This paper proposes techniques for building reliable distributed systems with two central contributions: (1) We propose a module system based on the theory of compositional trace refinement for dynamic systems consisting of asynchronously-communicating state machines, where state machines can be created dynamically and communication topology of the existing state machines can change at runtime; (2) We present ModP, a programming system that implements our module system to enable compositional (assume-guarantee) reasoning of distributed systems.

We demonstrate the efficacy of our framework by building two practical distributed systems, a fault-tolerant transaction commit service and a fault-tolerant distributed hashtable. Our framework helps implement these systems modularly and validate them via compositional systematic testing. We empirically demonstrate that using abstraction-based compositional reasoning helps amplify the coverage during testing and scale it to real-world distributed systems. The distributed services built using ModP achieve performance comparable to open-source equivalents.