End-to-End Arguments in System Design

This paper presents an argument against excessive low-level functionality in distributed systems, especially those with a data communication component. Initially, this line of reasoning is illustrated through the example of a file transfer mechanism in which one host reads a file from disk and communicates it across a network to a second host, which then writes the data to disk. The communication subsystem is only one among several possible sources of error for the described mechanism. Since the file transfer application must perform an extensive end-to-end check even in the presence of low-level reliability measures, the communication subsystem should not implement expensive functionality to guarantee reliability. The degree of low-level reliability to strive for should be carefully determined based on the performance tradeoff. Low-level functionality may be efficient for applications that can perfectly leverage the provided reliability measures; on the other hand, it may force applications to bear the burden regardless of whether they actually require the reliability measures.

The author proceeds to provide further illustrative examples of this end-to-end principle such as delivery acknowledgment (the case where acknowledgments must be sent from the application layer), secure transmission (the case where data must not be vulnerable before or after network transmission) and duplicate message suppression (the case where the application generates duplicates beyond the recognition of lower layers). Finally, the end-to-end argument should not be used as an absolute guide but in conjunction with a careful assessment of application requirements and performance tradeoffs.

Given that this paper presents a principle that is the foundation of modern data communication networks, it should be a fundamental part of the syllabus. The author suggests that this end-to-end principle applies to general systems but only elaborates on applications that involve data communication systems. Some of the examples seem slightly far-fetched but overall, the argument is quite convincing.