Ryan Barrett, April 9 2002
Also see P4 and P4 Protocol Specification.
Toward an Application-Independent Distributed Network Platform
We propose a distributed, peer-to-peer network platform with an extensible plugin architecture. This will allow developers to build their own network-aware applications without having to recreate middle-level network services. We plan to deliver a network client as well as at least one, and possibly two plugins: a networked DJ and an anonymous web browsing proxy.
The unique advantages of a distributed network architecture are flexibility, fault-tolerance, and lawyer resistance. The platform will support both broadcast and direct send/receive operations. Users will be separate from hosts, so that a user can log onto the network from any computer. Both authentication and data encryption will be built in from the ground up.
Introduction
Distributed networks, especially peer-to-peer networks, are currently in vogue. Akamai has built a billion dollar company around a distributed content delivery platform. The peer-to-peer Gnutella network is the early successor to Napster’s throne, due in no small part to the interoperability of different clients that implement its protocol. The upcoming Emerging Technology conference, hailed as "one of the industry’s best trend-setters and predictors, bar none…" picked peer-to-peer networking as its theme this year.
Many people see the motivations for peer-to-peer networks as questionable. While this can’t be quantified, it is evident that two of the primary attractions of peer-to-peer networks are easy (and often illegal) file sharing and resistance to lawsuits. However, as a network paradigm, peer-to-peer is a viable alternative to more traditional options like server-client, hierarchical, and ring networks. Furthermore, peer-to-peer is not new. As far back as ARPANET, low-level routing in the Internet has been performed with distributed algorithms and a peer-to-peer architecture.
The original goal for the Internet’s core protocols (IP, UDP, and TCP) was to provide fault-tolerant per-hop and end-to-end network functionality. They have accomplished this goal admirably. However, they provide only low-level services. There is an insufficient number of standard, widespread protocols for middle-level network services. These include user location and authentication, secure communication, and common application discovery. To support popular services like instant messaging and file sharing, applications have been forced to create their own middle-level protocols. Possibly hundreds or even thousands of programmer-years have been spent duplicating these kinds of services. We hope to remedy this.
Proposal
We propose to create a generic distributed network platform. The most salient feature of the platform is its plugin architecture. We will provide application developers with a small number of basic network operations such as direct send and receive, broadcast, user location and authentication, data encryption, and common application discovery. Using these middle-level services, we hope that developers will be able to create network-aware applications more quickly. More importantly, we hope that this will encourage developers to move toward standard middle-level services and platforms (like ours) instead of recreating these services over and over.
We also propose to deliver at least one substantial plugin on this platform. We plan to create a "networked DJ" plugin that can coordinate multiple computers to play music at a party. When a client joins a party using this plugin, it will synchronize the client’s MP3 player to the currently playing song. It will also download the playlist and any songs that the client doesn’t have. Users with DJ permissions will be able to change the playlist and the currently playing song. We hope this will be popular in smaller hall parties and progressives, where the default source of music is a boombox on full volume at one end of the hall.
There are many other compelling ideas for plugins. We could port an existing instant messaging client to our platform or implement a new one. We could write a file sharing client in the vein of Napster, Morpheus, and Gnutella. We could create a distributed computing plugin that would download and complete "work units" from services like Seti@Home and Distributed.net. Lastly, we could design a web anonymizer that uses the platform as a proxy server. HTTP requests and replies could be routed through any node on the network, so that standard web transactions could take place but no personally identifying information (such as IP address) would be revealed. If we have time, we may implement one of these plugins.
Related Work
Sun’s recently announced JXTA project is somewhat similar to our platform. JXTA is a set of open protocols for peer-to-peer network communication, along with reference implementations in Java. Sun’s goals for JXTA are close to our own goals, but JXTA is more a toolkit of loosely connected parts than a unified platform. Also, JXTA is in the working stages right now, so production-level code is still some time away.
Many current peer-to-peer applications provided inspiration for the protocol design of our network. Gnutella is a peer-to-peer file sharing protocol implemented by a wide variety of clients. Gnutella is tailored specifically for file sharing – it provides no user authentication, and it uses a network horizon so each client only sees a fraction of the network at a time. Freenet and Free Haven are more sophisticated. They provide distributed, anonymous, high-performance data havens. They are provably anonymous for file requesters, and it can be shown that hosts cannot determine the contents of files that are replicated to their cache. Mojo Nation takes this a step further by introducing the concept of accountability – access control for a host is determined by the amount of resources that it provides.
Perhaps the closest available program to our project is the Brilliant platform. Brilliant is a trojan program built into KaZaa, a popular file sharing program. Brilliant allows its parent company, Brilliant Digital Entertainment, to use unused clock cycles and hard drive space on clients where it is installed. However, this was not disclosed to KaZaa users, and it was only recently discovered in an FTC filing. Needless to say, we do not plan to distribute our platform in this manner. Furthermore, there are no white papers or technical briefs on Brilliant’s network architecture, so we cannot learn much from it.
Usage Scenarios
We identify two primary usage scenarios: developers and end users. We plan to invest time into both the plugin API and the user interface of our network client. Since we are building a platform and not just a consumer program, we must consider the API we export to developers. Primarily, it should be complete and minimal. In other words, it should expose all of the platform’s functionality, but the number of discrete operations and entry points should be kept to a minimum. Compiling and testing plugins for the platform should be similar to using any other third-party libraries, such as windowing toolkit or text parsers. The API should also provide mechanisms to access debugging information.
The network client is a launching pad for plugins that use the platform. It will show all of the plugins that are currently installed, and will allow the user to load new plugins. It may show other relevant information, such as network statistics or a "buddy list" of users who are logged on. In the interest of simplicity, these will be the only features of the network client. All other user interaction will be handled by the plugins.
The "networked DJ" plugin will use Winamp to decode and play MP3 files. As such, it will also be a Winamp plugin. Winamp will be the main user interface, so pressing the play button will sync transparently. There may be a dialog box for configuration, but it should be minimal. The anonymous web proxy will be implemented as a proxy server with no user interface at all. It will be turned on or off in the network client. The instructions for configuring IE or Netscape to use a proxy server can be included with the plugin.
Known Issues
The main issue we foresee is that this project may be too ambitious in scope. All of us have significant experience with large programming projects, so size is probably not an issue. However, this project touches many different technical fields, such as networking, encryption and authentication, audio syncing, and statistical methods. No one group member is competent in all of these fields, and there are some fields with which only one or two members have significant experience.
Two areas that could cause problems are multithreading and the plugin architecture. To support arbitrarily complex plugins, we may need to allow plugins to run in their own thread(s). This is a concern because multithreaded network applications tend to be unwieldy. Also, I personally do not have experience developing pluggable programs, which could be a technical challenge.
Lastly, the platform must be built before any plugins can be tested on it. This may take a substantial amount of time, and it will be difficult to develop plugins until the platform is stable. It may be awkward to schedule development around this.
Mentions