Solipsis Protocol
From Solipsis
Please note: this page describes the latest stable version of the protocol. An obsolete version of the protocol is documented in the draft for the IETF (http://solipsis.netofpeers.net/docs/Solipsis.rfc).
Structure
It is a connection-less protocol, currently implemented on top of UDP. The message layout looks a lot like HTTP and other Internet protocols.
The first line in a message is the command, e.g.
HELLO SOLIPSIS/1.0
The next lines are parameters, with a syntax similar to HTTP headers:
Id: 456320321268-A458-000000 Position: 4096,53000,0 Awareness-Radius: 267 Pseudo: John
Currently almost all messages have no payload after the headers. The only exceptions are the SERVICEDATA and MIDDLEMAN messages.
Protocol versioning
Each message introduces, in the command-line, the version of the Solipsis protocol to which the message conforms. There are two versions of the Solipsis protocol right now: 1.0 and 1.1. This document tries to track the latest version of the protocol.
By default, when contacting an unknown peer, the node should use a "safe" protocol version (one which is assumed to be widely established among Solipsis users). It is possible to use a simple negotiation scheme when sending a HELLO message, so as to switch to the most recent protocol version understood by both peers. The fact that we can use different protocol versions depending on the peers we exchange messages with, helps keep an unified world without urging everyone to upgrade to the latest version.
Right now, the "safe" protocol version is 1.0, which means all implementations should be able to handle this version. It is planned to make 1.1 the "safe" version as soon as possible, since it introduces crucial features (see below).
History
The 1.0 version was the first stable protocol version.
The 1.1 version introduces functionality for:
- protocol versioning and negotiation
- hole-punching through "address-restricted" NATs
- enabling establishing a connection behind the same NAT (using the peers' private addresses)
(you can find more information about protocol 1.1 design (http://lists.berlios.de/pipermail/solipsis-tech/2005-August/000137.html) in the mailing-list archive)
See detailed changes for syntax modifications.
Message types
We hereafter list the different message types supported by the protocol type, their parameters and possible replies. It is important to remember that UDP being connection-less, "replies" are not to be taken in the HTTP meaning: a reply is simply an independent message that is sent by the receiver to the sender. Under certain conditions (network problems, etc.) there may be no reply, without any error notification.
Handshaking
HELLO
The sender tries to establish a connection with a peer.
- Parameters: Id, Address, Position, Awareness-Radius, Hold-Time, Send-Detects, Version, Needs-Middleman
- Possible replies: CLOSE, CONNECT
CONNECT
This is the response to a HELLO message, upon reception of this message both peers are considered connected
- Parameters: Id, Address, Position, Awareness-Radius, Hold-Time, Version, Needs-Middleman
- Possible reply: QUERYMETA
CLOSE
Close/refuse connection with a peer. This is either a response to a HELLO message or a spontaneous decision.
- Parameters: Id
QUERYMETA
Ask for meta-information about the destination peer, and send one's own metadata.
- Parameters: Id, Pseudo, Services, Languages
- Expected reply: META
META
Send one's own metadata. Either in reply to QUERYMETA, or spontaneously if the metadata have changed.
- Parameters: Id, Pseudo, Services, Languages
- Possible replies: any number of QUERYSERVICE messages
Finding one's position in the world
Queries
JUMPNEAR
The sender wants to jump near the receiver. It asks it to choose a position inside its own awareness-radius.
- Parameters: Id, Address
- Expected reply: SUGGEST
FINDNEAREST
The sender asks for the peer that is the closest to a target postion.
QUERYAROUND
The sender asks for the peer that is closest to a target position and in the right half-plane. The receiver might instead find a peer that is closer to the target than the specified Best-Distance.
Responses
SUGGEST
This is the reply to a JUMPNEAR message. It suggests a position inside the sender's awareness-radius. When the receiver gets this information, it will usually jump to the said position.
- Parameters: Id, Address, Position, Awareness-Radius, Remote-Position
NEAREST
This is the reply to a FINDNEAREST message (or possibly QUERYAROUND). It contains information about the peer that is the nearest to the target position.
- Parameters: Address, Remote-Id, Remote-Address, Remote-Position, Remote-Version, Remote-Needs-Middleman
BEST
This is the other possible reply to a FINDNEAREST message. The sender says that it is the closest peer to the target position.
- Parameters: Id, Address, Position, Awareness-Radius
AROUND
This the reply to a QUERYAROUND message. It contains information about a peer that is around the target position.
- Parameters: Address, Remote-Id, Remote-Address, Remote-Position, Remote-Awareness-Radius, Remote-Version, Remote-Needs-Middleman
Remaining up-to-date
SEARCH
The sender lacks connectivity and asks its neighbour(s) to find other peers in its awareness radius and in a given half-plane.
- Parameters: Id, Clockwise
- Possible replies: FOUND
FOUND
This is the reply to a SEARCH message. The sender sends information about a peer that satisfies the SEARCH criteria.
- Parameters: Address, Remote-Id, Remote-Address, Remote-Position, Remote-Version, Remote-Needs-Middleman
UPDATE
The sender has changed at least one of its primary characteristics (position, awareness radius). It sends its new characteristics to all neighbours.
- Parameters: Id, Address, Position, Awareness-Radius
- Possible replies: DETECT
DETECT
The sender has detected a peer that has become close to the receiver. This can be either a reply to an UPDATE message or a spontaneous message.
- Parameters: Address, Remote-Id, Remote-Address, Remote-Position, Remote-Version, Remote-Awareness-Radius, Remote-Needs-Middleman
HEARTBEAT
The sender reminds its neighbours that it is alive. Too many skipped HEARTBEATs means the connection will be CLOSEd by one or both peers.
- Parameters: Id
Exchanging service information
QUERYSERVICE
Ask for information about a service in the destination peer, and send one's own details about that service.
- Parameters: Id, Service-Id, Service-Address
- Expected reply: SERVICEINFO
SERVICEINFO
Send information about a given service. Usually in reply to QUERYSERVICE.
- Parameters: Id, Service-Id, Service-Address
SERVICEDATA
Send service-specific data. Usually this message is triggered by the navigator. The data must remain short: there is no reliable transport layer at the moment.
- Parameters: Id, Service-Id
- The message payload carries the service data to be transmitted, in any arbitrary format
Proxying a message
MIDDLEMAN
Ask the recipient to forward a Solipsis message to one of its peers on behalf of the sender. This is useful for sending a first message before a hole is punched through the destination's NAT.
- Parameters: Id, Remote-Id
- The message payload carries the plaintext of the message to be forwarded
Message parameters
Id
Id refers to the id of the sender. It is a string (exact format is not specified) and should be statistically unique accross the Solipsis network.
Address
Address is the UDP address of the sender. It is written as a string of the form "214.88.5.13:5116" (without the quotes). In case the sender is behind a NAT, it can optionally include its private address in addition to its public address. Then the syntax is of the form "214.88.5.13:5116@192.168.0.10:6010" where the part before the "@" character is the public address and the part after the "@" is the private address. This allows the peer to be contacted by other peers behind the same NAT (using the private address), as NATs usually aren't reentrant.
Position
Position is the position of the sender as a tuple of 3 numbers: e.g. "45555,6555,0". Currently the numbers are long integers in [0 ... 2^128] but this could be changed (see CoordinateUnits). (please note the z coordinate is currently unused and should be zero)
Awareness-Radius
Awareness-Radius is the radius of the awareness area of the sender (a disk centered onto the sender). It is expressed in the same coordinate units as the node's position.
Version
This parameters gives the highest version of the protocol that the sender supports. It may be different than the version that the message conforms to, since the recipient may not support (or not be known to support) all the versions that the sender supports. Thanks to this parameter, after a HELLO/CONNECT handshake, two peers can use the optimal protocol version for their communications.
- Example: 1.1.
Needs-Middleman
This parameter specifies whether the sender needs to be first contacted through a middleman when a "connection" has not been established. It is used for NAT hole-punching when the sender is behind an "address-restricted" or "address/port-restricted" NAT. Then a peer needing to contact the sender first goes through the peer by which it has been advertised.
- Examples: yes, no.
Remote-Id, Remote-Address, Remote-Position, Remote-Awareness-Radius, Remote-Version, Remote-Needs-Middleman
These parameters have the same meaning as their counterparts above, but give information about another peer instead of the sender itself.
Best-Id, Best-Distance
Best-Id gives the id of the "best" peer encountered until now on our search for a given target position. Best-Distance is the distance of this peer to the target. These two fields are required by the step-by-step the positioning algorithm.
Clockwise
Clockwise specifies in which angular direction the recipient of a SEARCH message must look for peers. The two possible values are "+1" (clockwise direction) and "-1" (counter-clockwise direction).
Hold-Time
Hold-Time is used in the HELLO/CONNECT handshake to negotiate the keepalive duration of the Solipsis "connection". Each peer tells to its neighbour how much time (as an integer number of seconds) it will remain connected while no message is received from the other peer. It is recommended that the HEARBEAT frequency be a fraction of this duration (for example one half, or one third). When a connection is considered timed out, the peer that detects this situation sends a CLOSE message to its ex-neighbour.
Send-Detects
Send-Detects is used in the HELLO message to indicate if the sender wants to receive information about all peers currently in its awareness radius. The two possible values are:
- "now": immediately send information about all peers
- "later": only send information about new peers when a movement is detected
The normal use is to ask for information when we've just lost our global connectivity and we don't want to miss any peers in our awareness radius.
Pseudo
This is the name, nickname or pseudonym of the sender. This information is exchanged during the META/QUERYMETA handshake. The pseudo is encoded in UTF-8, and can contain any non-ascii characters except control characters.
Languages
A comma-separated list of language codes describing which languages are recognized by the person controlling the node.
- Example: fr, en.
Services
A comma-separated list of service ids supported by the navigator attached to the node. In this context, a service id can optionally bear a direction specifier, such as d=in, d=out or d=bidir (bidir is the default and need not be specified).
- Example: chat, video;d=in, filesharing
Service-Id
The id of the service that is being described. It is a string and should uniquely identify each service available in the Solipsis network.
- Examples: chat, filesharing
Service-Address
The address (possibly empty) at which a given service can be contacted. The string format is left completely free. The specific service is responsible for parsing this string and extracting whatever information it is looking for. Likely examples are 45.78.45.23:2555 for a socket-based service, myaddy@domain.com for an email-based service, etc.
Open questions
Service-specific information
This is temporarily resolved by the SERVICEDATA message type. We must improve it to support large payloads, retransmission, etc.
How to make the protocol easily extendable so that additional services can also transmit important information on the Solipsis wire ? Solutions are:
- Rewrite the protocol in XML (à la Jabber), thus allowing services to add their data using their own namespace; this would be tedious, make the protocol less readable, and most of all may produce very big messages, hampering UDP reliability
- You could use an xml encoding like xbis, which reduces the size of the data transferred and the time needed to load it into memory.
- Add specific headers, using a prefix in front of them (Chat-Pseudo, Chat-Current-Mood, File-ID3Tag, etc.)
- Send service-specific information in the message payload, using their own custom format
- Use a specific message type, say SERVICEDATA
Message length and compression
UDP works best with very short messages. The current format is very simple and readable but may produce long messages as we add more and more parameters. It would be possible to define a "compressed" format. For example SIP defines such a compressed format where header names are shortened (Content-Length becomes cl, etc.). It might be a good compromise between readibility, debuggability, and efficiency.
