Changes, fixes, renaming, specs, ...

.gitignore

@@ -1,4 +1,4 @@
 target
 Cargo.lock
-ethcloud-*
+vpncloud-*
 ._*

Cargo.toml

@@ -1,5 +1,5 @@
 [package]
-name = "ethcloud"
+name = "vpncloud"
 version = "0.1.0"
 authors = ["Dennis Schwerdel <schwerdel@informatik.uni-kl.de>"]
 build = "build.rs"

ethcloud.md

@@ -1,72 +1,193 @@
-ethcloud(1) -- Layer 2 VPN over UDP
+vpncloud(1) -- Peer-to-peer VPN
-===================================
+===============================
 
 ## SYNOPSIS
 
-```
+`vpncloud [options] [-t <type>] [-d <device>] [-l <listen>] [-c <connect>...]`
-Usage:
+
-    ethcloud [options]
+
+## OPTIONS
+
+  * `-t <type>`, `--type <type>`:
+
+    Set the type of network. There are two options: **tap** devices process
+    Ethernet frames **tun** devices process IP packets. [default: tap]
+
+  * `-d <device>`, `--device <device>`:
+
+    Name of the virtual device. Any "%d" will be filled with a free number.
+    [default: vpncloud%d]
+
+  * `-m <mode>`, `--mode <mode>`:
+
+    The mode of the VPN. The VPN can like a router, a switch or a hub. A **hub**
+    will send all data always to all peers. A **switch** will learn addresses
+    from incoming data and only send data to all peers when the address is
+    unknown. A **router** will send data according to known subnets of the
+    peers and ignore them otherwise. The **normal** mode is switch for tap
+    devices and router for tun devices. [default: normal]
+
+  * `-l <listen>`, `--listen <listen>`:
+
+    The address to listen for data. [default: 0.0.0.0:3210]
+
+  * `-c <addr>`, `--connect <addr>`:
+
+    Address of a peer to connect to. The address should be in the form
+    `addr:port`. If the node is not started, the connection will be retried
+    periodically. This parameter can be repeated to connect to multiple peers.
+
+  * `--subnet <subnet>`:
+
+    The local subnets to use. This parameter should be in the form
+    `address/prefixlen` where address is an IPv4 address, an IPv6 address, or a
+    MAC address. The prefix length is the number of significant front bits that
+    distinguish the subnet from other subnets. Example: `10.1.1.0/24`.
+
+  * `--network-id <network_id>`:
+
+    An optional token that identifies the network and helps to distinguish it
+    from other networks.
+
+  * `--peer-timeout <peer_timeout>`:
+
+    Peer timeout in seconds. The peers will exchange information periodically
+    and drop peers that are silent for this period of time. [default: 1800]
+
+  * `--dst-timeout <dst_timeout>`:
+
+    Switch table entry timeout in seconds. This parameter is only used in switch
+    mode. Addresses that have not been seen for the given period of time  will
+    be forgot. [default: 300]
+
+  * `-v`, `--verbose`:
+
+    Print debug information, including information for data being received and
+    sent.
+
+  * `-q`, `--quiet`:
+
+    Only print errors and warnings.
+
+  * `-h`, `--help`:
+
+    Display the help.
 
-Options:
-    -d <device>, --device <device>         Name of the tap device [default: ethcloud%d]
-    -l <listen>, --listen <listen>         Address to listen on [default: 0.0.0.0:3210]
-    -t <token>, --token <token>            Token that identifies the network [default: 0]
-    -c <connect>, --connect <connect>      List of peers (addr:port) to connect to
-    --peer-timeout <peer_timeout>          Peer timeout in seconds [default: 1800]
-    --mac-timeout <mac_timeout>            Mac table entry timeout in seconds [default: 300]
-    -v, --verbose                          Log verbosely
-    -q, --quiet                            Only print error messages
-```
 
 ## DESCRIPTION
 
-**Ethcloud** is a simple layer 2 VPN over UDP. It creates an ethernet based
+**VpnCloud** is a simple VPN over UDP. It creates a virtual network interface on
-network interface on the host and forwards all received frames via UDP to the
+the host and forwards all received data via UDP to the destination. It can work
-destination.
+in 3 different modes:
-The forwarding is based on traditional switch behavior with MAC address
-learning. Whenever a frame is received, the sender UDP address and MAC address
-are associated and used for replies. Frames for unknown addresses will be
-broadcast to all peers.
-All connected ethcloud programs will form a peer-to-peer network and
-cross-connect automatically until the network is fully connected.
 
-The token is used to distinguish different networks and discard foreign packets.
+  * **Switch mode**: In this mode, the VPN will dynamically learn addresses
-It should be unique.
+    as they are used as source addresses and use them to forward data to its
+    destination. Addresses that have not been seen for some time
+    (option `dst_timeout`) will be forgot. Data for unknown addresses will be
+    broadcast to all peers. This mode is the default mode for TAP devices that
+    process Ethernet frames but it can also be used with TUN devices and IP
+    packets.
 
-Ethcloud does not implement any loop-avoidance. Since data received on the UDP
+  * **Hub mode**: In this mode, all data will always be broadcast to all peers.
-socket will only be sent to the local network interface and vice versa, ethcloud
+    This mode uses lots of bandwidth and should only be used in special cases.
-cannot produce loops on its own.
 
-IEEE 802.1q frames (VLAN tagged) are detected and forwarded based on separate
+  * **Router mode**: In this mode, data will be forwarded based on preconfigured
-MAC tables. All frames without a tag will be treated as having tag `0`.
+    address ranges ("subnets"). Data for unknown nodes will be silently ignored.
+    This mode is the default mode for TUN devices that work with IP packets but
+    it can also be used with TAP devices and Ethernet frames.
 
-The peer-to-peer protocol will cause nodes to exchange information about their
+All connected vpncloud nodes will form a peer-to-peer network and cross-connect
-peers. For nodes behind a firewall or a NAT, this can function as hole-punching.
+automatically until the network is fully connected. The nodes will periodically
+exchange information with the other nodes to signal that they are still active
+and to allow the automatic cross-connect behavior. There are some important
+things to note:
+
+  - To avoid that different networks that reuse each others addresses merge due
+    to the cross-connect behavior, the `network_id` option can be used and set
+    to any unique string to identify the network. The `network_id` must be the
+    same on all nodes of the same VPN network.
+
+  - The cross-connect behavior can be able to connect nodes that are behind
+    firewalls or NATs as it can function as hole-punching.
+
+  - The management traffic will increase with the peer number quadratically.
+    It should still be reasonably small for high node numbers (below 10 KiB/s
+    for 10.000 nodes). A longer `peer_timeout` can be used to reduce the traffic
+    further. For high node numbers, router mode should be used as it never
+    broadcasts data.
+
+VpnCloud does not implement any loop-avoidance. Since data received on the UDP
+socket will only be sent to the local network interface and vice versa, VpnCloud
+cannot produce loops on its own. On the TAP device, however STP data can be
+transported to avoid loops caused by other network components.
+
+For TAP devices, IEEE 802.1q frames (VLAN tagged) are detected and forwarded
+based on separate MAC tables. Any nested tags (Q-in-Q) will be ignored.
+
+
+## EXAMPLES
 
-Ethcloud should be able to scale to a few thousand nodes with reasonable
-management traffic (below 10 KiB/s for 10.000 nodes). However, such huge
-networks will cause a lot of traffic due to broadcasts. At this point, a routed
-approach should be preferred.
 
 
 ## NETWORK PROTOCOL
 
-The protocol of `ethcloud` is kept as simple as possible to allow other
+The protocol of VpnCloud is kept as simple as possible to allow other
 implementations and to maximize the performance.
 
-The first 7 bytes of each packet are the token that is used to distinguish
+Every packet sent over UDP contains the following header (in order):
-different networks and sort out stray packets that do not belong.
 
-After that, the 8th byte is a switch that determines the structure of the rest
+  * 3 bytes `magic constant` = [0x76, 0x70, 0x6e] ("vpn")
-of the packet:
 
-  * **Frame packet** (value `0`):
+    This field is used to identify the packet and to sort out packets that do
-    This packet contains an actual ethernet frame which starts right after the
+    not belong.
-    switch byte and ends at the end of the packet. It contains the main
-    ethernet frame data starting with the destination MAC and ending with the
-    payload. It does not contain the preamble, SFD, padding, and CRC fields.
 
-  * **Peer list** (value `1`):
+  * 1 byte `version number` = 1 (currently)
+
+    This field specifies the version and helps nodes to parse the rest of the
+    header and the packet.
+
+  * 2 `reserved bytes` that are currently unused
+
+  * 1 byte for `flags`
+
+    This byte contains flags that specify the presence of additional headers.
+    The flags are enumerated from bit 1 (least significant bit) to bit 8
+    (most significant bit). The additional headers must be present in this same
+    order. Currently the following additional headers are supported:
+
+    - Bit 1: Network ID
+
+  * 1 byte for the `message type`
+
+    This byte specifies the type of message that follows after all additional
+    headers. Currently the following message types are supported:
+
+    - Type 0: Data packet
+    - Type 1: Peer list
+    - Type 2: Initial message
+    - Type 3: Closing message
+
+After this 8 byte header, the additional headers as specified in the `flags`
+field will follow in the order of their respective flag bits.
+
+  * **Network ID**:
+
+    The network id is encoded as 8 bytes.
+
+
+After the additional headers, message as specified in the `message type` field
+will follow:
+
+  * **Data packet** (message type 0):
+    This packet contains payload. The format of the data depends on the device
+    type. For TUN devices, this data contains an IP packet. For TAP devices it
+    contains an Ethernet frame. The data starts right after all additional
+    headers and ends at the end of the packet.
+    If it is an Ethernet frame, it will start with the destination MAC and end
+    with the payload. It does not contain the preamble, SFD, padding, and CRC
+    fields.
+
+  * **Peer list** (message type 1):
     This packet contains the peer list of the sender. The first byte after the
     switch byte contains the number of IPv4 addresses that follow.
     After that, the specified number of addresses follow, where each address
@@ -77,16 +198,29 @@ of the packet:
     each address is encoded in 18 bytes. The first 16 bytes are the IPv6 address
     and the later 2 bytes are port number (both in network byte order).
 
-  * **Get peer list** (value `2`):
+  * **Initial message** (message type 2):
-    This packet requests that the receiver sends its peer list to the sender.
+    This packet contains all the local subnets claimed by the nodes.
-    It does not contain any further data.
+    The subnet list is encoded in the following way: The first byte of data
+    contains the number of encoded subnets that follow. After that, the given
+    number of encoded subnets follow.
+    For each subnet, the first byte is the length of bytes in the base address
+    and is followed by the given number of base address bytes and one additional
+    byte that is the prefix length of the subnet.
+    The addresses for the subnet will be encoded like they are encoded in their
+    native protocol (4 bytes for IPv4, 16 bytes for IPv6, and 6 bytes for a MAC
+    address) with the exception of MAC addresses in a VLan which will be encoded
+    in 8 bytes where the first 2 bytes are the VLan number in network byte order
+    and the later 6 bytes are the MAC address.
 
-  * **Close** (value `3`):
+  * **Closing message** (message type 3):
-    This packet requests that the receiver removes the sender from its peer list
+    This packet does not contain any further data.
-    and stops sending data to it. It does not contain any further data.
 
-Nodes are expected to request the peer list from the initial nodes they are
+Nodes are expected to send an **initial message** whenever they connect to a
-connecting to. After that, they should periodically send their peer list to all
+node they were not connected to before. As a reply to this message, another
+initial should be sent if the node was not known before. Also a **peer list**
+message should be sent as a reply.
+
+When connected, nodes should periodically send their **peer list** to all
 of their peers to spread this information and to avoid peer timeouts.
 To avoid the cubic growth of management traffic, nodes should at a certain
 network size start sending partial peer lists instead of the full list.
@@ -95,13 +229,14 @@ The subsets can be selected using round robin (making sure all peers eventually
 receive all information) or randomly.
 
 Nodes should remove peers from their peer list after a certain period of
-inactivity or when receiving a `Close` message. Before shutting down, nodes
+inactivity or when receiving a **closing message**. Before shutting down, nodes
-should send the `Close` message to all of their peers in order to avoid
+should send the closing message to all of their peers in order to avoid
 receiving further data until the timeout is reached.
 
-Nodes should only add nodes to their peer list after receiving a message from
+Nodes should only add nodes to their peer list after receiving an initial
-them instead of adding them right from the peer list of another peer. This
+message from them instead of adding them right from the peer list of another
-is necessary to avoid the case of a large network keeping dead nodes alive
+peer. This is necessary to avoid the case of a large network keeping dead nodes
+alive.
 
 
 ## COPYRIGHT

src/cloud.rs

@@ -148,7 +148,7 @@ impl<P: Protocol> GenericCloud<P> {
                 }
             }
         }
-        info!("Connecting to {}", addr);
+        debug!("Connecting to {}", addr);
         if reconnect {
             let addr = addr.to_socket_addrs().unwrap().next().unwrap();
             self.reconnect_peers.push(addr);

src/ip.rs

@@ -53,6 +53,7 @@ impl Table for RoutingTable {
             Some(val) => val,
             None => addr.0.len() as u8 * 8
         };
+        info!("New routing entry: {:?}/{} => {}", addr, prefix_len, address);
         let group_len = (prefix_len as usize / 16) * 2;
         let group_bytes: Vec<u8> = addr.0[..group_len].iter().map(|b| *b).collect();
         let routing_entry = RoutingEntry{address: address, bytes: addr.0, prefix_len: prefix_len};
@@ -64,7 +65,7 @@ impl Table for RoutingTable {
 
     fn lookup(&self, addr: &Address) -> Option<SocketAddr> {
         let len = addr.0.len()/2 * 2;
-        for i in 0..len/2 {
+        for i in 0..(len/2)+1 {
             if let Some(group) = self.0.get(&addr.0[0..len-2*i]) {
                 for entry in group {
                     if entry.bytes.len() != addr.0.len() {

src/main.rs

@@ -21,7 +21,7 @@ use std::str::FromStr;
 use device::Device;
 use ethernet::SwitchTable;
 use ip::RoutingTable;
-use types::{Error, Behavior, Type, Range, Table};
+use types::{Error, Mode, Type, Range, Table};
 use cloud::{TapCloud, TunCloud};
 
 
@@ -44,34 +44,17 @@ impl log::Log for SimpleLogger {
     }
 }
 
-static USAGE: &'static str = "
+static USAGE: &'static str = include_str!("usage.txt");
-Usage:
-    ethcloud [options] [-t <type>] [-d <device>] [-l <listen>] [-c <connect>...]
-
-Options:
-    -t <type>, --type <type>               Set the type of network [default: tap]
-    --behavior <behavior>                  The behavior of the vpn [default: normal]
-    -d <device>, --device <device>         Name of the virtual device [default: cloud%d]
-    -l <listen>, --listen <listen>         Address to listen on [default: 0.0.0.0:3210]
-    -c <connect>, --connect <connect>      List of peers (addr:port) to connect to
-    --network-id <network_id>              Optional token that identifies the network
-    --peer-timeout <peer_timeout>          Peer timeout in seconds [default: 1800]
-    --subnet <subnet>...                   The local subnets to use
-    --dst-timeout <dst_timeout>            Switch table entry timeout in seconds [default: 300]
-    -v, --verbose                          Log verbosely
-    -q, --quiet                            Only print error messages
-    -h, --help                             Display the help
-";
 
 #[derive(RustcDecodable, Debug)]
 struct Args {
     flag_type: Type,
-    flag_behavior: Behavior,
+    flag_mode: Mode,
     flag_subnet: Vec<String>,
     flag_device: String,
     flag_listen: String,
     flag_network_id: Option<String>,
-    flag_connect: Vec<String>,
+    flag_addr: Vec<String>,
     flag_peer_timeout: usize,
     flag_dst_timeout: usize,
     flag_verbose: bool,
@@ -100,14 +83,14 @@ fn main() {
     }
     let dst_timeout = Duration::seconds(args.flag_dst_timeout as i64);
     let peer_timeout = Duration::seconds(args.flag_peer_timeout as i64);
-    let (learning, broadcasting, table): (bool, bool, Box<Table>) = match args.flag_behavior {
+    let (learning, broadcasting, table): (bool, bool, Box<Table>) = match args.flag_mode {
-        Behavior::Normal => match args.flag_type {
+        Mode::Normal => match args.flag_type {
             Type::Tap => (true, true, Box::new(SwitchTable::new(dst_timeout))),
             Type::Tun => (false, false, Box::new(RoutingTable::new()))
         },
-        Behavior::Router => (false, false, Box::new(RoutingTable::new())),
+        Mode::Router => (false, false, Box::new(RoutingTable::new())),
-        Behavior::Switch => (true, true, Box::new(SwitchTable::new(dst_timeout))),
+        Mode::Switch => (true, true, Box::new(SwitchTable::new(dst_timeout))),
-        Behavior::Hub => (false, true, Box::new(SwitchTable::new(dst_timeout)))
+        Mode::Hub => (false, true, Box::new(SwitchTable::new(dst_timeout)))
     };
     let network_id = args.flag_network_id.map(|name| {
         let mut s = SipHasher::new();
@@ -117,14 +100,14 @@ fn main() {
     match args.flag_type {
         Type::Tap => {
             let mut cloud = TapCloud::new(device, args.flag_listen, network_id, table, peer_timeout, learning, broadcasting, ranges);
-            for addr in args.flag_connect {
+            for addr in args.flag_addr {
                 cloud.connect(&addr as &str, true).expect("Failed to send");
             }
             cloud.run()
         },
         Type::Tun => {
             let mut cloud = TunCloud::new(device, args.flag_listen, network_id, table, peer_timeout, learning, broadcasting, ranges);
-            for addr in args.flag_connect {
+            for addr in args.flag_addr {
                 cloud.connect(&addr as &str, true).expect("Failed to send");
             }
             cloud.run()

src/types.rs

@@ -4,7 +4,7 @@ use std::{fmt, ptr};
 use std::os::unix::io::AsRawFd;
 use std::str::FromStr;
 
-use super::util::as_bytes;
+use super::util::{as_bytes, as_obj};
 
 pub type NetworkId = u64;
 
@@ -17,6 +17,11 @@ impl fmt::Debug for Address {
             4 => write!(formatter, "{}.{}.{}.{}", self.0[0], self.0[1], self.0[2], self.0[3]),
             6 => write!(formatter, "{:x}:{:x}:{:x}:{:x}:{:x}:{:x}",
                 self.0[0], self.0[1], self.0[2], self.0[3], self.0[4], self.0[5]),
+            8 => {
+                let vlan = u16::from_be( *unsafe { as_obj(&self.0[0..1]) });
+                write!(formatter, "vlan{}/{:x}:{:x}:{:x}:{:x}:{:x}:{:x}",
+                    vlan, self.0[0], self.0[1], self.0[2], self.0[3], self.0[4], self.0[5])
+            },
             16 => write!(formatter, "{:x}{:x}:{:x}{:x}:{:x}{:x}:{:x}{:x}:{:x}{:x}:{:x}{:x}:{:x}{:x}:{:x}{:x}",
                 self.0[0], self.0[1], self.0[2], self.0[3], self.0[4], self.0[5], self.0[6], self.0[7],
                 self.0[8], self.0[9], self.0[10], self.0[11], self.0[12], self.0[13], self.0[14], self.0[15]
@@ -93,7 +98,7 @@ pub enum Type {
 }
 
 #[derive(RustcDecodable, Debug)]
-pub enum Behavior {
+pub enum Mode {
     Normal, Hub, Switch, Router
 }
 

src/udpmessage.rs

@@ -53,7 +53,7 @@ impl<'a> fmt::Debug for Message<'a> {
                 }
                 write!(formatter, "]")
             },
-            &Message::Init(ref data) => write!(formatter, "Init(data: {} bytes)", data.len()),
+            &Message::Init(ref data) => write!(formatter, "Init{:?}", data),
             &Message::Close => write!(formatter, "Close"),
         }
     }
@@ -209,8 +209,8 @@ pub fn encode(options: &Options, msg: &Message, buf: &mut [u8]) -> usize {
                 assert!(buf.len() >= pos + 1 + len + 1);
                 buf[pos] = len as u8;
                 pos += 1;
-                unsafe { ptr::copy_nonoverlapping(base.0.as_ptr(), buf[pos..].as_mut_ptr(), base.0.len()) };
+                unsafe { ptr::copy_nonoverlapping(base.0.as_ptr(), buf[pos..].as_mut_ptr(), len) };
-                pos += base.0.len();
+                pos += len;
                 buf[pos] = range.prefix_len;
                 pos += 1;
             }

src/usage.txt

@@ -0,0 +1,21 @@
+Usage:
+    vpncloud [options] [-t <type>] [-d <device>] [-l <listen>] [-c <connect>...]
+
+Options:
+    -t <type>, --type <type>        Set the type of network ("tap" or "tun").
+                                    [default: tap]
+    -d <device>, --device <device>  Name of the virtual device.
+                                    [default: vpncloud%d]
+    -m <mode>, --mode <mode>        The mode of the VPN ("hub", "switch",
+                                    "router", or "normal"). [default: normal]
+    -l <listen>, --listen <listen>  The address to listen for data.
+                                    [default: 0.0.0.0:3210]
+    -c <addr>, --connect <addr>     Address of a peer to connect to.
+    --subnet <subnet>               The local subnets to use.
+    --network-id <network_id>       Optional token that identifies the network.
+    --peer-timeout <peer_timeout>   Peer timeout in seconds. [default: 1800]
+    --dst-timeout <dst_timeout>     Switch table entry timeout in seconds.
+                                    [default: 300]
+    -v, --verbose                   Print debug information.
+    -q, --quiet                     Only print errors and warnings.
+    -h, --help                      Display the help.