From 73aa8628ff51e4e245821faa007a46acaaac3d13 Mon Sep 17 00:00:00 2001
From: Dennis Schwerdel <schwerdel@googlemail.com>
Date: Wed, 25 Nov 2015 23:05:17 +0100
Subject: [PATCH] Updated measurements done with current kernel and

---
 performance.md | 128 ++++++++++++++++++++++++++++++++++++++++---------
 1 file changed, 105 insertions(+), 23 deletions(-)

diff --git a/performance.md b/performance.md
index 22bfcf1..c5c331f 100644
--- a/performance.md
+++ b/performance.md
@@ -3,14 +3,19 @@ Performance Tests
 
 ### Test setup
 
-Two nodes with:
-  * 2x Intel(R) Xeon(R) CPU L5420
-  * 16 GiB Ram
-  * Intel Corporation 80003ES2LAN Gigabit Ethernet Controller (Kernel driver e1000e)
-  * Debian Linux 7.0 (Kernel 2.6.32-26-pve)
-  * Connected by Cisco Catalyst 4506 switch
+Sender node:
+  * Intel(R) Core(TM) i5-2540M CPU @ 2.60GHz
+  * 8 GiB Ram
+  * Intel 82579LM Gigabit Network
+  * Ubuntu 14.04 (Kernel 3.13.0-65-generic)
 
-VpnCloud version: `VpnCloud v0.1.0 (without crypto support, protocol version 1)`
+Receiver node:
+  * Intel(R) Core(TM) i5-3450 CPU @ 3.10GHz
+  * 16 GiB Ram
+  * Realtek RTL8111/8168/8411 Gigabit Network
+  * Ubuntu 14.04 (Kernel 3.13.0-63-generic)
+
+VpnCloud version: `VpnCloud v0.1.0 (with crypto support, protocol version 1)`
 
 
 ### Test 1: Unencrypted throughput
@@ -40,11 +45,11 @@ $> iperf -c 10.2.1.2 -t 60
 ```
 
 **Results:**
-  * Throughput without VpnCloud: 938 Mbits/sec
-  * Throughput via VpnCloud (MTU=1400): 461 Mbits/sec
-  * CPU usage for VpnCloud (MTU=1400): maxed out at ~105% of one core
-  * Throughput via VpnCloud (MTU=16384): 949 Mbits/sec (no idea why this is higher)
-  * CPU usage for VpnCloud (MTU=16384): ~68% of one core
+  * Throughput without VpnCloud: 926 Mbits/sec
+  * Throughput via VpnCloud (MTU=1400): 885 Mbits/sec
+  * CPU usage for VpnCloud (MTU=1400): ~85%/ ~95% of one core (sender, receiver)
+  * Throughput via VpnCloud (MTU=16384): 947 Mbits/sec
+  * CPU usage for VpnCloud (MTU=16384): ~40%/ ~45% of one core (sender, receiver)
 
 
 ### Test 2: Unencrypted ping
@@ -63,29 +68,106 @@ $> ./vpncloud -t tap -l NODE2:3210 -c NODE1:3210 \
 
 Each test is first run without VpnCloud:
 ```
-$> ping NODE2 -c 1000 -i 0.01 -s SIZE -U -q
+$> ping NODE2 -c 10000 -i 0.001 -s SIZE -U -q
 ```
 
 and then with VpnCloud:
 ```
-$> ping 10.2.1.2 -c 1000 -i 0.01 -s SIZE -U -q
+$> ping 10.2.1.2 -c 10000 -i 0.001 -s SIZE -U -q
 ```
 
+For all the test, the best result out of 5 is selected.
+
 SIZE: 50 bytes
-  * Without VpnCloud: Ø= 200 µs, stddev= 28 µs
-  * With VpnCloud: Ø= 318 µs, stddev= 26 µs
+  * Without VpnCloud: Ø= 164 µs
+  * With VpnCloud: Ø= 433 µs
 
 SIZE: 500 bytes
-  * Without VpnCloud: Ø= 235 µs, stddev= 29 µs
-  * With VpnCloud: Ø= 351 µs, stddev= 26 µs
+  * Without VpnCloud: Ø= 330 µs
+  * With VpnCloud: Ø= 446 µs
 
-SIZE: 1400 bytes
-  * Without VpnCloud: Ø= 303 µs, stddev= 32 µs
-  * With VpnCloud: Ø= 421 µs, stddev= 31 µs
+SIZE: 1000 bytes
+  * Without VpnCloud: Ø= 356 µs
+  * With VpnCloud: Ø= 473 µs
+
+
+### Test 3: Encrypted throughput
+
+Node 1:
+```
+$> ./vpncloud -t tap -l NODE1:3210 -c NODE2:3210 \
+   --ifup 'ifconfig $IFNAME 10.2.1.1/24 mtu MTU up' --shared-key test &
+```
+
+Node 2:
+```
+$> ./vpncloud -t tap -l NODE2:3210 -c NODE1:3210 \
+   --ifup 'ifconfig $IFNAME 10.2.1.2/24 mtu MTU up' --shared-key test &
+$> iperf -s &
+$> top
+```
+
+First, the test is run **without VpnCloud**:
+```
+$> iperf -c NODE2 -t 60
+```
+
+and then **via VpnCloud**:
+```
+$> iperf -c 10.2.1.2 -t 60
+```
+
+**Results:**
+  * Throughput without VpnCloud: 926 Mbits/sec
+  * Throughput via VpnCloud (MTU=1400): 633 Mbits/sec
+  * CPU usage for VpnCloud (MTU=1400): 100% of one core on both sides
+  * Throughput via VpnCloud (MTU=16384): 918 Mbits/sec
+  * CPU usage for VpnCloud (MTU=16384): ~90% of one core on both sides
+
+
+### Test 4: Encrypted ping
+
+Node 1:
+```
+$> ./vpncloud -t tap -l NODE1:3210 -c NODE2:3210 \
+   --ifup 'ifconfig $IFNAME 10.2.1.1/24 mtu 1400 up' --shared-key test &
+```
+
+Node 2:
+```
+$> ./vpncloud -t tap -l NODE2:3210 -c NODE1:3210 \
+   --ifup 'ifconfig $IFNAME 10.2.1.2/24 mtu 1400 up' --shared-key test &
+```
+
+Each test is first run without VpnCloud:
+```
+$> ping NODE2 -c 10000 -i 0.001 -s SIZE -U -q
+```
+
+and then with VpnCloud:
+```
+$> ping 10.2.1.2 -c 10000 -i 0.001 -s SIZE -U -q
+```
+
+For all the test, the best result out of 5 is selected.
+
+SIZE: 50 bytes
+  * Without VpnCloud: Ø= 164 µs
+  * With VpnCloud: Ø= 456 µs
+
+SIZE: 500 bytes
+  * Without VpnCloud: Ø= 330 µs
+  * With VpnCloud: Ø= 492 µs
+
+SIZE: 1000 bytes
+  * Without VpnCloud: Ø= 356 µs
+  * With VpnCloud: Ø= 543 µs
 
 
 ### Conclusion
 
-* VpnCloud achieves about 460 MBit/s with default MTU settings.
-* At increased MTU, VpnCloud is able to saturate a Gigabit link.
+* VpnCloud achieves about 885 MBit/s with default MTU settings.
+* In encrypted mode, VpnCloud reaches aboud 663 MBit/s with default MTU settings.
+* At increased MTU, VpnCloud is able to saturate a Gigabit link even when encrypting.
 * VpnCloud adds about 120µs to the round trip times, i.e. 60µs latency increase.
+* Encryption adds an additional latency between 10µs and 35µs depending on the packet size.