From 67c1e233416547f92a22518d8eb6ce22a6dd438e Mon Sep 17 00:00:00 2001 From: Joseph Henry Date: Tue, 14 Jun 2016 16:36:57 -0700 Subject: [PATCH] updated formatting and organization --- src/README.md | 137 ++++++++++++++++++++++++++------------------------ 1 file changed, 70 insertions(+), 67 deletions(-) diff --git a/src/README.md b/src/README.md index dbbc63a..f4ab0a9 100644 --- a/src/README.md +++ b/src/README.md @@ -9,54 +9,6 @@ More discussion can be found in our [original blog announcement](https://www.zer The SDK is currently in **BETA** and is suitable for testing and experimentation. Only Linux is supported. Future updates will focus on compatibility, full stack support, and improved performance, and may also port to other OSes. -# Limitations and Compatibility - -The beta version of the SDK **only supports IPv4**. There is no IPv6 support and no support for ICMP (or RAW sockets). That means network-containerizing *ping* won't work. - -The virtual TCP/IP stack will respond to *incoming* ICMP ECHO requests, which means that you can ping it from another host on the same ZeroTier virtual network. This is useful for testing. - -## Controlling traffic - -**Network Containers are currently all or nothing.** If engaged, the intercept library intercepts all network I/O calls and redirects them through the new path. A network-containerized application cannot communicate over the regular network connection of its host or container or with anything else except other hosts on its ZeroTier virtual LAN. Support for optional "fall-through" to the host IP stack for outgoing connections outside the virtual network and for gateway routes within the virtual network is planned. (It will be optional since in some cases total network isolation might be considered a nice security feature.) - -#### Compatibility Test Results - -The following applications have been tested and confirmed to work for the beta release: - -Fedora 23: - - httpstub.c - nginx 1.8.0 - http 2.4.16, 2.4.17 - darkhttpd 1.11 - python 2.7.10 (python -m SimpleHTTPServer) - python 3.4.3 (python -m http.server) - redis 3.0.4 - node 6.0.0-pre - sshd - -CentOS 7: - - httpstub.c - nginx 1.6.3 - httpd 2.4.6 (debug mode -X) - darkhttpd 1.11 - node 4.2.2 - redis 2.8.19 - sshd - -Ubuntu 14.04.3: - - httpstub.c - nginx 1.4.6 - python 2.7.6 (python -m SimpleHTTPServer) - python 3.4.0 (python -m http.server) - node 5.2.0 - redis 2.8.4 - sshd - -It is *likely* to work with other things but there are no guarantees. - # Building the SDK The SDK works on Linux and has been lightly tested on OSX. To build the service host, IP stack, and intercept library, from the base of the ZeroTier One tree run: @@ -64,20 +16,20 @@ The SDK works on Linux and has been lightly tested on OSX. To build the service make clean make netcon -This will build a binary called *zerotier-netcon-service* and a library called *libzerotierintercept.so*. It will also build the IP stack as *netcon/liblwip.so*. +This will build a binary called `zerotier-sdk-service` and a library called `libztintercept.so`. It will also build the IP stack as `src/liblwip.so`. -To enable debug trace statements for Network Containers, use *-D\_NETCON\_DEBUG* +To enable debug trace statements for Network Containers, use `-D\_SDK\_DEBUG` -The *zerotier-netcon-service* binary is almost the same as a regular ZeroTier One build except instead of creating virtual network ports using Linux's */dev/net/tun* interface, it creates instances of a user-space TCP/IP stack for each virtual network and provides RPC access to this stack via a Unix domain socket. The latter is a library that can be loaded with the Linux *LD\_PRELOAD* environment variable or by placement into */etc/ld.so.preload* on a Linux system or container. Additional magic involving nameless Unix domain socket pairs and interprocess socket handoff is used to emulate TCP sockets with extremely low overhead and in a way that's compatible with select, poll, epoll, and other I/O event mechanisms. +The `zerotier-sdk-service` binary is almost the same as a regular ZeroTier One build except instead of creating virtual network ports using Linux's `/dev/net/tun` interface, it creates instances of a user-space TCP/IP stack for each virtual network and provides RPC access to this stack via a Unix domain socket. The latter is a library that can be loaded with the Linux `LD\_PRELOAD` environment variable or by placement into `/etc/ld.so.preload` on a Linux system or container. Additional magic involving nameless Unix domain socket pairs and interprocess socket handoff is used to emulate TCP sockets with extremely low overhead and in a way that's compatible with select, poll, epoll, and other I/O event mechanisms. -The intercept library does nothing unless the *ZT\_NC\_NETWORK* environment variable is set. If on program launch (or fork) it detects the presence of this environment variable, it will attempt to connect to a running *zerotier-netcon-service* at the specified Unix domain socket path. +The intercept library does nothing unless the `ZT\_NC\_NETWORK` environment variable is set. If on program launch (or fork) it detects the presence of this environment variable, it will attempt to connect to a running `zerotier-sdk-service` at the specified Unix domain socket path. -Unlike *zerotier-one*, *zerotier-netcon-service* does not need to be run with root privileges and will not modify the host's network configuration in any way. It can be run alongside *zerotier-one* on the same host with no ill effect, though this can be confusing since you'll have to remember the difference between "real" host interfaces (tun/tap) and network containerized endpoints. The latter are completely unknown to the kernel and will not show up in *ifconfig*. +Unlike `zerotier-one`, `zerotier-sdk-service` does not need to be run with root privileges and will not modify the host's network configuration in any way. It can be run alongside `zerotier-one` on the same host with no ill effect, though this can be confusing since you'll have to remember the difference between "real" host interfaces (tun/tap) and network containerized endpoints. The latter are completely unknown to the kernel and will not show up in `ifconfig`. # Modes of operation -There are generally two ways one might want to use this SDK/service. The first approach is a compile-time static linking of our SDK/service directly into your application. With this option you can bundle our entire functionality right into your app with no need to communicate with a service externally, it'll all be handled automatically. The second is a service-oriented approach where our SDK is dynamically-linked into your applications upon startup and will communicate to a single ZeroTier service on the host. This can be useful if you've already compiled your applications and can't perform a static linking. +There are generally two ways one might want to use this SDK/service. The first approach is a *compile-time static linking* of our SDK/service directly into your application. With this option you can bundle our entire functionality right into your app with no need to communicate with a service externally, it'll all be handled automatically. The second is a service-oriented approach where our SDK is *dynamically-linked* into your applications upon startup and will communicate to a single ZeroTier service on the host. This can be useful if you've already compiled your applications and can't perform a static linking. ![Image](../docs/img/methods.png) @@ -102,12 +54,12 @@ Run application You don't need Docker or any other container engine to try Network Containers. A simple test can be performed in user space (no root) in your own home directory. -First, build the netcon service and intercept library as described above. Then create a directory to act as a temporary ZeroTier home for your test netcon service instance. You'll need to move the *liblwip.so* binary that was built with *make netcon* into there, since the service must be able to find it there and load it. +First, build the netcon service and intercept library as described above. Then create a directory to act as a temporary ZeroTier home for your test netcon service instance. You'll need to move the `liblwip.so` binary that was built with `make netcon` into there, since the service must be able to find it there and load it. mkdir /tmp/netcon-test-home cp -f ./netcon/liblwip.so /tmp/netcon-test-home -Now you can run the service (no sudo needed, and *-d* tells it to run in the background): +Now you can run the service (no sudo needed, and `-d` tells it to run in the background): ./zerotier-netcon-service -d -p8000 /tmp/netcon-test-home @@ -115,7 +67,7 @@ As with ZeroTier One in its normal incarnation, you'll need to join a network fo ./zerotier-cli -D/tmp/netcon-test-home join 8056c2e21c000001 -If you don't want to use [Earth](https://www.zerotier.com/public.shtml) for this test, replace 8056c2e21c000001 with a different network ID. The *-D* option tells *zerotier-cli* not to look in /var/lib/zerotier-one for information about a running instance of the ZeroTier system service but instead to look in */tmp/netcon-test-home*. +If you don't want to use [Earth](https://www.zerotier.com/public.shtml) for this test, replace 8056c2e21c000001 with a different network ID. The `-D` option tells `zerotier-cli` not to look in `/var/lib/zerotier-one` for information about a running instance of the ZeroTier system service but instead to look in `/tmp/netcon-test-home`. Now type: @@ -123,15 +75,15 @@ Now type: Try it a few times until you see that you've successfully joined the network and have an IP address. Instead of a *zt#* device, a path to a Unix domain socket will be listed for the network's port. -Now you will want to have ZeroTier One (the normal *zerotier-one* build, not network containers) running somewhere else, such as on another Linux system or VM. Technically you could run it on the *same* Linux system and it wouldn't matter at all, but many people find this intensely confusing until they grasp just what exactly is happening here. +Now you will want to have ZeroTier One (the normal `zerotier-one` build, not network containers) running somewhere else, such as on another Linux system or VM. Technically you could run it on the *same* Linux system and it wouldn't matter at all, but many people find this intensely confusing until they grasp just what exactly is happening here. On the other Linux system, join the same network if you haven't already (8056c2e21c000001 if you're using Earth) and wait until you have an IP address. Then try pinging the IP address your netcon instance received. You should see ping replies. -Back on the host that's running *zerotier-netcon-service*, type *ip addr list* or *ifconfig* (ifconfig is technically deprecated so some Linux systems might not have it). Notice that the IP address of the network containers endpoint is not listed and no network device is listed for it either. That's because as far as the Linux kernel is concerned it doesn't exist. +Back on the host that's running `zerotier-sdk-service`, type `ip addr list` or `ifconfig` (ifconfig is technically deprecated so some Linux systems might not have it). Notice that the IP address of the network containers endpoint is not listed and no network device is listed for it either. That's because as far as the Linux kernel is concerned it doesn't exist. What are you pinging? What is happening here? -The *zerotier-netcon-service* binary has joined a *virtual* network and is running a *virtual* TCP/IP stack entirely in user space. As far as your system is concerned it's just another program exchanging UDP packets with a few other hosts on the Internet and nothing out of the ordinary is happening at all. That's why you never had to type *sudo*. It didn't change anything on the host. +The `zerotier-sdk-service` binary has joined a *virtual* network and is running a *virtual* TCP/IP stack entirely in user space. As far as your system is concerned it's just another program exchanging UDP packets with a few other hosts on the Internet and nothing out of the ordinary is happening at all. That's why you never had to type *sudo*. It didn't change anything on the host. Now you can run an application inside your network container. @@ -145,7 +97,7 @@ If you don't have node.js installed, an alternative test using python would be: python -m SimpleHTTPServer 80 -If you are running Python 3, use "-m http.server". +If you are running Python 3, use `-m http.server`. If all went well a small static HTTP server is now serving up the current directory, but only inside the network container. Going to port 80 on your machine won't work. To reach it, go to the other system where you joined the same network with a conventional ZeroTier instance and try: @@ -155,9 +107,9 @@ Replace *NETCON.INSTANCE.IP* with the IP address that *zerotier-netcon-service* # Installing in a Docker container (or any other container engine) -If it's not immediately obvious, installation into a Docker container is easy. Just install *zerotier-netcon-service*, *libzerotierintercept.so*, and *liblwip.so* into the container at an appropriate locations. We suggest putting it all in */var/lib/zerotier-one* since this is the default ZeroTier home and will eliminate the need to supply a path to any of ZeroTier's services or utilities. Then, in your Docker container entry point script launch the service with *-d* to run it in the background, set the appropriate environment variables as described above, and launch your container's main application. +If it's not immediately obvious, installation into a Docker container is easy. Just install `zerotier-sdk-service`, `libztintercept.so`, and `liblwip.so` into the container at an appropriate locations. We suggest putting it all in `/var/lib/zerotier-one` since this is the default ZeroTier home and will eliminate the need to supply a path to any of ZeroTier's services or utilities. Then, in your Docker container entry point script launch the service with *-d* to run it in the background, set the appropriate environment variables as described above, and launch your container's main application. -The only bit of complexity is configuring which virtual network to join. ZeroTier's service automatically joins networks that have *.conf* files in *ZTHOME/networks.d* even if the *.conf* file is empty. So one way of doing this very easily is to add the following commands to your Dockerfile or container entry point script: +The only bit of complexity is configuring which virtual network to join. ZeroTier's service automatically joins networks that have `.conf` files in `ZTHOME/networks.d` even if the `.conf` file is empty. So one way of doing this very easily is to add the following commands to your Dockerfile or container entry point script: mkdir -p /var/lib/zerotier-one/networks.d touch /var/lib/zerotier-one/networks.d/8056c2e21c000001.conf @@ -176,9 +128,9 @@ Now each new instance of your container will automatically join the specified ne # Docker-based Unit Tests -Each unit test will temporarily copy all required ZeroTier binaries into its local directory, then build the *netcon_dockerfile* and *monitor_dockerfile*. Once built, each container will be run and perform tests and monitoring specified in *netcon_entrypoint.sh* and *monitor_entrypoint.sh* +Each unit test will temporarily copy all required ZeroTier binaries into its local directory, then build the `netcon_dockerfile` and `monitor_dockerfile`. Once built, each container will be run and perform tests and monitoring specified in `netcon_entrypoint.sh` and `monitor_entrypoint.sh` -Results will be written to the *netcon/docker-test/_results/* directory which is a common shared volume between all containers involved in the test and will be a combination of raw and formatted dumps to files whose names reflect the test performed. In the event of failure, *FAIL.* will be prepended to the result file's name (e.g. *FAIL.my_application_1.0.2.x86_64*), likewise in the event of success, *OK.* will be prepended. +Results will be written to the `tests/docker-test/_results/` directory which is a common shared volume between all containers involved in the test and will be a combination of raw and formatted dumps to files whose names reflect the test performed. In the event of failure, `FAIL.` will be prepended to the result file's name (e.g. `FAIL.my_application_1.0.2.x86_64`), likewise in the event of success, `OK.` will be prepended. To run unit tests: @@ -215,10 +167,10 @@ To run unit tests: cp /tmp/netcon_second/identity.secret ./netcon/docker-test/monitor_identity.secret -5) Place a blank network config file in the *netcon/docker-test* directory (e.g. "8056c2e21c000001.conf") +5) Place a blank network config file in the `netcon/docker-test` directory (e.g. "8056c2e21c000001.conf") - This will be used to inform test-specific scripts what network to use for testing -After you've created your network and placed its blank config file in *netcon/docker-test* run the following to perform unit tests for httpd: +After you've created your network and placed its blank config file in `tests/docker-test` run the following to perform unit tests for httpd: ./build.sh httpd ./test.sh httpd @@ -233,3 +185,54 @@ iOS (NSStream) | NO | YES | iOS (BSD socket) | YES | NO | Android (socket) | Not reliably | NO | Android (Socket) | Not reliably | YES | + +# Limitations and Compatibility + +The beta version of the SDK **only supports IPv4**. There is no IPv6 support and no support for ICMP (or RAW sockets). That means network-containerizing *ping* won't work. + +The virtual TCP/IP stack will respond to *incoming* ICMP ECHO requests, which means that you can ping it from another host on the same ZeroTier virtual network. This is useful for testing. + +### Controlling traffic + +**Network Containers are currently all or nothing.** If engaged, the intercept library intercepts all network I/O calls and redirects them through the new path. A network-containerized application cannot communicate over the regular network connection of its host or container or with anything else except other hosts on its ZeroTier virtual LAN. Support for optional "fall-through" to the host IP stack for outgoing connections outside the virtual network and for gateway routes within the virtual network is planned. (It will be optional since in some cases total network isolation might be considered a nice security feature.) + +The exception to this rule is if you use a network library in your application that supports the use of a SOCKS5 proxy and if you configure your network library to use the proxy service provided by the ZeroTier service you can disable all other shims and only talk to ZeroTier virtual networks via the proxied connections you specifically set up. + +#### Compatibility Test Results + +The following applications have been tested and confirmed to work for the beta release: + +Fedora 23: + + httpstub.c + nginx 1.8.0 + http 2.4.16, 2.4.17 + darkhttpd 1.11 + python 2.7.10 (python -m SimpleHTTPServer) + python 3.4.3 (python -m http.server) + redis 3.0.4 + node 6.0.0-pre + sshd + +CentOS 7: + + httpstub.c + nginx 1.6.3 + httpd 2.4.6 (debug mode -X) + darkhttpd 1.11 + node 4.2.2 + redis 2.8.19 + sshd + +Ubuntu 14.04.3: + + httpstub.c + nginx 1.4.6 + python 2.7.6 (python -m SimpleHTTPServer) + python 3.4.0 (python -m http.server) + node 5.2.0 + redis 2.8.4 + sshd + +It is *likely* to work with other things but there are no guarantees. +