How the VPN works


OpenConnect VPN server is an Internet-layer VPN server. That is, it provides the client with an IP address and a list of routes that this IP may access. Its design takes a different path than other open source VPN servers. It delegates several protocol related-items considered today as core VPN features and diversifiers, to standards protocols, and proven system components. That delegation, moves the need for audit due to cryptographic and negotiation protocols to dedicated for that purpose components, enabling a minimal VPN core.

In particular OpenConnect VPN server utilizes standard protocols such as HTTP, TLS and DTLS to provide data security and authenticity. A high level description of OpenConnect and its history was presented in FOSDEM 2016, and an elaborate, description of the protocol has been submitted at IETF as an informational draft. An informal high level description of the protocol used by OpenConnect VPN server can be found in this blog post.

Setting the VPN up

The server utilizes a single configuration file, typically present in /etc/ocserv/ocserv.conf. The options in the file are documented, however, the only options that require modification in case to have a server running with the default settings are the subnet addresses, specified by the 'ipv4-network' and 'ipv4-netmask' configuration options. The routes that are pushed to the client are specified by the 'route' option. For each client two IPv4 addresses are assigned, its VPN address and its local image (remember this is a point-to-point connection). The image isn’t known to the client, as the openconnect protocol doesn’t forward it, but on recent ocserv releases it is the first address of the provided network (e.g., for

In order to allow high-speed transfers ocserv doesn’t do any packet forwarding or filtering between the networks. It is expected that the server has any required routes or firewall rules, set up. You may conditionally enable firewall rules, or even enable routing rules through the client using the 'connect-script' and 'disconnect-script' scripts based on the user who connected. Note that it is important for these scripts not to hang, and terminate without long delays.

See the documentation recipes for more information on setting up openconnect server.

The management interface

When deploying a server to handle several users, it is necessary to have a user management interface, not only to address potential user issues, but also to manage connected users, i.e., disconnect users who no longer have access and so on. The interface is provided by the occtl tool, which can be used either interactively, or as a back-end to another interface, as it can provide its output to JSON format. See the occtl manpage for more information.

Basic authentication

Basic password authentication in openconnect VPN server occurs in the initial HTTP over TLS secured session. In that session the client is provided with an XML authentication page. The server is authenticated using its certificate and the client, either by its certificate, or via a username and password pair which are forwarded to PAM, or a combination of both password and certificate. Because PAM supports various authentication types, the password entered by the user could be a one-time-password or whatever else supported by PAM. After the user is authenticated he is provided with a cookie that can be used for future connections. The lifetime of the cookie is configurable using the 'cookie-timeout' option.

After the user is authenticated, directly, or via the cookie, he issues a CONNECT HTTP command over the established TLS session which results to a direct connection with the VPN. Additionally the user could connect using UDP and Datagram TLS (DTLS) to a port provided by the server. That connection is authenticated either using DTLS with preshared keys with a key derived from the original TLS session, or in early versions, with session resumption and a master key provided by the server.

Security mechanisms

Currently, the server utilizes two privileged processes, the “master”, that spawns the unprivileged processes that handle all interactions with clients, and the “security module” process that performs all operations with the server private key. The separation between the two privileged processes is to prevent any leakage of the private key, or user credentials in the unprivileged processes. The unprivileged processes are sandboxed using seccomp, and can only access operations on the keys, rather than the keys themselves, as well as authenticate users via remote procedure calls.

VPN channel re-establishment using cookies

In typical day-to-day work-flows users roam to different network segments, or suspend and resume their equipment, e.g., switch from cable to wifi, have their tablets/mobiles suspend, thus fast VPN reconnection is a necessary feature to accommodate them. OpenConnect utilizes a “cookie” sent to the client as an authentication token to be used for quick roaming, e.g., when a user switches networks, or suspends his laptop and reconnects within the cookie’s validity time. The actual validity time of the cookie is the active session time (i.e., a cookie is always valid when the session is active), plus the ‘cookie-timeout’ value set in the server’s configuration file.

During the cookie’s validity time a user can re-establish the session associated with it and any session using that cookie is teared-down.

VPN channel operation

The main VPN channel is established over TCP and TLS. This is the control channel as well as the data backup channel. After its establishment a UDP channel using DTLS is initiated which serves as the main data channel. If the UDP channel fails to establish or is temporarily unavailable the backup channel over TCP/TLS is being used.


This server allows stateless compression of independent IP packets to reduce the transferred data size. The supported algorithms are LZS and LZ4.

Note, that the supported compression is intentionally stateless to prevent IP packets from unrelated sources to affect the size of each other. This will not prevent all plaintext recovery attacks which take advantage of compression, but will avoid easy exploitation over the VPN setting. The critical factor when using compression to recover the plaintext of encrypted data, is whether an attacker’s data can be combined with legitimate data. If unsure on the value and nature of data transmitted, it is always safe to keep compression disabled (the default on this server).

Design overview