Securing Go HTTP webservers

May 24, 2020 - Jan Pieter Bruins Slot

Introduction

For a project I am working on, I wanted to be able to expose the http server implementation in Go to the internet. After listening to the Go Time podcast and specifically the episode 101¹, I wanted to go over the common measures you can employ to make your Go HTTP webserver more secure. But before we begin:

DISCLAIMER: Web security is sensitive and vast subject, and I want to make sure that you do own research on how to set up a secure HTTP webserver. So don’t rely on this post solely, and always do some rigorous testing. To make it a bit easier to find resources I’ve added references and footnotes to the post. So I implore you to read sources to get a better understanding of the subject. Additionally, some measures taken here will not always be supported by all clients, such as browsers. This is because in this post I tried to use the latest versions and recommendations, and these are not always supported.

Since we’ve got that out of the way, let’s begin.

Simple webserver

So, let’s start with creating a simple HTTP webserver, you’ve probably come across this one several times.²

// server.go
package main

import (
    "log"
    "net/http"
)

func main() {
    http.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) {
        w.Write([]byte("hello, world\n"))
    })

    log.Fatal(http.ListenAndServe(":http", nil))
}

Let’s run it and check if it works:

$ go run server.go
$ curl http://localhost
hello world

Cool, we have a starting point. Let’s add our first security measures.

Timeouts / Limits

To make our HTTP webserver more secure we have the option to set some timeouts and limits for it. This will make sure that connections aren’t stuck, not making progress, or are stalling. To help us with setting some defaults We will be checking out the article written by Filippo Valsorda: “So you want to expose Go on the Internet” (Valsorda 2016) to guide us.

When we implement those defaults, the end result will resemble something like this:

// server.go
package main

import (
    "log"
    "net/http"
    "time"
)

func main() {
    srv := &http.Server{
        Addr: ":http",
        Handler: http.HandlerFunc(
            func(w http.ResponseWriter, r *http.Request) {
                r.Body = http.MaxBytesReader(w, r.Body, 1<<20)
                w.Write([]byte("hello, world\n"))
            },
        ),
        ReadTimeout:       5 * time.Second,
        WriteTimeout:      10 * time.Second,
        IdleTimeout:       120 * time.Second,
        ReadHeaderTimeout: 5 * time.Second,
        MaxHeaderBytes:    1 << 20,
    }

    log.Fatal(srv.ListenAndServe())
}

So, let’s go over it and investigate what fields we need to set.

ReadTimeout:       5 * time.Second,
WriteTimeout:      10 * time.Second,
IdleTimeout:       120 * time.Second,
ReadHeaderTimeout: 5 * time.Second,

The timeout ReadTimeout covers, from the server perspective, when the connection is established to when the request from the client is read. The timeout WriteTimeout covers the duration of the server writing its response. The timeout IdleTimeout covers the duration to wait for a new connection when ‘keep-alive’ is enabled. And the ReadHeaderTimeout covers the time that the request header (from the client) is completely read.³

Accept                                                          New request
                                         ┌──────────ServeHTTP─────────────┐
┏━━━━━━┳━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━┳━━━━━━━━━━┳━━━━━━┓
┃ Wait ┃ TLS Handshake ┃ Request Headers ┃ Request Body ┃ Response ┃ Idle ┃
┗━━━━━━┻━━━━━━━━━━━━━━━┻━━━━━━━━━━━━━━━━━┻━━━━━━━━━━━━━━┻━━━━━━━━━━┻━━━━━━┛
│                      └ReadHeaderTimeout┘              │          └──┬───┘
│                                        │              │       IdleTimeout
└──────────────ReadTimeout──────────────────────────────┘          │       
│                                        │                         │       
└┄┄┄┄┄┄┄┄┄┄┄┄┄┄WriteTimeout┄┄┄┄┄┄┄┄┄┄┄┄┄┄┴───────WriteTimeout──────┘       

Above an overview of the timeouts in the request/response cycle, note that when TLS is used, WriteTimeout will include the TLS handshake, as well as the header read as well.

We continue with setting the field MaxHeaderBytes:

MaxHeaderBytes: 1 << 20,  // Resolves to 1048576, and equals 1MB

This field makes sure that there is a maximum of bytes that the server will read when parsing the headers. Note that it does not limit the size of incoming request body. You’ll need to implement http.MaxBytesReader() in your handlers.⁴ And we can implement it as follows:

Handler: http.HandlerFunc(
    func(w http.ResponseWriter, r *http.Request) {
        r.Body = http.MaxBytesReader(w, r.Body, 1<<20)
        io.WriteString(w, "hello, world\n")
    },
),

Remember, that this is just an example, and you should think about the specific timeouts, and limits that serve your use-case.

TLS

To make our HTTP webserver more secure, and expose it to the internet we do well to implement a secure communication between the client and the server. And we do that by leveraging TLS, which stands for: Transport Layer Security. We use it to secure communication between the webserver and the client over the internet. The protocol is implemented at the application layer, directly on top of TCP, which enables protocols ‘above’⁵ it to continue working while leveraging the added communication security. (Grigorik 2013, 48)

First, let’s explore what TLS exactly does, so we get a sense of why we choose certain methods when coding an implementation in Go.

Overview

There are 3 main components to TLS:

1. Encryption: hides the data being transferred from third parties.
2. Authentication: ensures that the parties exchanging information are who they claim to be.
3. Integrity: verifies that the data has not been forged or tampered with.

We hide the data by encrypting it, and we do that by using symmetric cryptography.⁶ This means that a generated key is used for both to encrypt and decrypt the data. The keys used for this are created for each connection, and are based on a shared secret that was negotiated at the start of the session. (Wikipedia 2020f, 2020e) The chosen ‘ciphersuite’ determines details such as, which shared encryption keys, or session keys, will be used for the session. (Cloudflare 2020)

We make sure that the identity of both the server and client is correct by using public-key cryptography. In contrast to symmetric cryptography, this employs asymmetric cryptography where a pair of keys are used: the public and private key. Everyone with the public key is able to encrypt data, but only the holder of the private key is able to decrypt it. In our case that will be the server. (Wikipedia 2020f, 2020d)

This public-key cryptography is used only during session setup of the TLS tunnel. The server provides its public key to the client, and then the client generates a symmetric key, which it encrypts with the server’s public key, and returns the encrypted symmetric key, with which the data between the client and server is encrypted, to the server. Finally, the server can decrypt the symmetric key sent from the client with its own private key. This symmetric key is then used for all further encrypted communication since now both the server and client have the key to encrypt and decrypt data. (Grigorik 2013, 52)

When the data is encrypted and authenticated, it will be signed with a message authentication code (MAC) to ensure its integrity. (Cloudflare 2020) The MAC algorithm is a one-way cryptographic hash function, in a way a sort of checksum. The keys in order to generate the MAC are negotiated between the client and server. When a TLS record is sent, a MAC value is generated and appended to that message. The receiver is then able to compute and verify the MAC value to ensure that the message hasn’t been changed or altered. (Grigorik 2013, 49)

OK, so how is this all implemented? A TLS connection between the server and client will be established by the so-called “TLS Handshake”. This “handshake” creates the agreement on the version of the TLS protocol, the ciphersuite, and certificate verification. In the following diagram we’ll go over the steps of how such a “handshake” is realised.⁷

                                                     
        ┌─┐                             ┌─┐          
        └┬┘                             ╞ │          
        ▔▔▔                             └─┘          
       client                          server        
┏━━━━━━━━━━━━━━━━━━━┓                                
┃ SYN               ┃ ――――――――▶ ┏━━━━━━━━━━━━━━━━━━━┓
┗━━━━━━━━━━━━━━━━━━━┛           ┃ SYN ACK           ┃
┏━━━━━━━━━━━━━━━━━━━┓ ◀―――――――― ┗━━━━━━━━━━━━━━━━━━━┛
┃ ACK               ┃ ――――――――▶ ┏━━━━━━━━━━━━━━━━━━━┓
┣━━━━━━━━━━━━━━━━━━━┫           ┃ ServerHello       ┃
┃ ClientHello       ┃           ┣━━━━━━━━━━━━━━━━━━━┫
┗━━━━━━━━━━━━━━━━━━━┛           ┃ Certificate       ┃
┏━━━━━━━━━━━━━━━━━━━┓ ◀―――――――― ┣━━━━━━━━━━━━━━━━━━━┫
┃ ClientKeyExchange ┃           ┃ ServerHelloDone   ┃
┣━━━━━━━━━━━━━━━━━━━┫           ┗━━━━━━━━━━━━━━━━━━━┛
┃ ChangeCipherSpec  ┃ ――――――――▶ ┏━━━━━━━━━━━━━━━━━━━┓
┣━━━━━━━━━━━━━━━━━━━┫           ┃ ChangeCipherSpec  ┃
┃ Finished          ┃           ┣━━━━━━━━━━━━━━━━━━━┫
┗━━━━━━━━━━━━━━━━━━━┛           ┃ Finished          ┃
┏━━━━━━━━━━━━━━━━━━━┓ ◀―――――――― ┗━━━━━━━━━━━━━━━━━━━┛
┃ Application Data  ┃           ┏━━━━━━━━━━━━━━━━━━━┓
┗━━━━━━━━━━━━━━━━━━━┛ ――――――――▶ ┃ Application Data  ┃
                                ┗━━━━━━━━━━━━━━━━━━━┛

SYN, SYN ACK, ACK - Start with a TCP connection, and a three-way handshake. After this TCP connection, the TLS Handshake begins.

ClientHello - Client sends a ClientHello message and it will include a number of specifications in plain text, such as version of TLS protocol, list of supported ciphersuites (ciphers and hash functions).

ServerHello, Certificate, ServerHelloDone - Server selects the version, and the ciphersuite. It then presents its certificate. This contains: the domain name, the trusted certificate authority (CA), and the public encryption key, and sends its response back to the client.

ClientKeyExchange, ChangeCipherSpec, Finished - Client confirms the validity of the certificate. Then the client generates a new symmetric key, and encrypts that key with the server’s public key.

ChangeCipherSpec, Finished - The server decrypts the symmetric key sent by the client, checks the integrity of the message by verifying the MAC, and returns an encrypted “Finished” message back to the client.

Application Data - The client decrypts the message with the symmetric key it generated earlier, verifies the MAC, and if everything checks out application data can be sent.

So, this gives us a bit of a basic understanding of how TLS works. In the next section we’ll implement it into our HTTP webserver.

Implementation

We’ll be using the crypto/tls package for implementing TLS for our HTTP webserver.⁸ We also take note of the Mozilla SSL Configuration Generator, which allows us to implement the “Modern” recommended configuration made by Mozilla. (Mozilla 2020a) When implemented, our code will look something like this:

// server.go
package main

import (
    "crypto/tls"
    "flag"
    "log"
    "net/http"
    "time"
)

func main() {
    flgCert := flag.String("cert", "cert.pem", "path to cert")
    flgKey := flag.String("key", "key.pem", "path to key")
    flag.Parse()

    cert, err := tls.LoadX509KeyPair(*flgCert, *flgKey)
    if err != nil {
        log.Fatal(err)
    }

    tlsConfig := &tls.Config{
        Certificates:             []tls.Certificate{cert},
        CipherSuites:             nil,
        PreferServerCipherSuites: true,
        MinVersion:               tls.VersionTLS13,
        CurvePreferences: []tls.CurveID{
            tls.CurveP256,
            tls.X25519,
        },
    }

    go func() {
        srv := &http.Server{
            Addr: ":http",
            Handler: http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
                http.Redirect(
                    w, r,
                    "https://"+r.Host+r.URL.RequestURI(),
                    http.StatusMovedPermanently,
                )
            }),
        }
        log.Fatal(srv.ListenAndServe())
    }()

    srv := &http.Server{
        Addr: ":https",
        Handler: http.HandlerFunc(
            func(w http.ResponseWriter, r *http.Request) {
                r.Body = http.MaxBytesReader(w, r.Body, 1<<20)
                w.Write([]byte("hello, world\n"))
            },
        ),
        TLSConfig: tlsConfig,
        TLSNextProto: make(
            map[string]func(*http.Server, *tls.Conn, http.Handler), 0,
        ),
        ReadTimeout:    5 * time.Second,
        WriteTimeout:   10 * time.Second,
        IdleTimeout:    120 * time.Second,
        MaxHeaderBytes: 1 << 20,
    }

    log.Fatal(srv.ListenAndServeTLS("", ""))
}

Again, let’s go over the code and what investigate what we’ve changed.

flgCert := flag.String("cert", "cert.pem", "path to cert")
flgKey := flag.String("key", "key.pem", "path to key")
flag.Parse()

Here we’re parsing some command line arguments that we will use later on. Eventually, we’ll create self-signed certificates, whose paths we pass into our program here.

cert, err := tls.LoadX509KeyPair(*flgCert, *flgKey)
if err != nil {
    log.Fatal(err)
}

The LoadX509KeyPair() function reads and parses a public/private key pair from a pair of files. Both files must be ‘PEM’ (Privacy-Enhanced Mail) encoded.⁹

tlsConfig := tls.Config{
    // ...
}

Here we create the tls.Config struct that we use to configure our TLS functionality for our HTTP webserver.¹⁰

Certificates: []tls.Certificate{cert},

To the Certificates field we add the certificates we’ve loaded with the parsed public/private key pair that we provided.

CipherSuites: nil,

The field CipherSuites is a list of ciphersuites that the server will support. This is up to TLS version 1.2, when using version 1.3 this isn’t configurable. Because we want to use the most up to date version we keep it empty, which results in a default list of ciphersuites to be used with a preference order based on hardware performance.

PreferServerCipherSuites: true,

This boolean controls the server’s preferred ciphersuite to use, as provided by the CipherSuites, when false it will select the client’s preferred ciphersuite. Setting this will ensure that safer and faster ciphersuites are used. (Valsorda 2016)

MinVersion: tls.VersionTLS13,

Here we can specify the minimal TLS version the server is going to use. We’ll select version recommended by the “Modern” configuration from Mozilla. (Mozilla 2020a)

CurvePreferences: []tls.CurveID{
    tls.CurveP256,
    tls.X25519,
},

The array CurvePreferences contain the elliptic curves that will be used in an ECDHE handshake. We again use the Mozilla recommendation, however without tls.CurveP384, because a client using tls.CurveP384 would cause up to a second of CPU to be consumed on the server. (Valsorda 2016)

go func() {
    srv := &http.Server{
        Addr: ":http",
        Handler: http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
            http.Redirect(
                w, r,
                "https://"+r.Host+r.URL.RequestURI(),
                http.StatusMovedPermanently,
            )
        }),
    }
    log.Fatal(srv.ListenAndServe())
}()

Here, we create, and run an additional http.Server in a goroutine that will redirect all ‘http’ traffic to our ‘https’ port.

Addr: ":https",

For our ‘main’ http.Server we update the port number and set it to :https to resemble that we’re communicating on port 443.

TLSConfig: tlsConfig,

Here we load our TLS configuration we’ve defined before.

TLSNextProto: make(
    map[string]func(*http.Server, *tls.Conn, http.Handler), 0,
),

Setting TLSNextProto to an empty map will disable HTTP/2 for this server. If you want to enable HTTP/2 set it to nil or remove the field. Since Go 1.6 it is enabled by default. (Valsorda 2016)

log.Fatal(srv.ListenAndServeTLS("", ""))

Because we’re now using TLS, we start the server with ListenAndServeTLS(). It will then listen on the serv.Addr for incoming TLS connections. Because we already set the paths of the key and cert in the tlsConfig, we can set certFile and keyFile arguments to empty strings.

We’ve added TLS functionality to our HTTP webserver, but we’re still missing one part of the TLS components: the certificate.

Certificates

We’re almost ready to test out the program, but we need have the cert and key file. As was mentioned before with the 3 main components TLS consists of we need to ensure the authentication between the server and the client. Meaning that the parties exchanging information, are who they claim to be. This is employed by using public-key cryptography, with the creation of a public and private key. Where everyone with the public key is able to encrypt data, but only the holder of the private key is able to decrypt it.

So what methods are there to trust someone?

1. Manually specified certificates, in which you manually import any certificate you trust.
2. A Certificate Authority (CA), where a third party generates the certificate that both parties trust.
3. The operating system also contains a list with well-known certificate authorities that it trusts.

The most common solution is to make use of a ‘Certificate Authority’. However, first we’re going to create a self-signed certificate, because using a CA to generate a certificate will come with some costs, because you’ll be using a service to verify, audit, and revoke when a certificate is misused or compromised. And it’ll give us a bit more information on how such a thing is generated. (Grigorik 2013, 59)

By checking out the “Modern” configuration again from Mozilla, we can see that it states that the certificate type should be ‘ECDSA (P-256)’¹¹, which stand for elliptical curve digital signature algorithm, and that the maximum certificate lifespan should be 90 days. (Mozilla 2020a)

We’ll be using the openssl req command for this from the commandline, which is a certificate request and certificate generating utility.

# -X509
#   Output a self-signed certificate instead of a certificate request.
#
# -nodes
#   No DES, private key not protected by a passphrase, thus it won't be
#   encrypted.
#
# -newkey ec
#   Create a new certificate request and a new EC private key, usable for
#   both ECDSA or ECDH algorithms
#
# -pkeyopt ec_paramgen_curve:prime256v1
#   Set the public key algorithm ec_paramgen_curve to prime256v1
#
# -keyout key.pem
#   Filename to write the private key to.
#
# -out cert.pem
#   Specifies output filename to write to.
#
# -days 90
#   When using -X509 this specifies the number of days to certify for.

$ openssl req -x509 -nodes -newkey ec -pkeyopt ec_paramgen_curve:prime256v1 \
    -keyout key.pem -out cert.pem -days 90

This will output both the key and cert files in the PEM format, exactly what we needed for tls.Load509KeyPair() function. The ‘X509’ is the format for public-key certificates, an X.509 certificate contains a public key and an identity (a hostname, or an organization, or an individual) (Wikipedia 2020g)

We can then inspect the content of the certificate by again using openssl. By using the openssl x509 command we can look at the contents of a ‘X509’ certificate.

# -text
#   Prints out the certificate in text form.
#
# -in
#   Specifies the input filename to a certificate from. 
#
# -noout
#   Prevents output of the encoded version of the actual certificate.

$ openssl x509 -text -in cert.pem -noout

Certificate:
    Data:
        Version: 3 (0x2)
        Serial Number:
            6b:41:90:1d:ad:c2:bc:0b:9b:61:f5:14:f8:06:92:70:49:d7:99:81
        Signature Algorithm: ecdsa-with-SHA256
        Issuer: C = NL, ST = Some-State, L = Amsterdam, O = Internet Widgits Pty Ltd
        Validity
            Not Before: Apr 15 18:24:08 2020 GMT
            Not After : Jul 14 18:24:08 2020 GMT
        Subject: C = NL, ST = Some-State, L = Amsterdam, O = Internet Widgits Pty Ltd
        Subject Public Key Info:
            Public Key Algorithm: id-ecPublicKey
                Public-Key: (256 bit)
                pub:
                    04:a6:fa:3c:86:99:a9:30:04:65:7f:00:7a:18:e2:
                    bc:4f:b2:69:45:75:b5:a0:06:37:2c:08:d9:2e:d5:
                    bc:a5:e5:14:fb:0a:20:28:b6:c8:e7:30:78:4d:be:
                    7f:5f:12:2c:6e:0a:76:4d:e9:2b:38:c8:9c:cf:d2:
                    db:2b:22:9d:1f
                ASN1 OID: prime256v1
                NIST CURVE: P-256
        X509v3 extensions:
            X509v3 Subject Key Identifier:
                75:BB:CA:93:7A:CF:FB:CF:E5:0A:FB:1C:32:9E:C8:A9:61:D0:76:3F
            X509v3 Authority Key Identifier:
                keyid:75:BB:CA:93:7A:CF:FB:CF:E5:0A:FB:1C:32:9E:C8:A9:61:D0:76:3F

            X509v3 Basic Constraints: critical
                CA:TRUE
    Signature Algorithm: ecdsa-with-SHA256
         30:45:02:21:00:b4:5b:81:65:bd:6e:48:a8:f4:fb:d5:47:1a:
         a0:e2:44:ec:a0:90:7f:b0:9c:de:ba:64:33:d6:98:c3:97:60:
         ba:02:20:26:41:74:a9:bc:c9:78:8b:0e:e3:12:c6:fd:ba:10:
         93:87:cb:8f:4b:15:99:e3:6e:89:d2:f6:b2:e0:48:bb:fc

Here we can see some important information such as the issuer, its validity, and the public key. With this clients knows how to recognize it when this server has signed a message with the corresponding private key.

Testing it out

Now, that we’ve got the certificate ready, let’s test out our program.

$ go run server.go -cert=cert.pem -key=key.pem

When our server is running, we’ll be able to test it with curl. Note that we’re going to add the --insecure flag to our command. This is because we’ve self-signed the certificate and curl rightfully doesn’t trust that. We’re also adding the --verbose flag to output more information about the request.¹²

$ curl --verbose --insecure https://localhost

*   Trying ::1:443...
* Connected to localhost (::1) port 443 (#0)
* ALPN, offering http/1.1
* successfully set certificate verify locations:
*   CAfile: /etc/ssl/certs/ca-certificates.crt
  CApath: none
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.3 (IN), TLS handshake, Encrypted Extensions (8):
* TLSv1.3 (IN), TLS handshake, Certificate (11):
* TLSv1.3 (IN), TLS handshake, CERT verify (15):
* TLSv1.3 (IN), TLS handshake, Finished (20):
* TLSv1.3 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.3 (OUT), TLS handshake, Finished (20):
* SSL connection using TLSv1.3 / TLS_AES_128_GCM_SHA256
* ALPN, server accepted to use http/1.1
* Server certificate:
*  subject: C=NL; ST=Some-State; O=Internet Widgits Pty Ltd
*  start date: Apr  3 09:20:12 2020 GMT
*  expire date: Jul  2 09:20:12 2020 GMT
*  issuer: C=NL; ST=Some-State; O=Internet Widgits Pty Ltd
*  SSL certificate verify result: self signed certificate (18), continuing anyway.
> GET / HTTP/1.1
> Host: localhost:443
> User-Agent: curl/7.52.1
> Accept: */*
>
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* Mark bundle as not supporting multiuse
< HTTP/1.1 200 OK
< Strict-Transport-Security: max-age=63072000
< Date: Fri, 03 Apr 2020 09:20:15 GMT
< Content-Length: 14
< Content-Type: text/plain; charset=utf-8
<
hello, world
* Connection #0 to host localhost left intact

OK, that’s pretty cool. We’re able to see exactly what the request/response cycle is like, including the ‘TLS Handshake’ that we’ve explored in the ‘TLS’ section. You can however see the following line: SSL certificate verify result: self signed certificate (18), continuing anyway. So is there however a possibility to make curl trust our certificate without skipping and/or ignoring the certificate verification? We can with mkcert, and it allows us to create locally-trusted development certificates. It automatically creates and installs a local CA in the system root store, and generates locally-trusted certificates. So let’s create one for localhost.

$ mkcert -install
$ mkcert -ecdsa -key-file=key.pem -cert-file=cert.pem localhost

Now, run restart the server, and make another request with curl without the --insecure flag.

$ curl --verbose https://localhost

*   Trying ::1:443...
* Connected to localhost (127.0.0.1) port 443 (#0)
* ALPN, offering http/1.1
* successfully set certificate verify locations:
*   CAfile: /etc/ssl/certs/ca-certificates.crt
  CApath: none
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.3 (IN), TLS handshake, Encrypted Extensions (8):
* TLSv1.3 (IN), TLS handshake, Certificate (11):
* TLSv1.3 (IN), TLS handshake, CERT verify (15):
* TLSv1.3 (IN), TLS handshake, Finished (20):
* TLSv1.3 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.3 (OUT), TLS handshake, Finished (20):
* SSL connection using TLSv1.3 / TLS_AES_128_GCM_SHA256
* ALPN, server accepted to use http/1.1
* Server certificate:
*  subject: O=mkcert development certificate; OU=jp@aztlan
*  start date: Jun  1 00:00:00 2019 GMT
*  expire date: Apr 15 18:58:07 2030 GMT
*  subjectAltName: host "localhost" matched cert's "localhost"
*  issuer: O=mkcert development CA; OU=jp@aztlan; CN=mkcert jp@aztlan
*  SSL certificate verify ok.
> GET / HTTP/1.1
> Host: localhost:443
> User-Agent: curl/7.52.1
> Accept: */*
>
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* Mark bundle as not supporting multiuse
< HTTP/1.1 200 OK
< Date: Wed, 15 Apr 2020 19:07:29 GMT
< Content-Length: 13
< Content-Type: text/plain; charset=utf-8
<
hello, world
* Connection #0 to host localhost left intact

However, we aren’t able to actually use this in production, since we still used a self-signed certificate. Considering getting a certificate from a CA has some costs associated with it, we’ll create our certificate with ‘Let’s Encrypt’.

Let’s Encrypt

‘Let’s Encrypt’ allows us to create free TLS certificates from nonprofit Certificate Authority. And we’ll be able to implement it in our program with the golang.org/x/crypto/acme/autocert package.¹³ This package automatically generates, and when necessary, renews your certificate. Let’s update our code and implement the autocert functionality.

// server.go
package main

import (
    "crypto/tls"
    "flag"
    "log"
    "net/http"
    "time"

    "golang.org/x/crypto/acme/autocert"
)

func main() {
    flgProd := flag.Bool("prod", false, "run server in production")
    flgHost := flag.String("host", "", "host name of the server")
    flgCert := flag.String("cert", "cert.pem", "path to cert")
    flgKey := flag.String("key", "key.pem", "path to key")
    flag.Parse()

    tlsConfig := &tls.Config{
        CipherSuites:             nil,
        PreferServerCipherSuites: true,
        MinVersion:               tls.VersionTLS13,
        CurvePreferences: []tls.CurveID{
            tls.CurveP256,
            tls.X25519,
        },
    }

    if *flgProd {
        if *flgHost == "" {
            log.Fatal("please specify a host")
        }

        m := autocert.Manager{
            Prompt:     autocert.AcceptTOS,
            HostPolicy: autocert.HostWhitelist(*flgHost),
            Cache:      autocert.DirCache("certs"),
        }

        tlsConfig.GetCertificate = m.GetCertificate

        go func() {
            h := m.HTTPHandler(nil)
            log.Fatal(http.ListenAndServe(":http", h))
        }()
    } else {
        cert, err := tls.LoadX509KeyPair(*flgCert, *flgKey)
        if err != nil {
            log.Fatal(err)
        }

        tlsConfig.Certificates = []tls.Certificate{cert}

        go func() {
            srv := &http.Server{
                Addr: ":http",
                Handler: http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
                    http.Redirect(
                        w, r, "https://"+r.Host+r.URL.RequestURI(),
                        http.StatusMovedPermanently,
                    )
                }),
            }
            log.Fatal(srv.ListenAndServe())
        }()
    }

    srv := &http.Server{
        Addr: ":https",
        Handler: http.HandlerFunc(
            func(w http.ResponseWriter, r *http.Request) {
                r.Body = http.MaxBytesReader(w, r.Body, 1<<20)
                w.Write([]byte("hello, world\n"))
            },
        ),
        TLSConfig: tlsConfig,
        TLSNextProto: make(
            map[string]func(*http.Server, *tls.Conn, http.Handler), 0,
        ),
        ReadTimeout:    5 * time.Second,
        WriteTimeout:   10 * time.Second,
        IdleTimeout:    120 * time.Second,
        MaxHeaderBytes: 1 << 20,
    }

    log.Fatal(srv.ListenAndServeTLS("", ""))
}

Again, let’s go over the code and explore what we’ve changed.

flgProd := flag.Bool("prod", false, "run server in production")
flgHost := flag.String("host", "", "host name of the server")

We’ve added two new flags, flgProd and flgHost. The flag flgProd allows us to signal to the program, by setting the command line flag, that the program is running in production, and should be using the ‘Let’s Encrypt’ certificate created by autocert. The flag flgHost is necessary for autocert to let it know for which host we’re generating a certificate.

if *flgProd {
    if *flgHost == "" {
        log.Fatal("please specify a host")
    }

    m := autocert.Manager{
        Prompt:     autocert.AcceptTOS,
        HostPolicy: autocert.HostWhitelist(*flgHost),
        Cache:      autocert.DirCache("certs"),
    }

    tlsConfig.GetCertificate = m.GetCertificate

    go func() {
        h := m.HTTPHandler(nil)
        log.Fatal(http.ListenAndServe(":http", h))
    }()
} else {
    cert, err := tls.LoadX509KeyPair(*flgCert, *flgKey)
    if err != nil {
        log.Fatal(err)
    }

    tlsConfig.Certificates = []tls.Certificate{cert}

    go func() {
        srv := &http.Server{
            Addr: ":http",
            Handler: http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
                http.Redirect(
                    w, r, "https://"+r.Host+r.URL.RequestURI(),
                    http.StatusMovedPermanently,
                )
            }),
        }
        log.Fatal(srv.ListenAndServe())
    }()
}

We’ve removed the Certificates field from tlsConfig. And with this conditional we choose either by generating a certificate with autocert, or using our self-signed certificate.

m := autocert.Manager{
    Prompt:     autocert.AcceptTOS,
    HostPolicy: autocert.HostWhitelist(*flgHost),
    Cache:      autocert.DirCache("certs"),
}

The field Prompt specifies a function to accept a CA’s Terms of Service, here we choose autocert.AcceptTOS which always accept the terms. The field HostPolicy controls which domains the Manager will attempt to retrieve new certificates for. It accepts a HostPolicy, which specifies which host names the Manager is allowed to respond to. We create that by calling HostWhiteList() with the host name for our server. The field Cache is needed to write our key and certificate to, so that we can keep reusing it with server restarts for the days that the certificate is valid. We also do this because there is a rate-limit for requesting certificates from Let’s Encrypt.¹⁴ We set the directory to "certs" in this example.

tlsConfig.GetCertificate = m.GetCertificate

We specify the GetCertificate field of tls.Config to use the GetCertificate function of the autocert.Manager, because this implements the tls.Config.GetCertificate hook.

go func() {
    h := m.HTTPHandler(nil)
    log.Fatal(http.ListenAndServe(":http", h))
}()

Additionally, we will use the Manager.HTTPHandler from autocert that implements a handler fallback for requests received on port 80, and redirects it to the ‘https’ port.¹⁵

tlsConfig.Certificates = []tls.Certificate{cert}

Because we removed the Certficates field from tlsConfig, and we’re using the autocert.Manager.GetCertificate function when using Let’s Encrypt to get our certificate when we are using Let’s Encrypt. We need to specify our tlsConfig.Certificates field, when we aren’t using ‘Let’s Encrypt’ and using our self-signed certificate.

You can run it locally, but the certificate won’t be valid. Since, we’re running it locally and using localhost, ‘Let’s Encrypt’ isn’t able to create certificates for this because nobody uniquely owns localhost. So local development deployments we continue using our self-signed certificates. (See Let’s Encrypt 2020 for some alternate solutions)

OK, now if you have a production environment and domain name that points to your production environment, you’ll be able to run the server as follows:

$ go run server.go -prod=true -host=yourawesomedomain.org

Secure headers

In order to make our server more secure it is advised that we add some secure headers to our server.¹⁶ This is done in order to mitigate several vulnerabilities. (OWASP 2020b). To our Handler we will add these response headers:

Handler: http.HandlerFunc(
    func(w http.ResponseWriter, r *http.Request) {
        w.Header().Add("Strict-Transport-Security", "max-age=63072000; includeSubdomains")
        w.Header().Add("Content-Security-Policy", "default-src 'self'")
        w.Header().Add("X-XSS-Protection", "1; mode=block")
        w.Header().Add("X-Frame-Options", "DENY")
        w.Header().Add("Referrer-Policy", "strict-origin-when-cross-origin")
        w.Header().Add("X-Content-Type-Options", "nosniff")
        w.Header().Add("Content-Type", "text/plain")

        r.Body = http.MaxBytesReader(w, r.Body, 1<<20)
        w.Write([]byte("hello, world!\n"))
    },
),

Let’s go over the code again, and check what headers we’ve added, what they protect us from, and how we set it.

w.Header().Add("Strict-Transport-Security", "max-age=63072000; includeSubdomains")

Here we set the ‘HSTS’ header (HTTP Strict Transport Security). This will tell that clients should automatically interact with the server using HTTPS connections. By setting the response header we specify the period of time the client should only access the server in a secure fashion. It protects against ‘protocol downgrade attacks’, and ‘cookie hijacking’. (Wikipedia 2020c; OWASP 2020b) The value max-age specifies the time the client should remember that the site should be accessed by HTTPS. The value includeSubdomains implies that this rule applies to all subdomains as well.¹⁷

w.Header().Add("Content-Security-Policy", "default-src 'self'")

The ‘CSP’ header (Content Security Policy), allows you to define a policy of what resources the client is allowed to load/run. It specifies the domains the client should consider valid sources. Let’s say you have a web application that allows content to be loaded within the web application that originates from somewhere else. With CSP you’re able to white list origins for scripts, images, fonts, stylesheets, etc. It protects against certain ‘cross-site scripting attacks’ (XSS), which exploit the clients’ trust of the content received from the server. Malicious scripts can be thus be executed by the client because it trusts the source of the content, even when it not from where it seems to be from. The policy "default-src 'self'" defines that all content should come from the site’s own origin, this excludes subdomains.¹⁸

w.Header().Add("X-XSS-Protection", "1; mode=block")

The ‘X-XSS-Protection’ header stops pages from loading when they detect XSS (Cross-site scripting) attacks. This is where the attacker causes a page to load some malicious JavaScript, and does so by sending the malicious payload as part of the request. With this header specify to load or block the page from loading. The setting "1; mode=block;" will completely block the page from loading. This is a special feature header for the Chrome and Internet Explorer browsers, and seems to be unnecessary for API’s and should be covered by a restrictive Content Security Policy.¹⁹

w.Header().Add("X-Frame-Options", "DENY")

The ‘X-Frame-Options’ header let’s the browser know whether it is allowed to render a page in a <embed>, <frame>, <iframe> or <object> tag. This is used to avoid ‘click-jacking’ such that the content isn’t embedded into other sites. Here we set it to "DENY", which blocks the content from being embedded in other pages.²⁰

w.Header().Add("Referrer-Policy", "strict-origin-when-cross-origin")

The ‘Referrer-Policy’ header lets us define what information is being sent to external sites/resources in the Referer request header. When a client accesses an url from hyperlink, or a webpage loads an external resource, the browser adds the header Referer (yes that is intentionally misspelled), to let the destination know the origin of that request. Imagine that your webpage has a link to an external site, and that external site receives the Referer header with information that should only be used internally. With this header we can control what is being sent to the destination. In this example we will set it to strict-origin-when-cross-origin, which sends the full referer to the sources on the same origin, and url withouth path to external origin destinations, and send no header to less-secure destinations (HTTPS→HTTP).²¹

w.Header().Add("X-Content-Type-Options", "nosniff")

The ‘X-Content-Type-Options’ header indicates the MIME types set by the "Content-Type" header, which should not be changed and be followed. It prevents browsers from interpreting files as something else than what was specified the "Content-Type" header. It opts out of ‘Mime type sniffing’: guessing the correct Mime type by looking at the bytes of the resource done by browser. Without this header, browsers can incorrectly detect files as scripts and stylesheets, leading to XSS attacks. Setting the header to "nosniff" indicates that browsers should prevent incorrectly detecting non-scripts as scripts.²²

w.Header().Add("Content-Type", "text/plain; charset=UTF-8")

In order to make ‘X-Content-Type-Options’ to work correctly, we need to set the ‘Content-Type’ to the correct MIME type. For this example we’ll set it to "text/plain; charset=UTF-8". This tells the client that the type is text, the subtype is plain, and that the character encoding is utf-8.²³

In this section we’ve set up some response headers that will mitigate some common vulnerabilities.²⁴ For brevity, I’ve added the response header to the Handler for a single response. In an actual implementation you would likely add this to a sort of middleware for your handlers.²⁵

Again, we can test out our HTTP webserver:

$ go run server.go -cert=cert.pem -key=key.pem
$ curl --verbose --insecure https://localhost

*   Trying ::1:443...
* Connected to localhost (::1) port 443 (#0)
* ALPN, offering http/1.1
* successfully set certificate verify locations:
*   CAfile: /etc/ssl/certs/ca-certificates.crt
  CApath: none
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.3 (IN), TLS handshake, Encrypted Extensions (8):
* TLSv1.3 (IN), TLS handshake, Certificate (11):
* TLSv1.3 (IN), TLS handshake, CERT verify (15):
* TLSv1.3 (IN), TLS handshake, Finished (20):
* TLSv1.3 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.3 (OUT), TLS handshake, Finished (20):
* SSL connection using TLSv1.3 / TLS_AES_128_GCM_SHA256
* ALPN, server accepted to use http/1.1
* Server certificate:
*  subject: O=mkcert development certificate; OU=jp@aztlan
*  start date: Jun  1 00:00:00 2019 GMT
*  expire date: Apr 15 18:58:07 2030 GMT
*  subjectAltName: host "localhost" matched cert's "localhost"
*  issuer: O=mkcert development CA; OU=jp@aztlan; CN=mkcert jp@aztlan
*  SSL certificate verify ok.
> GET / HTTP/1.1
> Host: localhost
> User-Agent: curl/7.52.1
> Accept: */*
>
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* Mark bundle as not supporting multiuse
< HTTP/1.1 200 OK
< Content-Security-Policy: default-src 'self'
< Content-Type: text/plain
< Referrer-Policy: strict-origin-when-cross-origin
< Strict-Transport-Security: max-age=63072000; includeSubdomains
< X-Content-Type-Options: nosniff
< X-Frame-Options: DENY
< X-Xss-Protection: 1; mode=block
< Date: Sun, 24 May 2020 14:21:49 GMT
< Content-Length: 13
<
hello, world
* Connection #0 to host localhost left intact

And, now you can see that we’ve implemented the secure headers.

CORS / CSRF

In this post we’ve added timeouts, a cryptographic protocol, and secure headers to our HTTP webserver. And depending on the size, utility, and what the server is going to be used for, you likely want to add additional security measures such as CORS, and CSRF. Let’s check out what they are, and investigate at what point we want to implement those measures.

CORS

CORS²⁶ (Cross-Origin Resource Sharing) specifies which cross-domain requests for a server are allowed from a webbrowser. Meaning: are request to, for instance, an api allowed from somewhere else than from the domain of the original api. One would likely disable CORS header, when cross-domain calls are not supported or expected. (OWASP 2020c)

CORS are disabled by default by your browser, meaning you can’t access resources from different origins. As a server you specify to the client which domains (origins), are allowed to make requests. It allows the client’s browser to determine if and when cross-domain requests should be allowed. Suppose a user visits example.com and the page attempts a cross-origin request to fetch the user’s data from service.example.com. The server (service.example.com) can then reply that requests from all domains are allowed. So requests from the origin example.com (and any other for that matter), are allowed to be made against the server ‘service.example.com’. (Wikipedia 2020a)

You can set CORS with the Access-Control headers (Mozilla 2020b):

• Access-Control-Allow-Origin - specifies either a single origin, which tells browsers to allow that origin to access the resource; or else — for requests without credentials — the * wildcard, to tell browsers to allow any origin to access the resource.
• Access-Control-Expose-Headers - lets a server whitelist headers that browsers are allowed to access.
• Access-Control-Max-Age - indicates how long the results of a preflight request can be cached.
• Access-Control-Allow-Credentials - indicates whether the response to the request can be exposed when the credentials flag is true.
• Access-Control-Allow-Methods - specifies the method or methods allowed when accessing the resource.
• Access-Control-Allow-Headers - is used in response to a preflight request to indicate which HTTP headers can be used when making the actual request.

CSRF

CSRF²⁷ (Cross Site Request Forgery) is to execute an undesired action on the server when the user is authenticated. So executing requests on your behalf from somewhere else. Unlike cross-site scripting (XSS), which exploits the trust a user has for a particular site, CSRF exploits the trust that a site has in a user’s browser. (Wikipedia 2020b)

Imagine being authenticated at myexamplebank.com, and an attacker is able to get your browser to send a request to myexamplebank.com. By either letting you, for instance, click a link or visit a page that has instruction to send a request to myexamplebank.com. Now, the request is sent by the attacker with the instructions to transfer money from your account to his, by including cookies that are associated with myexamplebank.com. And when the cookies contain authentication data, the attacker is able to execute the transfer.

In order to mitigate this we need to make sure that HTTP methods such as GET, HEAD, OPTIONS are not allowed to change any server side state. And that the HTTP methods POST, PUT, PATCH, DELETE use a token based mitigation protocol. This method is the most popular, and recommended method for protecting your web application from CSRF attacks. This method works by requiring that requests that are able to change server side state to include a token. The token is generated server side once per user session, or for each request. It should be unique per user session, secret, and unpredictable. This will prevent CSRF attacks because an attack will not be able to create a valid request without this token. (OWASP 2020a) We can implement these token in Go by using the package gorilla/mux.

Conclusion

In this posts we’ve explored how we can make our Go based HTTP webserver more secure by implementing timeouts, TLS, and secure headers. This gives us a good starting-point, and as your application grows, and depending on its application and utility, additional security measures should be implemented. To reiterate what was mentioned in the introduction, we do well to check, and continue checking for best-practices, and there are some very good checklists²⁸ available that can help you make your HTTP webservers and web applications more secure.

References