jetty-documentation/src/main/asciidoc/administration/http2/configuring-haproxy.adoc - jetty - Git at Google

 //  ========================================================================
 //  Copyright (c) 1995-2017 Mort Bay Consulting Pty. Ltd.
 //  ========================================================================
 //  All rights reserved. This program and the accompanying materials
 //  are made available under the terms of the Eclipse Public License v1.0
 //  and Apache License v2.0 which accompanies this distribution.
 //
 //      The Eclipse Public License is available at
 //      http://www.eclipse.org/legal/epl-v10.html
 //
 //      The Apache License v2.0 is available at
 //      http://www.opensource.org/licenses/apache2.0.php
 //
 //  You may elect to redistribute this code under either of these licenses.
 //  ========================================================================

 [[http2-configuring-haproxy]]
 === Configuring HAProxy and Jetty

 Typical website deployments have Apache (or Nginx) configured as reverse proxy to talk to one or more backend Jetty instances.
 This configuration cannot be used for HTTP/2 because Apache does not yet support HTTP/2 (nor does Nginx).

 http://haproxy.org[HAProxy] is an open source solution that offers load balancing and proxying for TCP and HTTP based application, and can be used as a replacement for Apache or Nginx when these are used as reverse proxies and has the major benefit that supports HTTP/2.
 It also offers load balancing and several other features which can position it as a complete replacement for Apache or Nginx.

 The deployment proposed here will have HAProxy play the role that Apache and Nginx usually do: to perform the TLS offloading (that is, decrypt and encrypt TLS) and then forwarding the now clear-text traffic to a backend Jetty server, speaking either HTTP/1.1 or HTTP/2.

 The instructions that follow are for Linux.

 [[http2-haproxy-install]]
 ==== Installing HAProxy

 You will need HAProxy 1.5 or later, because it provides support for SSL and ALPN, both required by HTTP/2. Most Linux distributions have the HAProxy package available to be installed out of the box. For example on Ubuntu 15.04:

 [source,screen, subs="{sub-order}"]
 ....
 $ sudo apt-get install haproxy
 ....

 Alternatively you can download the HAProxy source code and build it on your environment by following the README bundled with the HAProxy source code tarball.

 ____
 [NOTE]
 HAProxy supports ALPN only if built with OpenSSL 1.0.2 or greater.
 Alternatively, HAProxy supports NPN when built with OpenSSL 1.0.1 or greater.
 You must upgrade OpenSSL if you have a version earlier than 1.0.1.
 Use `haproxy -vv` to know with which OpenSSL version HAProxy has been built.
 ____

 [[http2-haproxy-ssl]]
 ==== Setup SSL for HAProxy

 HAProxy will perform the TLS decryption and encryption much more efficiently than a Java implementation.

 HAProxy will need a single file containing the X509 certificates and the private key, all in https://en.wikipedia.org/wiki/X.509[PEM format], with the following order:

 1.  The site certificate; this certificate's Common Name refers to the site domain (for example: CN=*.webtide.com) and is signed by Certificate Authority #1.
 2.  The Certificate Authority #1 certificate; this certificate may be signed by Certificate Authority #2.
 3.  The Certificate Authority #2 certificate; this certificate may be signed by Certificate Authority #3; and so on until the Root Certificate Authority.
 4.  The Root Certificate Authority certificate.
 5.  The private key corresponding to the site certificate.

 Let's use `keytool` to generate a self signed certificate:

 [source,screen, subs="{sub-order}"]
 ....
 $ keytool -genkeypair -keyalg RSA -keystore keystore.p12 -storetype pkcs12 -storepass storepwd -ext SAN=DNS:domain.com
 What is your first and last name?
 [Unknown]:  *.domain.com
 What is the name of your organizational unit?
 [Unknown]:  Unit
 What is the name of your organization?
 [Unknown]:  Domain
 What is the name of your City or Locality?
 [Unknown]:  Torino
 What is the name of your State or Province?
 [Unknown]:  TO
 What is the two-letter country code for this unit?
 [Unknown]:  IT
 Is CN=*.domain.com, OU=Unit, O=Domain, L=Torino, ST=TO, C=IT correct?
 [no]:  yes
 ....

 The above command will generate a self signed certificate and private key for `domain.com` and subdomains, stored in the `keystore.p12` file in PKCS#12 format.
 We need to extract the certificate and the private key in PEM format.

 To extract the certificate into `certificate.pem`:

 [source,screen, subs="{sub-order}"]
 ....
 $ keytool -exportcert -keystore keystore.p12 -storetype pkcs12 -storepass storepwd -rfc -file certificate.pem
 ....

 To export the private key into `private_key.pem`:

 [source,screen, subs="{sub-order}"]
 ....
 $ openssl pkcs12 -in keystore.p12 -nodes -nocerts -out private_key.pem -passin pass:storepwd
 ....

 At this point you just need to concatenate the two files into one, in the correct order:

 [source,screen, subs="{sub-order}"]
 ....
 $ cat certificate.pem private_key.pem > domain.pem
 ....

 The `domain.pem` file will be used later by HAProxy.

 [[http2-haproxy-cfg]]
 ==== HAProxy Configuration File

 Now we can setup `haproxy.cfg` to configure HAProxy.
 This is a minimal configuration:

 [source, ,subs="{sub-order}"]
 ....
 global
 tune.ssl.default-dh-param 1024

 defaults
 timeout connect 10000ms
 timeout client 60000ms
 timeout server 60000ms

 frontend fe_http
 mode http
 bind *:80
 # Redirect to https
 redirect scheme https code 301

 frontend fe_https
 mode tcp
 bind *:443 ssl no-sslv3 crt domain.pem ciphers TLSv1.2 alpn h2,http/1.1
 default_backend be_http

 backend be_http
 mode tcp
 server domain 127.0.0.1:8282
 ....

 The HAProxy configuration file works in the following way.
 The `fe_http` front-end accepts connections on port 80 and redirects them to use the `https` scheme.

 The `fe_https` front-end accepts connections on port 443 and it is where the TLS decryption/encryption happens.
 You must specify the path to the PEM file containing the TLS key material (the `crt domain.pem` part), the ciphers that are suitable for HTTP/2 (the `ciphers TLSv1.2`), and the ALPN protocols supported (the `alpn h2,http/1.1` ).
 This front-end then forwards the now decrypted bytes to the back-end in `mode tcp`. The `mode tcp` says that HAProxy will not try to interpret the bytes as HTTP/1.1 but instead opaquely forward them to the back-end.

 The `be_http` back-end will forward (again in `mode tcp`) the clear-text bytes to a Jetty connector that talks clear-text HTTP/2 and HTTP/1.1 on port 8282.

 [[http2-haproxy-jetty]]
 ==== Setup Jetty for HTTP/2 and HTTP/1.1

 The Jetty setup follows the steps of having Jetty installed in the `JETTY_HOME` directory, creating a `JETTY_BASE` directory and initializing it using Jetty's command line tools.
 You must enable the `http2c` module, that is the module that speaks clear-text HTTP/2.
 Since the `http2c` module depends on the `http` module, the `http` module will be enabled transitively, and the final setup will therefore support both HTTP/2 and HTTP/1.1 in clear text.

 Additionally, you will also enable the `deploy` module to be able to deploy a sample web application:

 [source,screen, subs="{sub-order}"]
 ....
 $ JETTY_BASE=haproxy-jetty-http2
 $ mkdir $JETTY_BASE
 $ cd $JETTY_BASE
 $ java -jar $JETTY_HOME/start.jar --add-to-start=http2c,deploy
 ....

 Now let's deploy a demo web application and start Jetty:

 [source,screen, subs="{sub-order}"]
 ....
 $ cd $JETTY_BASE
 $ cp $JETTY_HOME/demo-base/webapps/async-rest.war $JETTY_BASE/webapps/
 $ java -jar $JETTY_HOME/start.jar jetty.http.host=127.0.0.1 jetty.http.port=8282
 ....

 Now you can browse https://domain.com/async-rest (replace `domain.com` with your own domain, or with `localhost`, to make this example work).

 ____
 [NOTE]
 You want the Jetty connector that listens on port 8282 to be available only to HAProxy, and not to remote clients.
 For this reason, you want to specify the `jetty.http.host` property on the command line (or in `start.ini`/ `start.d/http.ini` to make this setting persistent) to bind the Jetty connector only on the loopback interface (127.0.0.1), making it available to HAProxy but not to remote clients.
 If your Jetty instance runs on a different machine and/or on a different (sub)network, you may want to adjust both the back-end section of the HAProxy configuration file and the `jetty.http.host` property to match accordingly.
 ____

 Browsers supporting HTTP/2 will connect to HAProxy, which will decrypt the traffic and send it to Jetty.
 Likewise, HTTP/1.1 clients will connect to HAProxy, which will decrypt the traffic and send it to Jetty.

 The Jetty connector, configured with the `http2c` module (and therefore transitively with the `http` module) is able to distinguish whether the incoming bytes are HTTP/2 or HTTP/1.1 and will handle the request accordingly.

 The response is relayed back to HAProxy, which will encrypt it and send it back to the remote client.

 This configuration offers you efficient TLS offloading, HTTP/2 support and transparent fallback to HTTP/1.1 for clients that don't support HTTP/1.1.
	// ========================================================================
	// Copyright (c) 1995-2017 Mort Bay Consulting Pty. Ltd.
	// ========================================================================
	// All rights reserved. This program and the accompanying materials
	// are made available under the terms of the Eclipse Public License v1.0
	// and Apache License v2.0 which accompanies this distribution.
	//
	// The Eclipse Public License is available at
	// http://www.eclipse.org/legal/epl-v10.html
	//
	// The Apache License v2.0 is available at
	// http://www.opensource.org/licenses/apache2.0.php
	//
	// You may elect to redistribute this code under either of these licenses.
	// ========================================================================

	[[http2-configuring-haproxy]]
	=== Configuring HAProxy and Jetty

	Typical website deployments have Apache (or Nginx) configured as reverse proxy to talk to one or more backend Jetty instances.
	This configuration cannot be used for HTTP/2 because Apache does not yet support HTTP/2 (nor does Nginx).

	http://haproxy.org[HAProxy] is an open source solution that offers load balancing and proxying for TCP and HTTP based application, and can be used as a replacement for Apache or Nginx when these are used as reverse proxies and has the major benefit that supports HTTP/2.
	It also offers load balancing and several other features which can position it as a complete replacement for Apache or Nginx.

	The deployment proposed here will have HAProxy play the role that Apache and Nginx usually do: to perform the TLS offloading (that is, decrypt and encrypt TLS) and then forwarding the now clear-text traffic to a backend Jetty server, speaking either HTTP/1.1 or HTTP/2.

	The instructions that follow are for Linux.

	[[http2-haproxy-install]]
	==== Installing HAProxy

	You will need HAProxy 1.5 or later, because it provides support for SSL and ALPN, both required by HTTP/2. Most Linux distributions have the HAProxy package available to be installed out of the box. For example on Ubuntu 15.04:

	[source,screen, subs="{sub-order}"]
	....
	$ sudo apt-get install haproxy
	....

	Alternatively you can download the HAProxy source code and build it on your environment by following the README bundled with the HAProxy source code tarball.

	____
	[NOTE]
	HAProxy supports ALPN only if built with OpenSSL 1.0.2 or greater.
	Alternatively, HAProxy supports NPN when built with OpenSSL 1.0.1 or greater.
	You must upgrade OpenSSL if you have a version earlier than 1.0.1.
	Use `haproxy -vv` to know with which OpenSSL version HAProxy has been built.
	____

	[[http2-haproxy-ssl]]
	==== Setup SSL for HAProxy

	HAProxy will perform the TLS decryption and encryption much more efficiently than a Java implementation.

	HAProxy will need a single file containing the X509 certificates and the private key, all in https://en.wikipedia.org/wiki/X.509[PEM format], with the following order:

	1. The site certificate; this certificate's Common Name refers to the site domain (for example: CN=*.webtide.com) and is signed by Certificate Authority #1.
	2. The Certificate Authority #1 certificate; this certificate may be signed by Certificate Authority #2.
	3. The Certificate Authority #2 certificate; this certificate may be signed by Certificate Authority #3; and so on until the Root Certificate Authority.
	4. The Root Certificate Authority certificate.
	5. The private key corresponding to the site certificate.

	Let's use `keytool` to generate a self signed certificate:

	[source,screen, subs="{sub-order}"]
	....
	$ keytool -genkeypair -keyalg RSA -keystore keystore.p12 -storetype pkcs12 -storepass storepwd -ext SAN=DNS:domain.com
	What is your first and last name?
	[Unknown]: *.domain.com
	What is the name of your organizational unit?
	[Unknown]: Unit
	What is the name of your organization?
	[Unknown]: Domain
	What is the name of your City or Locality?
	[Unknown]: Torino
	What is the name of your State or Province?
	[Unknown]: TO
	What is the two-letter country code for this unit?
	[Unknown]: IT
	Is CN=*.domain.com, OU=Unit, O=Domain, L=Torino, ST=TO, C=IT correct?
	[no]: yes
	....

	The above command will generate a self signed certificate and private key for `domain.com` and subdomains, stored in the `keystore.p12` file in PKCS#12 format.
	We need to extract the certificate and the private key in PEM format.

	To extract the certificate into `certificate.pem`:

	[source,screen, subs="{sub-order}"]
	....
	$ keytool -exportcert -keystore keystore.p12 -storetype pkcs12 -storepass storepwd -rfc -file certificate.pem
	....

	To export the private key into `private_key.pem`:

	[source,screen, subs="{sub-order}"]
	....
	$ openssl pkcs12 -in keystore.p12 -nodes -nocerts -out private_key.pem -passin pass:storepwd
	....

	At this point you just need to concatenate the two files into one, in the correct order:

	[source,screen, subs="{sub-order}"]
	....
	$ cat certificate.pem private_key.pem > domain.pem
	....

	The `domain.pem` file will be used later by HAProxy.

	[[http2-haproxy-cfg]]
	==== HAProxy Configuration File

	Now we can setup `haproxy.cfg` to configure HAProxy.
	This is a minimal configuration:

	[source, ,subs="{sub-order}"]
	....
	global
	tune.ssl.default-dh-param 1024

	defaults
	timeout connect 10000ms
	timeout client 60000ms
	timeout server 60000ms

	frontend fe_http
	mode http
	bind *:80
	# Redirect to https
	redirect scheme https code 301

	frontend fe_https
	mode tcp
	bind *:443 ssl no-sslv3 crt domain.pem ciphers TLSv1.2 alpn h2,http/1.1
	default_backend be_http

	backend be_http
	mode tcp
	server domain 127.0.0.1:8282
	....

	The HAProxy configuration file works in the following way.
	The `fe_http` front-end accepts connections on port 80 and redirects them to use the `https` scheme.

	The `fe_https` front-end accepts connections on port 443 and it is where the TLS decryption/encryption happens.
	You must specify the path to the PEM file containing the TLS key material (the `crt domain.pem` part), the ciphers that are suitable for HTTP/2 (the `ciphers TLSv1.2`), and the ALPN protocols supported (the `alpn h2,http/1.1` ).
	This front-end then forwards the now decrypted bytes to the back-end in `mode tcp`. The `mode tcp` says that HAProxy will not try to interpret the bytes as HTTP/1.1 but instead opaquely forward them to the back-end.

	The `be_http` back-end will forward (again in `mode tcp`) the clear-text bytes to a Jetty connector that talks clear-text HTTP/2 and HTTP/1.1 on port 8282.

	[[http2-haproxy-jetty]]
	==== Setup Jetty for HTTP/2 and HTTP/1.1

	The Jetty setup follows the steps of having Jetty installed in the `JETTY_HOME` directory, creating a `JETTY_BASE` directory and initializing it using Jetty's command line tools.
	You must enable the `http2c` module, that is the module that speaks clear-text HTTP/2.
	Since the `http2c` module depends on the `http` module, the `http` module will be enabled transitively, and the final setup will therefore support both HTTP/2 and HTTP/1.1 in clear text.

	Additionally, you will also enable the `deploy` module to be able to deploy a sample web application:

	[source,screen, subs="{sub-order}"]
	....
	$ JETTY_BASE=haproxy-jetty-http2
	$ mkdir $JETTY_BASE
	$ cd $JETTY_BASE
	$ java -jar $JETTY_HOME/start.jar --add-to-start=http2c,deploy
	....

	Now let's deploy a demo web application and start Jetty:

	[source,screen, subs="{sub-order}"]
	....
	$ cd $JETTY_BASE
	$ cp $JETTY_HOME/demo-base/webapps/async-rest.war $JETTY_BASE/webapps/
	$ java -jar $JETTY_HOME/start.jar jetty.http.host=127.0.0.1 jetty.http.port=8282
	....

	Now you can browse https://domain.com/async-rest (replace `domain.com` with your own domain, or with `localhost`, to make this example work).

	____
	[NOTE]
	You want the Jetty connector that listens on port 8282 to be available only to HAProxy, and not to remote clients.
	For this reason, you want to specify the `jetty.http.host` property on the command line (or in `start.ini`/ `start.d/http.ini` to make this setting persistent) to bind the Jetty connector only on the loopback interface (127.0.0.1), making it available to HAProxy but not to remote clients.
	If your Jetty instance runs on a different machine and/or on a different (sub)network, you may want to adjust both the back-end section of the HAProxy configuration file and the `jetty.http.host` property to match accordingly.
	____

	Browsers supporting HTTP/2 will connect to HAProxy, which will decrypt the traffic and send it to Jetty.
	Likewise, HTTP/1.1 clients will connect to HAProxy, which will decrypt the traffic and send it to Jetty.

	The Jetty connector, configured with the `http2c` module (and therefore transitively with the `http` module) is able to distinguish whether the incoming bytes are HTTP/2 or HTTP/1.1 and will handle the request accordingly.

	The response is relayed back to HAProxy, which will encrypt it and send it back to the remote client.

	This configuration offers you efficient TLS offloading, HTTP/2 support and transparent fallback to HTTP/1.1 for clients that don't support HTTP/1.1.