SSL certificate chain order matters

The certs we trust are usually stored in the CApath on Linux systems. The file is a simple text file with all the certs concatenated one after another.

Problem  

Does the order of certificated stored in the CAfile chain file matter for the client or server?

Analysis and verification  

The simple answer is it depends. As the certs from the CApath/CAfile are used by the client it is independent of the SSL/TLS server we are connecting to. The implementation details of the servers should matter.

That means that the certificate order is important only to the local client itself. In the SSL handshaking the content of this file is never sent to the server. An example handshaking can be found: here: http://www.cisco.com/web/about/ac123/ac147/archived_issues/ipj_1-1/ssl.html.

To verify of the order of the certs matters for the openssl client we can run the following test. Both files ca1 and ca2 have the same certs but in different order. Example output.

$ openssl s_client -connect 1.1.1.1:443 -state -msg -CAfile ca1
CONNECTED(00000003)
SSL_connect:before/connect initialization
>>> SSL 2.0 [length 0077], CLIENT-HELLO
    01 03 01 00 4e 00 00 00 20 00 00 39 00 00 38 00
    ...
    ab 3b be 51 9d fa 43
SSL_connect:SSLv2/v3 write client hello A
<<< TLS 1.0 Handshake [length 002a], ServerHello
    02 00 00 26 03 01 2b ae 63 1e ec a0 82 a4 dc 25
    a9 4b 71 14 0a 54 2a ce 3d 6f 38 f5 26 e4 dd 8b
    7e e7 94 d5 02 b7 00 00 04 00
SSL_connect:SSLv3 read server hello A
<<< TLS 1.0 Handshake [length 0e16], Certificate
    11 11 0e 12 00 0e 0f 00 05 69 30 82 05 65 30 82
    22 22 a0 03 02 01 02 02 07 2b 86 02 70 e7 be 22
    ...
    09 0c 4d f6 a7 6b b4 99 84 65 ca 7a 88 e2 e2 44
    be 5c f7 ea 1c f5
depth=2 /C=US/ST=Arizona/L=Scottsdale/O=GoDaddy.com, Inc./CN=Go Daddy Root Certificate Authority - G2
verify return:1
depth=1 /C=US/ST=Arizona/L=Scottsdale/O=GoDaddy.com, Inc./OU=http://certs.godaddy.com/repository//CN=Go Daddy Secure Certificate Authority - G2
verify return:1
depth=0 /OU=Domain Control Validated/CN=mydomain.mysite.com
verify return:1
SSL_connect:SSLv3 read server certificate A
<<< TLS 1.0 Handshake [length 0004], ServerHelloDone
    0e 00 00 00
SSL_connect:SSLv3 read server done A
>>> TLS 1.0 Handshake [length 0106], ClientKeyExchange
    11 10 21 32 11 10 13 11 13 11 10 7b 1c c1 d1 10
    ...
    81 1f 71 f1 10 12
SSL_connect:SSLv3 write client key exchange A
>>> TLS 1.0 ChangeCipherSpec [length 0001]
    01
SSL_connect:SSLv3 write change cipher spec A
>>> TLS 1.0 Handshake [length 0010], Finished
    11 11 11 1c 1f 13 6f 1d 11 12 1a 19 ed 64 e8 4b
SSL_connect:SSLv3 write finished A
SSL_connect:SSLv3 flush data
<<< TLS 1.0 ChangeCipherSpec [length 0001]
    01
<<< TLS 1.0 Handshake [length 0010], Finished
    14 00 11 1c ed 9d fd 1f ab db ee ef 29 9a 1c 32
SSL_connect:SSLv3 read finished A
---
Certificate chain
 0 s:/OU=Domain Control Validated/CN=mydomain.mysite.com
   i:/C=US/ST=Arizona/L=Scottsdale/O=GoDaddy.com, Inc./OU=http://certs.godaddy.com/repository//CN=Go Daddy Secure Certificate Authority - G2
 1 s:/C=US/ST=Arizona/L=Scottsdale/O=GoDaddy.com, Inc./OU=http://certs.godaddy.com/repository//CN=Go Daddy Secure Certificate Authority - G2
   i:/C=US/ST=Arizona/L=Scottsdale/O=GoDaddy.com, Inc./CN=Go Daddy Root Certificate Authority - G2
 2 s:/C=US/ST=Arizona/L=Scottsdale/O=GoDaddy.com, Inc./CN=Go Daddy Root Certificate Authority - G2
   i:/C=US/ST=Arizona/L=Scottsdale/O=GoDaddy.com, Inc./CN=Go Daddy Root Certificate Authority - G2
---
Server certificate
-----BEGIN CERTIFICATE-----
AAAAAAAAAAAAAAAABBBBBBBBBBBBBBBCCCCCCCCCCCCCCDDDDDDDDEEEEEEFFFFF
...
111111111111111111111111111111111111111111111111111ah6I=
-----END CERTIFICATE-----
subject=/OU=Domain Control Validated/CN=mydomain.mysite.com
issuer=/C=US/ST=Arizona/L=Scottsdale/O=GoDaddy.com, Inc./OU=http://certs.godaddy.com/repository//CN=Go Daddy Secure Certificate Authority - G2
---
No client certificate CA names sent
---
SSL handshake has read 3710 bytes and written 431 bytes
---
New, TLSv1/SSLv3, Cipher is RC4-MD5
Server public key is 2048 bit
Secure Renegotiation IS NOT supported
Compression: NONE
Expansion: NONE
SSL-Session:
    Protocol  : TLSv1
    Cipher    : RC4-MD5
    Session-ID:
    Session-ID-ctx:
    Master-Key: 11861BE5828519468B6C59B0F01D3FF3126EA2B59DFB985E1C7D88B68E63BF399BCDEF7451D68421C2CE344765CDE572
    Key-Arg   : None
    Krb5 Principal: None
    Start Time: 1392721077
    Timeout   : 300 (sec)
    Verify return code: 0 (ok)
---

References

http://blog.edgecloud.com/post/19519955133/ssl-certificate-chain-order-matters
http://stackoverflow.com/questions/8431528/nginx-ssl-certificate-authentication-signed-by-intermediate-ca-chain
http://rtomaszewski.blogspot.co.uk/search/label/openssl
http://jw35.blogspot.co.uk/2010/05/doing-certificate-verification-in.html

How to host 2 ssl sites on a single public IP on F5 load balancer

As the pool of free IPv4 is getting lower it is important to efficiently mange our existing assigned public IPs. One thing that can be used is to use SSL offloading and hosting multiple HTTPS sites on single public IP using wild cart certificates.

The core of this solution is the configuration of the lb and the vhost setting on your server. The server reads the HTTP Host header and base on it decide what site the user try to access.

Problem

How to configure servers and load balancer to host 2 ssl sites on a single public IP.

Analisis and example configuration

• Example F5 configuration

For SSL to work we need a wild card certificate.
We have to import the certificate and key to F5 and create client site ssl profile.

Example wild cart certificate:

 
# openssl x509 -in /var/tmp/wildcard.rado.net.crt -noout -text
Certificate:
    Data:
        Version: 3 (0x2)
        Serial Number:
            04:9c:4a:4b:11:11:bc
        Signature Algorithm: sha1WithRSAEncryption
        Issuer: C=US, ST=Arizona, L=Scottsdale, O=GoDaddy.com, Inc., OU=http://certificates.godaddy.com/repository, CN=Go Daddy Secure Certification Authority/serialNumber=111111
        Validity
            Not Before: Nov 21 11:57:16 2012 GMT
            Not After : Nov 20 09:31:37 2013 GMT
        Subject: O=*.rado.net, OU=Domain Control Validated, CN=*.rado.net
        ....

SSL profile:

 
# tmsh list ltm profile client-ssl  wildcard-client-ssl-profile
ltm profile client-ssl wildcard-client-ssl-profile {
    cert wildcard-rado.net.crt
    chain CA.crt
    defaults-from clientssl
    key wildcard-rado.net.key
}

Virtual server to terminate and load balance the traffic:

 
# tmsh list ltm virtual VS-1.1.1.1-443
ltm virtual VS-1.1.1.1-443 {
    destination 192.168.99.68:https
    ip-protocol tcp
    mask 255.255.255.255a
    pool POOL-192.168.99.68-80
    profiles {
        http { }
        tcp { }
        wildcard-client-ssl-profile {
            context clientside
        }
    }
}

Servers where the traffic is going to be load balanced:

 
# tmsh list ltm pool POOL-192.168.99.68-80
ltm pool POOL-192.168.99.68-80 {
    load-balancing-mode least-connections-member
    members {
        10.177.1.1:http {
            session monitor-enabled
        }
        10.177.1.2:http {
            session monitor-enabled
        }
    }
    monitor http
}

• Apache configuration on one of the servers


This is a simple example configuration for the http server.


 
# cat vhost1.conf
<VirtualHost *:80>
        ServerName vhost1.rado.net
        DocumentRoot /var/www/vhost1
</VirtualHost>

# cat vhost2.conf
<VirtualHost *:80>
        ServerName vhost2.rado.net
        DocumentRoot /var/www/vhost2
</VirtualHost>

# cat /var/www/vhost1/index.html
<html><body><h1>It works!</h1>
<p>
This is VHOST 1
</p>
</body></html>

# a2ensite vhost1.conf 
# a2ensite vhost2.conf 

# service apache2 reload


A quick local test on the server will confirm if the config is correct.


 
# curl -v -H "Host: vhost1.rado.net" http://5.1.1.1
* About to connect() to 5.1.1.1 port 80 (#0)
*   Trying 5.1.1.1... connected
> GET / HTTP/1.1
> User-Agent: curl/7.22.0 (x86_64-pc-linux-gnu) libcurl/7.22.0 OpenSSL/1.0.1 zlib/1.2.3.4 libidn/1.23 librtmp/2.3
> Accept: */*
> Host: vhost1.rado.net
>
< HTTP/1.1 200 OK
< Date: Sat, 30 Mar 2013 23:42:20 GMT
< Server: Apache/2.2.22 (Ubuntu)
< Last-Modified: Sat, 30 Mar 2013 23:19:21 GMT
< ETag: "b46bb-47-4d92c9e17f040"
< Accept-Ranges: bytes
< Content-Length: 71
< Vary: Accept-Encoding
< Content-Type: text/html
<
<html><body><h1>It works!</h1>
<p>
This is VHOST 1
</p>
</body></html>
* Connection #0 to host 5.79.21.166 left intact
* Closing connection #0