SSL certificate chain order matters

The certs we trust are usually stored in the CApath on Linux systems. The file is a simple text file with all the certs concatenated one after another.

Problem  

Does the order of certificated stored in the CAfile chain file matter for the client or server?

Analysis and verification  

The simple answer is it depends. As the certs from the CApath/CAfile are used by the client it is independent of the SSL/TLS server we are connecting to. The implementation details of the servers should matter.

That means that the certificate order is important only to the local client itself. In the SSL handshaking the content of this file is never sent to the server. An example handshaking can be found: here: http://www.cisco.com/web/about/ac123/ac147/archived_issues/ipj_1-1/ssl.html.

To verify of the order of the certs matters for the openssl client we can run the following test. Both files ca1 and ca2 have the same certs but in different order. Example output.

$ openssl s_client -connect 1.1.1.1:443 -state -msg -CAfile ca1
CONNECTED(00000003)
SSL_connect:before/connect initialization
>>> SSL 2.0 [length 0077], CLIENT-HELLO
    01 03 01 00 4e 00 00 00 20 00 00 39 00 00 38 00
    ...
    ab 3b be 51 9d fa 43
SSL_connect:SSLv2/v3 write client hello A
<<< TLS 1.0 Handshake [length 002a], ServerHello
    02 00 00 26 03 01 2b ae 63 1e ec a0 82 a4 dc 25
    a9 4b 71 14 0a 54 2a ce 3d 6f 38 f5 26 e4 dd 8b
    7e e7 94 d5 02 b7 00 00 04 00
SSL_connect:SSLv3 read server hello A
<<< TLS 1.0 Handshake [length 0e16], Certificate
    11 11 0e 12 00 0e 0f 00 05 69 30 82 05 65 30 82
    22 22 a0 03 02 01 02 02 07 2b 86 02 70 e7 be 22
    ...
    09 0c 4d f6 a7 6b b4 99 84 65 ca 7a 88 e2 e2 44
    be 5c f7 ea 1c f5
depth=2 /C=US/ST=Arizona/L=Scottsdale/O=GoDaddy.com, Inc./CN=Go Daddy Root Certificate Authority - G2
verify return:1
depth=1 /C=US/ST=Arizona/L=Scottsdale/O=GoDaddy.com, Inc./OU=http://certs.godaddy.com/repository//CN=Go Daddy Secure Certificate Authority - G2
verify return:1
depth=0 /OU=Domain Control Validated/CN=mydomain.mysite.com
verify return:1
SSL_connect:SSLv3 read server certificate A
<<< TLS 1.0 Handshake [length 0004], ServerHelloDone
    0e 00 00 00
SSL_connect:SSLv3 read server done A
>>> TLS 1.0 Handshake [length 0106], ClientKeyExchange
    11 10 21 32 11 10 13 11 13 11 10 7b 1c c1 d1 10
    ...
    81 1f 71 f1 10 12
SSL_connect:SSLv3 write client key exchange A
>>> TLS 1.0 ChangeCipherSpec [length 0001]
    01
SSL_connect:SSLv3 write change cipher spec A
>>> TLS 1.0 Handshake [length 0010], Finished
    11 11 11 1c 1f 13 6f 1d 11 12 1a 19 ed 64 e8 4b
SSL_connect:SSLv3 write finished A
SSL_connect:SSLv3 flush data
<<< TLS 1.0 ChangeCipherSpec [length 0001]
    01
<<< TLS 1.0 Handshake [length 0010], Finished
    14 00 11 1c ed 9d fd 1f ab db ee ef 29 9a 1c 32
SSL_connect:SSLv3 read finished A
---
Certificate chain
 0 s:/OU=Domain Control Validated/CN=mydomain.mysite.com
   i:/C=US/ST=Arizona/L=Scottsdale/O=GoDaddy.com, Inc./OU=http://certs.godaddy.com/repository//CN=Go Daddy Secure Certificate Authority - G2
 1 s:/C=US/ST=Arizona/L=Scottsdale/O=GoDaddy.com, Inc./OU=http://certs.godaddy.com/repository//CN=Go Daddy Secure Certificate Authority - G2
   i:/C=US/ST=Arizona/L=Scottsdale/O=GoDaddy.com, Inc./CN=Go Daddy Root Certificate Authority - G2
 2 s:/C=US/ST=Arizona/L=Scottsdale/O=GoDaddy.com, Inc./CN=Go Daddy Root Certificate Authority - G2
   i:/C=US/ST=Arizona/L=Scottsdale/O=GoDaddy.com, Inc./CN=Go Daddy Root Certificate Authority - G2
---
Server certificate
-----BEGIN CERTIFICATE-----
AAAAAAAAAAAAAAAABBBBBBBBBBBBBBBCCCCCCCCCCCCCCDDDDDDDDEEEEEEFFFFF
...
111111111111111111111111111111111111111111111111111ah6I=
-----END CERTIFICATE-----
subject=/OU=Domain Control Validated/CN=mydomain.mysite.com
issuer=/C=US/ST=Arizona/L=Scottsdale/O=GoDaddy.com, Inc./OU=http://certs.godaddy.com/repository//CN=Go Daddy Secure Certificate Authority - G2
---
No client certificate CA names sent
---
SSL handshake has read 3710 bytes and written 431 bytes
---
New, TLSv1/SSLv3, Cipher is RC4-MD5
Server public key is 2048 bit
Secure Renegotiation IS NOT supported
Compression: NONE
Expansion: NONE
SSL-Session:
    Protocol  : TLSv1
    Cipher    : RC4-MD5
    Session-ID:
    Session-ID-ctx:
    Master-Key: 11861BE5828519468B6C59B0F01D3FF3126EA2B59DFB985E1C7D88B68E63BF399BCDEF7451D68421C2CE344765CDE572
    Key-Arg   : None
    Krb5 Principal: None
    Start Time: 1392721077
    Timeout   : 300 (sec)
    Verify return code: 0 (ok)
---

References

http://blog.edgecloud.com/post/19519955133/ssl-certificate-chain-order-matters
http://stackoverflow.com/questions/8431528/nginx-ssl-certificate-authentication-signed-by-intermediate-ca-chain
http://rtomaszewski.blogspot.co.uk/search/label/openssl
http://jw35.blogspot.co.uk/2010/05/doing-certificate-verification-in.html