Search This Blog

Tuesday, March 27, 2012

Network data analize for simple scenario with Rackspace Cloud Load Balancer (CLB)

A load balancer becomes de facto a standard for many distributed deployments. You are going to need it even if you don't know it yet.

Let's take a look at the Rackspace Cloud Load Balancer (CLB) feature [1] and dig a litter bit more how it actually works. We are going to base our analysis on the diagram below. As the scenario is very simple to set up and configure I'm leaving it for the reader alone.




Test scenario
  1. The client sends HTTP request to LB1.
  2. The LB1 load balance it to his only one pool member.
  3. The pool member replays with HTTP 200.
  4. LB1 sends the replay back to the client.

As we don't have access to the LB1 itself it is hard to say what is really happening there. The only way at the moment is to take concurrent tcpdumps on the client and server. To simulate the client requests we are going to use the curl. All commands look like that:

# run on client
tcpdump -nn -s0 -i any -w /var/tmp/client.pcap  port 80

# run on server
tcpdump -nn -s0 -i any -w /var/tmp/server-urado1.pcap  port 80

# run on client on a separate session for example
curl -v http://31.222.175.142

The data from the captured tcpdump can be seen below. The original files can be found here [2].

[root@crado1 tmp]# tshark  -n -r client.pcap
  1   0.000000 31.222.191.246 -> 31.222.175.142 TCP 47267 > 80 [SYN] Seq=0 Win=5840 Len=0 MSS=1460 TSV=3064226 TSER=0 WS=4
  2   0.000324 31.222.175.142 -> 31.222.191.246 TCP 80 > 47267 [SYN, ACK] Seq=0 Ack=1 Win=17896 Len=0 MSS=8960 TSV=1082311203 TSER=3064226 WS=9
  3   0.000344 31.222.191.246 -> 31.222.175.142 TCP 47267 > 80 [ACK] Seq=1 Ack=1 Win=5840 Len=0 TSV=3064226 TSER=1082311203
  4   0.000441 31.222.191.246 -> 31.222.175.142 HTTP GET / HTTP/1.1
  5   0.000565 31.222.175.142 -> 31.222.191.246 TCP 80 > 47267 [ACK] Seq=1 Ack=159 Win=19456 Len=0 TSV=1082311203 TSER=3064226
  6   0.018854 31.222.175.142 -> 31.222.191.246 HTTP HTTP/1.1 200 OK  (text/html)
  7   0.018873 31.222.191.246 -> 31.222.175.142 TCP 47267 > 80 [ACK] Seq=159 Ack=301 Win=6912 Len=0 TSV=3064231 TSER=1082311205
  8   0.019107 31.222.191.246 -> 31.222.175.142 TCP 47267 > 80 [FIN, ACK] Seq=159 Ack=301 Win=6912 Len=0 TSV=3064231 TSER=1082311205
  9   0.019323 31.222.175.142 -> 31.222.191.246 TCP 80 > 47267 [FIN, ACK] Seq=301 Ack=160 Win=19456 Len=0 TSV=1082311205 TSER=3064231
 10   0.019337 31.222.191.246 -> 31.222.175.142 TCP 47267 > 80 [ACK] Seq=160 Ack=302 Win=6912 Len=0 TSV=3064231 TSER=1082311205


root@urado1:~/tmp# tshark -n -r server-urado1.pcap
  1   0.000000 10.190.254.7 -> 10.177.132.15 TCP 76 40293 > 80 [SYN] Seq=0 Win=17920 Len=0 MSS=8960 SACK_PERM=1 TSval=1082311203 TSecr=0 WS=512
  2   0.000061 10.177.132.15 -> 10.190.254.7 TCP 76 80 > 40293 [SYN, ACK] Seq=0 Ack=1 Win=14480 Len=0 MSS=1460 SACK_PERM=1 TSval=117399288 TSecr=1082311203 WS=4
  3   0.000382 10.190.254.7 -> 10.177.132.15 TCP 68 40293 > 80 [ACK] Seq=1 Ack=1 Win=17920 Len=0 TSval=1082311203 TSecr=117399288
  4   0.000390 10.190.254.7 -> 10.177.132.15 HTTP 321 GET / HTTP/1.1
  5   0.000422 10.177.132.15 -> 10.190.254.7 TCP 68 80 > 40293 [ACK] Seq=1 Ack=254 Win=15552 Len=0 TSval=117399288 TSecr=1082311203
  6   0.017298 10.177.132.15 -> 10.190.254.7 HTTP 368 HTTP/1.1 200 OK  (text/html)
  7   0.017790 10.190.254.7 -> 10.177.132.15 TCP 68 40293 > 80 [ACK] Seq=254 Ack=301 Win=19456 Len=0 TSval=1082311205 TSecr=117399293
  8   5.032706 10.177.132.15 -> 10.190.254.7 TCP 68 80 > 40293 [FIN, ACK] Seq=301 Ack=254 Win=15552 Len=0 TSval=117400546 TSecr=1082311205
  9   5.063446 10.190.254.7 -> 10.177.132.15 TCP 68 40293 > 80 [ACK] Seq=254 Ack=302 Win=19456 Len=0 TSval=1082311710 TSecr=117400546

Analysis

Base on the tcpdumps from the server we can clearly see that the LB1 is changing the original source ip address. It means that our server can't directly relay on the original IP of the client. 

Looking further at the payload we can see that as the traffic is sent from LB to the pool member the load balancer inserts additional headers into the original GET request.

[root@crado1 tmp]# tshark -n -r client.pcap -V http
    GET / HTTP/1.1\r\n
        Request Method: GET
        Request URI: /
        Request Version: HTTP/1.1
    User-Agent: curl/7.15.5 (x86_64-redhat-linux-gnu) libcurl/7.15.5 OpenSSL/0.9.8b zlib/1.2.3 libidn/0.6.5\r\n
    Host: 31.222.175.142\r\n
    Accept: */*\r\n
    \r\n

    HTTP/1.1 200 OK\r\n
        Request Version: HTTP/1.1
        Response Code: 200
    Date: Mon, 26 Mar 2012 21:50:16 GMT\r\n
    Server: Apache/2.2.20 (Ubuntu)\r\n
    Last-Modified: Mon, 26 Mar 2012 21:29:38 GMT\r\n
    ETag: "7c030-2c-4bc2c1245f480"\r\n
    Accept-Ranges: bytes\r\n
    Content-Length: 44\r\n
        [Content length: 44]
    Vary: Accept-Encoding\r\n
    Content-Type: text/html\r\n
    \r\n
Line-based text data: text/html
    <html><body>It works! urado1\n
    </body></html>\n


root@urado1:~/tmp# tshark -n -r server-urado1.pcap -V http
    GET / HTTP/1.1\r\n
        [Expert Info (Chat/Sequence): GET / HTTP/1.1\r\n]
            [Message: GET / HTTP/1.1\r\n]
            [Severity level: Chat]
            [Group: Sequence]
        Request Method: GET
        Request URI: /
        Request Version: HTTP/1.1
    User-Agent: curl/7.15.5 (x86_64-redhat-linux-gnu) libcurl/7.15.5 OpenSSL/0.9.8b zlib/1.2.3 libidn/0.6.5\r\n
    X-Forwarded-For: 31.222.191.246\r\n
    Accept: */*\r\n
    X-Forwarded-Proto: http\r\n
    Host: 31.222.175.142\r\n
    X-Cluster-Client-Ip: 31.222.191.246\r\n
    \r\n
    [Full request URI: http://31.222.175.142/]

    HTTP/1.1 200 OK\r\n
        [Expert Info (Chat/Sequence): HTTP/1.1 200 OK\r\n]
            [Message: HTTP/1.1 200 OK\r\n]
            [Severity level: Chat]
            [Group: Sequence]
        Request Version: HTTP/1.1
        Status Code: 200
        Response Phrase: OK
    Date: Mon, 26 Mar 2012 21:50:16 GMT\r\n
    Server: Apache/2.2.20 (Ubuntu)\r\n
    Last-Modified: Mon, 26 Mar 2012 21:29:38 GMT\r\n
    ETag: "7c030-2c-4bc2c1245f480"\r\n
    Accept-Ranges: bytes\r\n
    Content-Length: 44\r\n
        [Content length: 44]
    Vary: Accept-Encoding\r\n
    Content-Type: text/html\r\n
    \r\n
Line-based text data: text/html
    It works! urado1\n
    \n

Summary

As the load balancer manipulates the IP addresses on the client and server site it still provides a method to identify the original source ip address of the client. There will be inserted a HTTP header X-Forwarded-For with the value of the original client IP address.


Labels: , , , , ,

No comments:

Post a Comment

Subscribe to: Post Comments (Atom)