When working with F5 I have run once into an interesting SSL/TLS problem. It is documented and described SOL6475: Overview of SSL TPS licensing limits.
The most important part from the solutions is:
The BIG-IP system measures SSL TPS based on client-side connection attempts to any virtual server configured with a Client SSL profile. SSL TPS is enforced across a sliding time window. The BIG-IP system utilizes a 10ms window (1/100 of a second) to calculate the current TPS. If the number of TPS requests within any 10ms window exceeds 1/100 of the licensed TPS, an error message regarding the TPS limit being reached is sent to the /var/log/ltm file.
How to know what clients IPs cause the error to be logged. How to measure and calculate the number of SSL connection per seconds for even 10ms.
As there are no tools on F5 that helps you to find this out I thought that a simple way to get some visibility of it would be to capture all TCP SYN packets hitting the LB and then later do some analysis of it. An implementation of this ideas in a form of a python script can be found here .
To test sslAnalyze.py script we need first a tcpdump file. For this purpose we can use the nmap command and run a SYN flood. For the desciption of the nmap options you can take a look here .
$ nmap -P0 -TNormal -D 220.127.116.11,18.104.22.168,22.214.171.124,126.96.36.199,188.8.131.52,184.108.40.206,220.127.116.11 -iR 10
All what we have to do now is to run on one session a tcpump and on the other the nmap command. As we are only interested in the TCP SYN packets we should tailor the tcpdump filtering syntax properly. A tcpdump that will capture only the SYN packets:
$ tcpdump -vvv -nn -i eth0 -w /var/tmp/syn-flood-example.pcap 'tcp&2!=0 and tcp&16==0'
All what we have to do is not run our script to see the statistics.
I have to quickly explain the script itself. Once run it will prints on stdout a listing of found connections and additionally will create a log file with a name sslConnHigh.txt for only these connections that are over the threshold.
The parameters that you have to specify are:
- param1 - tcpdump file (it has to have only SYN packets)
- param2 - time fractions in microseconds ( 1000000 microseconds -> 1 second )
- param3 - connection threshold per time to log this result to a sslConnHigh.txt file
# Example 1: to see a number of connection per 1 second $ python sslAnalyze.py syn-flood-example.pcap 1000000 1 # Example 2: to see a number of connection per every 500ms (half a second) $ python sslAnalyze.py syn-flood-example.pcap 500000 1 # Example3: to see a number of connection per every 500ms (half a second) and log only # these timestamps that have more than 100 connection in a single half a second # some example output has been attached as well below $ python sslAnalyze.py syn-flood-example.pcap 500000 100 keeping the line: reading from file syn-flood-example.pcap, link-type EN10MB (Ethernet) date timestamp sumOfConn [... 500000 microsecond periods ... ] Tue Aug 14 23:33:30 2012 1344983610 sum:183 0 183 Tue Aug 14 23:33:31 2012 1344983611 sum:95 6 89 Tue Aug 14 23:33:32 2012 1344983612 sum:614 430 184 Tue Aug 14 23:33:33 2012 1344983613 sum:520 216 304
To better understand why F5 logs the error message and what trigger the TPS log error messages we have to run this command:
# 10 milliseconds = 10000 microseconds $ python sslAnalyze.py syn-flood-example.pcap 10000 [F5_SSL_total_TPS] $ cat sslConnHigh.txt
In the output you are going to see the timestamps (rounded to 1 second) where the number of connections in a single 10ms window are above the licensing limit you device has. For further analize you can extract these data from the tcpdump with a help of tcpslice tool.
# 1268649656 is an example timestamp from above $ tcpslice 1268649656 +1 syn-flood-example.pcap -w 1268649656.pcap $ tcpdump -tt -nr 1268649656.pcap reading from file 1268649656.pcap, link-type EN10MB (Ethernet) 1268649656.042723 vlan 4093, p 0, IP 18.104.22.168.4598 - 22.214.171.124.443: S 2973530156:2973530156(0) win 64512 mss 1460,nop,nop,sackOK 1268649656.056163 vlan 4093, p 0, IP 126.96.36.199.1622 - 188.8.131.52.443: S 1522394445:1522394445(0) win 64512 mss 1460,nop,wscale 0,nop,nop,sackOK