There are many programs that allow us to save network traffic into a file for later analyze. Once you have the file(s) we could like to quickly investigate the data inside and verify what we sent and what we revived.
If we are interested, let's say only in the HTTP traffic than the command line 'tshark' has only limited capabilities in a way to present us the data for review and investigation.
We can still play with the various options: '-Tfields' and multiple '-e' but still don't get the complete headers output.
The complete list that can be used with '-e' can be found
here .
Alternatively we can experiment with the '-Tpdml' that will create alike XML file. But even with all this flexibility we still can't print a custom name header like in this curl request bellow:
curl -H "Rado: my_value" -v -o /tmp/page.html http://rtomaszewski.blogspot.com/2011/04/tshark-in-network-troubleshooting.html
To solve this little problem we have created a small program written in python. It takes the output from 'tshark -S -V' and parses it to present the data in a way we want.
Example how to use it:
# tshark -r /tmp/net.pcap -R http -V | parse.py -d
# tshark -r /tmp/net.pcap -R http -V | parse.py
# tshark -nn -s0 -i any -w /tmp/net.pcap -f tcp -R http -l -S -V | parse.py
# tshark -nn -s0 -i any -w /tmp/net.pcap -f tcp -R http -l -S -V | parse.py -d
An example output from the 'parse.py' when running on the command line:
$ curl -H "Rado: my_value" -v -o /tmp/page.html http://rtomaszewski.blogspot.com/2011/04/tshark-in-network-troubleshooting.html
* About to connect() to rtomaszewski.blogspot.com port 80 (#0)
* Trying 209.85.229.132... connected
* Connected to rtomaszewski.blogspot.com (209.85.229.132) port 80 (#0)
> GET /2011/04/tshark-in-network-troubleshooting.html HTTP/1.1
> User-Agent: curl/7.19.7 (i486-pc-linux-gnu) libcurl/7.19.7 OpenSSL/0.9.8k zlib/1.2.3.3 libidn/1.15
> Host: rtomaszewski.blogspot.com
> Accept: */*
> Rado: my_value
>
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0< HTTP/1.1 200 OK
< Content-Type: text/html; charset=UTF-8
< ETag: "b6d5837e-31bb-4473-95af-da3c1d466295"
< X-Content-Type-Options: nosniff
< X-XSS-Protection: 1; mode=block
< Server: GSE
< Age: 1202
< Date: Sun, 10 Apr 2011 19:14:00 GMT
< Expires: Sun, 10 Apr 2011 19:14:00 GMT
< Last-Modified: Sun, 10 Apr 2011 19:03:05 GMT
< Cache-Control: public, must-revalidate, proxy-revalidate, max-age=0
< Transfer-Encoding: chunked
<
{ [data not shown]
100 49993 0 49993 0 0 28925 0 --:--:-- 0:00:01 --:--:-- 57529* Connection #0 to host rtomaszewski.blogspot.com left intact
* Closing connection #0
# tshark -nn -s0 -i wlan0 -w /tmp/net.pcap -R http -f tcp -S -V | ./parse.py
Running as user "root" and group "root". This could be dangerous.
Capturing on wlan0
Internet Protocol, Src: 192.168.43.111 (192.168.43.111), Dst: 209.85.229.132 (209.85.229.132)
Transmission Control Protocol, Src Port: 38391 (38391), Dst Port: 80 (80), Seq: 1, Ack: 1, Len: 226
[Stream index: 1]
GET /2011/04/tshark-in-network-troubleshooting.html HTTP/1.1\r\n
User-Agent: curl/7.19.7 (i486-pc-linux-gnu) libcurl/7.19.7 OpenSSL/0.9.8k zlib/1.2.3.3 libidn/1.15\r\n
Host: rtomaszewski.blogspot.com\r\n
Accept: */*\r\n
Rado: my_value\r\n
Internet Protocol, Src: 209.85.229.132 (209.85.229.132), Dst: 192.168.43.111 (192.168.43.111)
Transmission Control Protocol, Src Port: 80 (80), Dst Port: 38391 (38391), Seq: 49070, Ack: 227, Len: 1375
[Stream index: 1]
HTTP/1.1 200 OK\r\n
Content-Type: text/html; charset=UTF-8\r\n
ETag: "b6d5837e-31bb-4473-95af-da3c1d466295"\r\n
X-Content-Type-Options: nosniff\r\n
X-XSS-Protection: 1; mode=block\r\n
Server: GSE\r\n
Age: 1202\r\n
Date: Sun, 10 Apr 2011 19:14:00 GMT\r\n
Expires: Sun, 10 Apr 2011 19:14:00 GMT\r\n
Last-Modified: Sun, 10 Apr 2011 19:03:05 GMT\r\n
Cache-Control: public, must-revalidate, proxy-revalidate, max-age=0\r\n
Transfer-Encoding: chunked\r\n
2 packets captured
The program can be downloaded form here:
parse.py Additionally the source code of this short tool can be seen here:
#!/usr/bin/python
#
## tested on python 2.6.5
#
# author : radoslaw tomaszewski
import sys
import re
import inspect
class ParseTsharkOut:
no=0
debugYes=0
ipRe=None
tcpRe=None
protRe=None
ipInfo=[]
tcpInfo=[]
protInfo=[]
auxStart=0
tcpAuxStart=0
protAux=1
def __init__(self):
self.ipRe="Internet Protocol,(.*)$"
self.tcpRe=["Transmission Control Protocol,(.*)$", " (\[Stream index:.*)$"]
self.protRe=["Hypertext Transfer Protocol", " ([^\[ ].*)$", "^$| *(\\\\r|\\\\n)"]
def debug(self, s):
if self.debugYes :
parent=inspect.stack()[1][3]
#parent=inspect.stack()
print("debug:[" + str(parent) + "] " + s.rstrip())
def usage(self):
print("todo")
def ipParse(self,s):
self.debug(s)
tmp=re.match(self.ipRe, s)
# tmp=re.match("..", s)
if tmp is None :
return 0
else:
self.ipInfo.append(tmp.group(0))
return 1
def tcpParse(self,s):
self.debug(s)
ret=0
tmp=re.match(self.tcpRe[self.tcpAuxStart], s)
if tmp is None :
return 0
else:
self.tcpInfo.append(tmp.group(0))
ret=self.tcpAuxStart
self.tcpAuxStart=(self.tcpAuxStart + 1 ) % 2
return ret
def protParse(self,s):
self.debug(s)
if ( self.protAux ) :
if re.match(self.protRe[0], s):
self.protAux=0
return 0
else :
if re.match(self.protRe[2], s):
self.protAux=1
self.show()
return 1
tmp=re.match(self.protRe[1], s)
if tmp is None :
return 0
else:
self.protInfo.append(tmp.group(0))
return 0
def parse(self, s):
funcs=[self.ipParse, self.tcpParse, self.protParse]
if funcs[self.auxStart](s):
self.auxStart= ( self.auxStart + 1 ) % 3
def show(self):
self.debug("start")
for i in self.ipInfo:
print(i)
for i in self.tcpInfo:
print(i)
for i in self.protInfo:
print(i)
print("")
self.ipInfo=[]
self.tcpInfo=[]
self.protInfo=[]
def main(self):
try:
if sys.argv[1] == '-d':
self.debugYes=1
self.debug("debuging is turn on")
except (IndexError):
None
self.debug("main start")
for l in sys.stdin:
self.parse(l)
return 0
if __name__ == "__main__":
sys.exit(ParseTsharkOut().main())
aaa
Posts
