How to dynamically inspect and manipulate the content of your loaded page inside chrome browser

The next generation browsers, like Google Chrome, can offer an powerful feature (Chrome DevTools) that allows you to inspect, debug and manipulate any web page element that you browse.

Demonstration and results description

• Open any web page in your browser
• Open the Chrome DevTools console tab: Ctrl+Shift+J 
• In the console type the following code and press Enter

document.body.innerHTML="this is now your new page"
"this is now your new page"

• Your loaded page within the browser window should be replaced now with our custom string
• By running different JavaScript code you can change the behavior and content of your page

Now to start exploring this all what you need to know is basic JavaScript knowledge and some understanding of the HTML standard (a good reference can be found here HTML - Living Standard).

References

http://www.whatwg.org/specs/web-apps/current-work/multipage/
https://developers.google.com/chrome-developer-tools/
http://www.w3schools.com/js/js_htmldom_navigation.asp

Basic tutorial on how to use and debug Cloud API for the NextGene (Openstack based) or FirsGen (build by Mosso) Rackspace cloud infrastructure

Before we start writing code and play with API we have to install the necessary python libraries first. I used the Ubuntu 11.04 for my testing.

Tools installation

• Openstack API library

There is a packet with the Openstack libraries in repositories unfortunately this is not the latest code that is compatible with the NextGen Rackspace cloud release. We have to install the tools manually as described here [1].

# don't do this i Ubuntu 11.04 because it is based on older version of the library 

# and will not work

aptitude install python-novaclient

aptitude install python-setuptools
easy_install pip
pip install python-novaclient
pip install --upgrade  python-novaclient

# To verify that the files were installed 
find /usr -name novaclient
/usr/share/pyshared/novaclient
/usr/local/lib/python2.7/dist-packages/novaclient

• FirstGen

# to install the library
aptitude install python-rackspace-cloudservers

# to verify where the files were installed
find /usr -name cloudservers
/usr/share/pyshared/cloudservers
/usr/lib/pymodules/python2.7/cloudservers

Debugging Nova Openstack API

 As Rackspace lunched his cloud first in the USA and Europe is following a few weeks later the URL below is for a US base cloud account. More about the URLs can be found here [2].

nova_example.py

# cat nova_example.py 
import httplib2
httplib2.debuglevel = 1

OS_USERNAME="user"
OS_PASSWORD="api"
OS_AUTH_URL="https://identity.api.rackspacecloud.com/v2.0/"

from novaclient.v1_1 import client
nt=client.Client(OS_USERNAME,OS_PASSWORD,'',OS_AUTH_URL)
nt.flavors.list()

You can than run this like python -i nova_example.py or run python and copy the code into it like that:

$ python -u
>>> import httplib2
>>> httplib2.debuglevel = 1
>>> 
>>> OS_USERNAME="user"
>>> OS_PASSWORD="api"
>>> OS_AUTH_URL="https://identity.api.rackspacecloud.com/v2.0/"
>>> 
>>> from novaclient.v1_1 import client
>>> nt=client.Client(OS_USERNAME,OS_PASSWORD,'',OS_AUTH_URL)
>>> nt.flavors.list()
connect: (dfw.servers.api.rackspacecloud.com, 443)
send: 'GET /v2/672114/flavors/detail HTTP/1.1\r\nHost: dfw.servers.api.rackspacecloud.com\r\nx-auth-token: mytoken\r\naccept-encoding: gzip, deflate\r\naccept: application/json\r\nuser-agent: python-novaclient\r\n\r\n'
reply: 'HTTP/1.1 200 OK\r\n'
header: Date: Tue, 07 Aug 2012 21:16:28 GMT
header: Content-Length: 2448
header: Content-Type: application/json
header: X-Compute-Request-Id: req-4e7dda9c-7eee-402f-acc8-b98dc08e21b5
header: Server: Jetty(8.0.y.z-SNAPSHOT)
[Flavor: 512MB Standard Instance, Flavor: 1GB Standard Instance, Flavor: 2GB Standard Instance, Flavor: 4GB Standard Instance, Flavor: 8GB Standard Instance, Flavor: 15GB Standard Instance, Flavor: 30GB Standard Instance]

Debugging FirstGen API

firstgen_example.py

# cat  firstgen_example.py
import httplib2
httplib2.debuglevel = 1

u='user'
k='api'

from cloudservers import CloudServers
cs=CloudServers(u,k)
cs.flavors.list()

An example output

$ python -u
>>> import httplib2
>>> httplib2.debuglevel = 1
>>>
>>> from cloudservers import CloudServers
>>> u='user'
>>> k='api'
>>> cs=CloudServers(u,k)
>>> cs.flavors.list()
connect: (lon.auth.api.rackspacecloud.com, 443)
send: 'GET /v1.0 HTTP/1.1\r\nHost: lon.auth.api.rackspacecloud.com\r\nx-auth-key: key\r\naccept-encoding: gzip, deflate\r\nx-auth-user: hugoalmeidauk\r\nuser-agent: python-cloudservers/1.0a1\r\n\r\n'
reply: 'HTTP/1.1 204 No Content\r\n'
header: Server: Apache/2.2.3 (Red Hat)
header: vary: X-Auth-User,X-Auth-Key,X-Storage-User,X-Storage-Pass
header: X-Storage-Url: https://storage101.lon3.clouddrive.com/v1/MossoCloudFS_c99f3ebf-f27d-4933-94b7-9e9ebf2bc7cd
header: Cache-Control: s-maxage=60806
header: Content-Type: text/xml
header: Date: Tue, 07 Aug 2012 21:19:36 GMT
header: X-Auth-Token: token
header: X-Storage-Token: token2
header: X-Server-Management-Url: https://lon.servers.api.rackspacecloud.com/v1.0/10001641
header: Connection: Keep-Alive
header: X-CDN-Management-Url: https://cdn3.clouddrive.com/v1/MossoCloudFS_c99f3ebf-f27d-4933-94b7-9e9ebf2bc7cd
header: Content-Length: 0
connect: (lon.servers.api.rackspacecloud.com, 443)
send: 'GET /v1.0/10001641/flavors/detail?fresh HTTP/1.1\r\nHost: lon.servers.api.rackspacecloud.com\r\nx-auth-token: token\r\naccept-encoding: gzip, deflate\r\nuser-agent: python-cloudservers/1.0a1\r\n\r\n'
reply: 'HTTP/1.1 203 OK\r\n'
header: Server: Apache-Coyote/1.1
header: vary:  Accept, Accept-Encoding, X-Auth-Token
header: Content-Encoding: gzip
header: Vary: Accept-Encoding
header: Last-Modified: Tue, 21 Jun 2011 21:09:45 GMT
header: X-PURGE-KEY: /flavors
header: Cache-Control: s-maxage=1800
header: Content-Type: application/json
header: Content-Length: 175
header: Date: Tue, 07 Aug 2012 21:19:36 GMT
header: X-Varnish: 1664913388 1664913324
header: Age: 44
header: Via: 1.1 varnish
header: Connection: keep-alive
[Flavor: 256 server, Flavor: 512 server, Flavor: 1GB server, Flavor: 2GB server, Flavor: 4GB server, Flavor: 8GB server, Flavor: 15.5GB server, Flavor: 30GB server]

References

1. http://docs.rackspace.com/servers/api/v2/cs-gettingstarted/content/section_gs_install_nova.html
2. http://docs.rackspace.com/servers/api/v2/cs-gettingstarted/content/section_gs_auth.html
3. http://pypi.python.org/pypi/python-novaclient
4. https://github.com/openstack/python-novaclient
5. http://www.rackspace.com/knowledge_center/article/cloud-servers-how-to-articles-other-resources

My python script buffers the output and it causes delays before the text appiers on the console

Linux bash is exelent tool for every day use. It allows you to combine tools and chain them toggethr to achieve remarkable results. As one of my favorite I use this one when testing:

$ python some_script.py | tee log.$(date +%s).txt 

Problem

The problem is that althoug I get all the output on the console it appers to be bufffered and I can't monitor the logs in live when my script runs. An example code can be seen below.
 
Solution

You have to tell python to stop buffering the data sent the the stream you are using (stdin, stdout, stderr). On on the way I found convenient is by using the command line '-u' options.

References

• http://stackoverflow.com/questions/107705/python-output-buffering
• http://stackoverflow.com/questions/3515757/python-print-statements-being-buffered-with-output-redirection
• http://www.pixelbeat.org/programming/stdio_buffering/