Search This Blog

Showing posts with label cloud. Show all posts
Showing posts with label cloud. Show all posts

Sunday, April 27, 2014

Overlay technologies in data center

Everyone speaks about SDN an the benefits its brings when deploying cloud or enterprise infrastructures. But do we actually know or have any understanding what this all SDN is about? If you want be fluent in the language of virtual networking and network overlays in modern data centers you need to understand at least the following concepts:
In the remaining of the post we will concentrate solely on existing overlay technologies. These information was extracted from Cisco doc: Cisco Nexus 9000 Series Switches - Data Center Overlay Technologies).

Network-Based Overlay Networks
  1. IEEE 802.1ad Provider Bridging or IEEE 802.1q Tunneling also known as IEEE 802.1QinQ or simply Q-in-Q
  2. IEEE 802.1ah Provider Backbone Bridges (PBB) or Mac-in-Mac Tunnels
  3. Cisco FabricPath allows multipath networking at Layer 2
  4. TRILL - IETF Transparent Interconnection of Lots of Links is a Layer 2 multipathing technology
  5. Shortest-Path Bridging (SPB) is defined in IEEE 802.1aq and is targeted as a replacement for Spanning Tree Protocol (example info based on Avaya documentation)
  6. Cisco Overlay Transport Virtualization (OTV) is a Layer 2-over-Layer 3 encapsulation "MAC-in-IP" technology
  7. The Cisco Location/Identifier Separation Protocol (LISP) is currently defined as a Layer 3 overlay scheme over a Layer 3 network
  8. Multiprotocol Label Switching (MPLS)
  9. Virtual Private LAN Service (VPLS) a Layer 2 tunneling protocols
  10. Virtual Private Routed Network (VPRN) also known as BGP/MPLS or IP-VPN provides IP VPN services
Host-Based Overlay Networks
    1. Virtual Extensible LAN (VXLAN) is a Layer 2 overlay scheme over a Layer 3 networ that uses IP/UDP encapsulation
    2. Network Virtualization Using Generic Routing Encapsulation (NVGRE) allows creation of virtual Layer 2 topologies on top of a physical Layer 3 network
    3. Stateless transport tunneling (STT) is an overlay encapsulation scheme over Layer 3 networks that use a TCP-like header

    Sunday, February 23, 2014

    How to design a network for Openstack or cloud deployment

    Designing a network is a topic within itself and there is no way we can cover all of it in this single post. Cisco has its own certification path path CCDE for these who want to know more.

    Cisco design and implementation guide - old best practices

    In a very simplistic view back in the old days a network used to be design similar to the picture below (although it is hard to say when the new era started ;)). Every big network had to have a core, distribution(sometimes called aggregation) and access layer. The network was engineered mainly to help with North to South traffic in the data center or in another words to help get the data out and into the data center.


    New cloud friendly data center design best practice

    With the advent and popularization of new networking devises that support layer 2 routing, commonly know as networks fabrics (more info about TRILL and fabric) the network design has shifted in data centers. The way we design the networks today is to maximize the East to West traffic instead of the North to South ( old design above). The purpose of the new network is to allow more efficiently exchange data between the servers within the rack or data center.

    They say that a picture is worth more than a thousand words. To help us to visualize how a new data center network/cloud network is designed these demonstration pictures (taken from Cisco document: Cisco Massively Salable Data Center) will shed some more light on it. Please note that we no longer use the core, distribution or access keyword but instead: spine, leaf or superspine to describe the different network layers :).

    Openstack reference architecture using Cisco UCS hardware platform

    There are many vendors to chose from when selecting your hardware for a complete Openstack deployment (example list of data center friendly networking vendors to look at ). And to make it even more difficult you need to think about all spectrum of vendors like networking, storage and compute.

    In one of my previous posts (How to estimate hardware requirements for your private Openstack deployment) I tried to demonstrate an example hardware recommendation for an Openstack deployment. Today we are going to look at this topic once again but exploring the Cisco UCS product line instead.

    The data below are taken from the Cisco PDF white paper Red Hat Openstack Architecture on Cisco UCS platform from the DesignZone for Cloud Automation Solution section on Cisco site (http://www.cisco.com/c/en/us/solutions/enterprise/data-center-designs-cloud-computing/could_automation.html).


    Openstack on Cisco UCS hardware platform

    Cisco is no longer only a networking vendor. With the UCS they offer as well computer platform where you can put together a server with specific hard drive size,  mount of RAM or type of CPU, interconnection card etc. An example configuration taken from the Cisco document above:


    That means if we put togheter the UCS servers and the Cisco Nexus switches and Openstack software we can build a simple POC like this one:


    In the white paper we can actually find a full list of hardware if you would like to build it yourself.


    Monday, January 27, 2014

    Google Compute Engine Console

    In one of our previous posts (Links to Cloud provider web management consoles) we took a look at some of the major cloud providers out there. In this post we take a look how Cloud Google Compute platform looks like.
    • Google Cloud Platform
    https://cloud.google.com/


    Monday, January 13, 2014

    Top Openstack contributors

    There is a new site on the main Openstack portal: http://activity.openstack.org/. When looking around to see what information you can find there I stumble upon this post below.

    What came as a surprise:
    • High position for IBM. 
    • Not Rackspace or Canonical or Mirantis but Red Hat as a number one contributor.
    Top 10 Openstack organizations activities in 2013/2014


    Monday, January 6, 2014

    Using Qemu on cloud server to run emulated virtual machines

    We know that there is not support for nested hypervisors on cloud instances: Nested virtualization support on Rackspace public cloud.

    Problem

    How to use Qemu on cloud server and start virtual machine to overcome the nested virtualization limitation.

    Demonstration and results description

    To overcome this limitation we will use Qemu in its emulated mode. Qemu in this mode doesn't require any specials virtualization support in CPU (HVM - Hardware-assisted virtualization).

    The VM image was downloaded from here: http://people.debian.org/~aurel32/qemu/i386/. The default u/p is root.


    Alternatively we could use Virtualbox. Although I'm not quite sure what would work better yet and give more options to customizing the VMs.

    References

    https://wiki.debian.org/QEMU
    http://www.linux-kvm.org/page/FAQ - this is more to show what can be missing as the CS don't support KVM
    http://en.wikipedia.org/wiki/QEMU
    http://www.linuxforu.com/2012/05/virtualisation-faceoff-qemu-virtualbox-vmware-player-parallels-workstation/



    Sunday, January 5, 2014

    Hypervisor hardware differences on Openstack Rackspace Cloud

    You can spin up test cloud servers and extract CPU flags with the help of this little script using csplit.
     
    cat /proc/cpuinfo | csplit -z  - '/processor/' '{*}'
    diff xx0*
    grep flags xx01 | cut -d ':' -f 2 | xargs -n1 echo | sort > flags.txt
    

    By comparing the results we can definitely say that:
    • Performance1 and performance2 cloud servers are running on the same hardware.
    • New performance cloud servers are hosted on Intel CPU.
    • The standard (next generation) series are being hosted on AMD CPU.
    The CPU flags for comparison.
     
    processor       : 1
    vendor_id       : AuthenticAMD
    cpu family      : 16
    model           : 4
    model name      : Quad-Core AMD Opteron(tm) Processor 2374 HE
    stepping        : 2
    microcode       : 0x1000086
    cpu MHz         : 2200.096
    cache size      : 512 KB
    fpu             : yes
    fpu_exception   : yes
    cpuid level     : 5
    wp              : yes
    flags           : fpu de tsc msr pae cx8 cmov pat clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt lm 3dnowext 3dnow rep_good nopl pni cx16 popcnt hypervisor lahf_lm cmp_legacy extapic cr8_legacy abm sse4a misalignsse 3dnowprefetch hw_pstate
    bogomips        : 4400.19
    TLB size        : 1024 4K pages
    clflush size    : 64
    cache_alignment : 64
    address sizes   : 48 bits physical, 48 bits virtual
    power management: ts ttp tm stc 100mhzsteps hwpstate
    

    processor       : 1
    vendor_id       : GenuineIntel
    cpu family      : 6
    model           : 45
    model name      : Intel(R) Xeon(R) CPU E5-2670 0 @ 2.60GHz
    stepping        : 7
    microcode       : 0x70d
    cpu MHz         : 2600.068
    cache size      : 20480 KB
    physical id     : 0
    siblings        : 2
    core id         : 0
    cpu cores       : 1
    apicid          : 0
    initial apicid  : 43
    fpu             : yes
    fpu_exception   : yes
    cpuid level     : 13
    wp              : yes
    flags           : fpu de tsc msr pae cx8 sep cmov pat clflush mmx fxsr sse sse2 ss ht syscall nx lm constant_tsc rep_good nopl pni pclmulqdq ssse3 cx16 sse4_1 sse4_2 popcnt tsc_deadline_timer aes hypervisor lahf_lm arat pln pts dtherm
    bogomips        : 5200.13
    clflush size    : 64
    cache_alignment : 64
    address sizes   : 46 bits physical, 48 bits virtual
    power management:
    

    Wednesday, January 1, 2014

    OpenShift PaaS platform from Redhat

    We've looked before at AppFog PaaS platform that is leveraging CloudFoundry and Openstack. Today we will investigate the Redhat PaaS alternative, namely OpenShift.

    What is OpenShift

    OpenShift is a Platform as a Service solution according the the cloud systems taxonomy. It offers a solution, a software development platform that facilitates easy and rapid application development in the cloud.

    It can provide a preintegrated software stack environment and can leverage IaaS providers for the virtual infrastructure management (How to Deploy OpenShift Enterprise on Red Hat OpenStack).

    A comprehensive presentation that provide further details can be found here: http://rhsummit.files.wordpress.com/2013/06/noceda_t_0120_consumepaasinthecloudwithopenshift.pdf

    Demo
    • Sign in for free account on https://openshift.redhat.com
    • Login and from "My App" tab create new application by selecting "Add Application ..." button.
    • I've used as simple Python app based on CherryPy framework. 
    • Once your gear (a Openshift name for your app container) is created you can take a look at the hello world code 
    • To modify the source code clone the git repo. You will need to first register your public key.
    • Generate a new ssh key pair.
    $  ssh-keygen
    Generating public/private rsa key pair.
    Enter file in which to save the key (/home/rado/.ssh/id_rsa): /tmp/tmp.key
    
    $ ssh-agent | tee -a agent.sh
    SSH_AUTH_SOCK=/tmp/ssh-pMmGGPP15635/agent.15635; export SSH_AUTH_SOCK;
    SSH_AGENT_PID=15636; export SSH_AGENT_PID;
    echo Agent pid 15636;
    
    $source agent.sh
    $ssh-add tmp.key
    Identity added: tmp.key (tmp.key)
    • Clone your source repo to modify your app source code.
    $ git clone ssh://52c499a1e0b8cdda0f000016@rado-rado1stapp.rhcloud.com/~/git/rado.git/
    Cloning into 'rado'...
    remote: Counting objects: 62, done.
    remote: Compressing objects: 100% (41/41), done.
    remote: Total 62 (delta 16), reused 62 (delta 16)
    Receiving objects: 100% (62/62), 18.93 KiB, done.
    Resolving deltas: 100% (16/16), done.
    • Modify the code to personalize it.
    $cd rado/
    
    $ find  | grep -v git
    ./.openshift
    ./.openshift/markers
    ./.openshift/action_hooks
    ./.openshift/action_hooks/README.md
    ./.openshift/cron
    ./.openshift/cron/minutely
    ./.openshift/cron/monthly
    ./.openshift/cron/weekly
    ./.openshift/cron/weekly/chrono.dat
    ./.openshift/cron/weekly/chronograph
    ./.openshift/cron/weekly/jobs.deny
    ./.openshift/cron/weekly/README
    ./.openshift/cron/weekly/jobs.allow
    ./.openshift/cron/README.cron
    ./.openshift/cron/daily
    ./.openshift/cron/hourly
    ./README.md
    ./LICENSE
    ./data
    ./libs
    ./wsgi
    ./wsgi/static
    ./wsgi/static/README
    ./wsgi/application
    ./app.py.disabled
    ./setup.py
    
    $vim ./wsgi/application
    $ cat wsgi/application
    import sys
    sys.stdout = sys.stderr
    
    import atexit
    import threading
    import cherrypy
    
    cherrypy.config.update({'environment': 'embedded'})
    
    if cherrypy.__version__.startswith('3.0') and cherrypy.engine.state == 0:
        cherrypy.engine.start(blocking=False)
        atexit.register(cherrypy.engine.stop)
    
    class Root(object):
        def index(self):
            return 'Hello from rado 1st app on openshift ;)!'
        index.exposed = True
    
    application = cherrypy.Application(Root(), script_name=None, config=None)
    • Once you are happy with t he code it is time to push changes back to git and redeploy it inside OpenShift.
    $ git commit ./wsgi/application
    [master 0eccd4d] init
     1 file changed, 1 insertion(+), 1 deletion(-)
    
    $ git push origin master
    Counting objects: 7, done.
    Compressing objects: 100% (4/4), done.
    Writing objects: 100% (4/4), 393 bytes, done.
    Total 4 (delta 2), reused 0 (delta 0)
    remote: Stopping PYTHON cart
    remote: [Wed Jan 01 17:59:08 2014] [warn] PassEnv variable SHELL was undefined
    remote: [Wed Jan 01 17:59:08 2014] [warn] PassEnv variable USER was undefined
    remote: [Wed Jan 01 17:59:08 2014] [warn] PassEnv variable LOGNAME was undefined
    remote: Waiting for stop to finish
    remote: Building git ref 'master', commit 0eccd4d
    remote: running develop
    remote: running egg_info
    remote: creating Example_CherryPy.egg-info
    remote: writing requirements to Example_CherryPy.egg-info/requires.txt
    remote: writing Example_CherryPy.egg-info/PKG-INFO
    remote: writing top-level names to Example_CherryPy.egg-info/top_level.txt
    remote: writing dependency_links to Example_CherryPy.egg-info/dependency_links.txt
    remote: writing requirements to Example_CherryPy.egg-info/requires.txt
    remote: writing Example_CherryPy.egg-info/PKG-INFO
    remote: writing top-level names to Example_CherryPy.egg-info/top_level.txt
    remote: writing dependency_links to Example_CherryPy.egg-info/dependency_links.txt
    remote: writing manifest file 'Example_CherryPy.egg-info/SOURCES.txt'
    remote: reading manifest file 'Example_CherryPy.egg-info/SOURCES.txt'
    remote: writing manifest file 'Example_CherryPy.egg-info/SOURCES.txt'
    remote: running build_ext
    remote: Creating /var/lib/openshift/52c499a1e0b8cdda0f000016/app-root/runtime/dependencies/python/virtenv/lib/python2.7/site-packages/Example-CherryPy.egg-link (link to .)
    remote: Example-CherryPy 1.0 is already the active version in easy-install.pth
    remote:
    remote: Installed /var/lib/openshift/52c499a1e0b8cdda0f000016/app-root/runtime/repo
    remote: Processing dependencies for Example-CherryPy==1.0
    remote: Searching for CherryPy==3.2.4
    remote: Best match: CherryPy 3.2.4
    remote: Processing CherryPy-3.2.4-py2.7.egg
    remote: CherryPy 3.2.4 is already the active version in easy-install.pth
    remote: Installing cherryd script to /var/lib/openshift/52c499a1e0b8cdda0f000016/python/virtenv/bin
    remote:
    remote: Using /var/lib/openshift/52c499a1e0b8cdda0f000016/app-root/runtime/dependencies/python/virtenv/lib/python2.7/site-packages/CherryPy-3.2.4-py2.7.egg
    remote: Finished processing dependencies for Example-CherryPy==1.0
    remote: Script /var/lib/openshift/52c499a1e0b8cdda0f000016/python//virtenv/bin/activate.fish cannot be made relative (it's not a normal script that starts with #!/var/lib/openshift/52c499a1e0b8cdda0f000016/python/virtenv/bin/python)
    remote: Script /var/lib/openshift/52c499a1e0b8cdda0f000016/python//virtenv/bin/activate.csh cannot be made relative (it's not a normal script that starts with #!/var/lib/openshift/52c499a1e0b8cdda0f000016/python/virtenv/bin/python)
    remote: Preparing build for deployment
    remote: Deployment id is f17888e7
    remote: Activating deployment
    remote: Script /var/lib/openshift/52c499a1e0b8cdda0f000016/python//virtenv/bin/activate.fish cannot be made relative (it's not a normal script that starts with #!/var/lib/openshift/52c499a1e0b8cdda0f000016/python/virtenv/bin/python)
    remote: Script /var/lib/openshift/52c499a1e0b8cdda0f000016/python//virtenv/bin/activate.csh cannot be made relative (it's not a normal script that starts with #!/var/lib/openshift/52c499a1e0b8cdda0f000016/python/virtenv/bin/python)
    remote: Starting PYTHON cart
    remote: Result: success
    remote: Activation status: success
    remote: Deployment completed with status: success
    • Final test that all worked fine ;)
    $ curl -v -s http://rado-rado1stapp.rhcloud.com ;echo
    * About to connect() to rado-rado1stapp.rhcloud.com port 80 (#0)
    *   Trying 54.221.76.112... connected
    > GET / HTTP/1.1
    > User-Agent: curl/7.22.0 (x86_64-pc-linux-gnu) libcurl/7.22.0 OpenSSL/1.0.1 zlib/1.2.3.4 libidn/1.23 librtmp/2.3
    > Host: rado-rado1stapp.rhcloud.com
    > Accept: */*
    >
    < HTTP/1.1 200 OK
    < Date: Wed, 01 Jan 2014 23:33:20 GMT
    < Server: Apache/2.2.22 (Red Hat Enterprise Web Server)
    < Content-Length: 40
    < Content-Type: text/html;charset=utf-8
    < Vary: Accept-Encoding
    <
    * Connection #0 to host rado-rado1stapp.rhcloud.com left intact
    * Closing connection #0
    
    Hello from rado 1st app on openshift ;)!
    

    Redhat Cloudforms product explained

    As the cloud technology continues to evolve we see even more new cool product names that vendors advertise. One of these is Cloudforms from Redhat. But what does Redhat Cloudforms do and where it can be used? The answer can be found in one of the slides from Redhat Summit 2013.

    From Introduction to Red Hat Openstack slides we can learn that Cloudforms:
    • Is a management platform for heterogeneous clouds.
    • It does support RHVE, Openstack, Redhat RDO and proprietary virtualization solutions.

    Thursday, November 21, 2013

    How to create a static URL link to a file stored in Google Drive

    You can upload and store files in Google Drive. With this cloud files service you have the ability to view online your files using Google Drive Viewer. An example link https://docs.google.com/file/d/0B7ftIvGKDqYMMWxrbTZmWGxJb0E/edit may look like:


    Problem

    How to create a static URL links for files stored in a directory on Google drive so you can use them in your HTML code or blog postings.

    Solution

    For detail instruction take a look at the reference section below. In short you have to first publish the Google drive directory and make it available for everyone on Internet. The link to share will look similar to:

    https://drive.google.com/folderview?id=0B7ftIvGKDqYMTGk0Ujd2QzhYS1U

    Please note the last part that represent an ID for your directory that we are going to use below.

    With this Id you can now see a directory listing by navigating to:

    https://googledrive.com/host/0B7ftIvGKDqYMTGk0Ujd2QzhYS1U/

    And to see a single file by adding the file name at the end to the URL :

    https://googledrive.com/host/0B7ftIvGKDqYMTGk0Ujd2QzhYS1U/shBrushBash.js
     
    $ curl -s -v -o tmp https://googledrive.com/host/0B7ftIvGKDqYMTGk0Ujd2QzhYS1U/shBrushBash.js 2>&1 | egrep '[><]'
    > GET /host/0B7ftIvGKDqYMTGk0Ujd2QzhYS1U/shBrushBash.js HTTP/1.1
    > User-Agent: curl/7.29.0
    > Host: googledrive.com
    > Accept: */*
    >
    < HTTP/1.1 200 OK
    < Content-Type: application/x-javascript
    < Date: Thu, 21 Nov 2013 00:38:16 GMT
    < Content-Length: 2835
    < Content-MD5: LXgFS0eQZq4VVenD/ymC6A==
    < Last-Modified: Wed, 20 Nov 2013 22:57:22 GMT
    < Cache-Control: private, max-age=60
    < Expires: Thu, 21 Nov 2013 00:39:16 GMT
    < Access-Control-Allow-Origin: *
    < Access-Control-Allow-Credentials: false
    < Access-Control-Allow-Headers: Accept, Accept-Language, Authorization, Cache-Control, Content-Disposition, Content-Encoding, Content-Language, Content-Length, Content-MD5, Content-Range, Content-Type, Date, GData-Version, Host, If-Match, If-Modified-Since, If-None-Match, If-Unmodified-Since, Origin, OriginToken, Pragma, Range, Slug, Transfer-Encoding, X-ClientDetails, X-GData-Client, X-GData-Key, X-Goog-AuthUser, X-Goog-Encode-Response-If-Executable, X-Goog-Correlation-Id, X-Goog-Upload-Command, X-Goog-Upload-Content-Disposition, X-Goog-Upload-Content-Length, X-Goog-Upload-Content-Type, X-Goog-Upload-Offset, X-Goog-Upload-Protocol, X-Goog-Visitor-Id, X-HTTP-Method-Override, X-JavaScript-User-Agent, X-Origin, X-Referer, X-Upload-Content-Length, X-Upload-Content-Type, X-Use-HTTP-Status-Code-Override, X-YouTube-VVT, X-YouTube-Page-CL, X-YouTube-Page-Timestamp
    < Access-Control-Allow-Methods: GET,OPTIONS
    < Server: HTTP Upload Server Built on Nov 14 2013 12:41:34 (1384461694)
    < Alternate-Protocol: 443:quic
    

    References

    http://googleappsdeveloper.blogspot.co.uk/2012/11/announcing-google-drive-site-publishing.html
    http://lifehacker.com/5964262/host-web-pages-on-google-drive
    http://stackoverflow.com/questions/14928287/webviewlink-now-returning-404-not-found
    https://code.google.com/p/google-apps-script-issues/issues/detail?id=2324
    http://www.myblogger-tricks.com/2013/02/how-to-host-your-web-page-content-on.html
    https://googledrive.com/host/0B0ap0WDOII5mbjdwajNBMlRoZUU/index.html

    Thursday, August 29, 2013

    VMware NSX network virtualization platform

    VMware announced its new product NSX. In comparison it should be the same for network virtualization what flagship iESX product was for host virtualization.

    This 2 links give a glimpse into the architecture design:

    http://blogs.vmware.com/networkvirtualization/2013/08/vmware-nsx-network-operations.html
    http://blogs.vmware.com/networkvirtualization/2013/08/vmware-nsx.html

    These 2 pictures from the links above give a high level overview what it does and how it should work:


    On the VMware page we can find as well as these two videos below that present the virtual network challenge for cloud deployments and shows how to solve it with the help of VMware NSX platform. 

    http://bcove.me/idrpiovw
    http://bcove.me/qe36t4fc

    Sunday, August 18, 2013

    How to create a PTR DNS record on Rackspace Cloud

    There is an easy way to create a PTR record on MyCloud portal. But with constant code upgrades and new changes on the portal the button sometimes don't work.




    Problem

    How to create a reverse DNS (PTR) record for a Rackspace cloud server using API.

    Solution

    This little script create a PTR records using Openstack Cloud API: dns-ptr-record.py.

    Example

    Example snippets from ipython console:
     
    In [57]: rec = {'data': '162.13.2.xyz', 'name': 'yourservername.mydomain.com', 'type': 'PTR'}
    
    In [62]: dns.add_ptr_records(a0,rec)
    Out[62]:
    [{u'created': u'2013-08-18T18:15:33.000+0000',
      u'data': u'162.13.2.xyz',
      u'id': u'PTR-579695',
      u'name': u'yourservername.mydomain.com',
      u'ttl': 3600,
      u'type': u'PTR',
      u'updated': u'2013-08-18T18:15:33.000+0000'}]
    
    In [67]: dns.list_ptr_records(a0)
    Out[67]: [<CloudDNSPTRRecord id=PTR-579695, data=162.13.2.xyz, name=yourservername.mydomain.com, ttl=3600>]
    

     
    root@manage2:~# dig -x 162.13.2.xyz  @dns1.stabletransit.com
    
    ; <<>> DiG 9.8.1-P1 <<>> -x 162.13.2.xyz @dns1.stabletransit.com
    ;; global options: +cmd
    ;; Got answer:
    ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 12333
    ;; flags: qr aa rd; QUERY: 1, ANSWER: 1, AUTHORITY: 2, ADDITIONAL: 0
    ;; WARNING: recursion requested but not available
    
    ;; QUESTION SECTION:
    ;xyz.2.13.162.in-addr.arpa.     IN      PTR
    
    ;; ANSWER SECTION:
    xyz.2.13.162.in-addr.arpa. 3600 IN      PTR     yourservername.mydomain.com.
    
    ;; AUTHORITY SECTION:
    2.13.162.in-addr.arpa.  300     IN      NS      ns.rackspace.com.
    2.13.162.in-addr.arpa.  300     IN      NS      ns2.rackspace.com.
    
    ;; Query time: 109 msec
    ;; SERVER: 69.20.95.4#53(69.20.95.4)
    ;; WHEN: Sun Aug 18 18:18:49 2013
    ;; MSG SIZE  rcvd: 130
    

    References
    1. https://github.com/rackspace/pyrax/issues/79
    2. http://docs.rackspace.com/cdns/api/v1.0/cdns-devguide/content/ReverseDNS-123457003.html
    3. http://www.rackspace.com/knowledge_center/article/rackspace-cloud-essentials-6-creating-a-reverse-dns-record
    4. http://www.rackspace.com/knowledge_center/comment/155017



    Thursday, August 15, 2013

    Rackspace Openstack lesson learned from continues integration and continuous deployment

    When a software like Openstack gets bigger and bigger and some of the old features matures and new are added over time, there are going to be a number of challenges and complexities that you will see.

    These challenges will be for the engineering team who wants to keep the development pace and rolling out new features, for the operational team that needs to look after the existing deployed infrastructure and be capable of planning and executing upgraded when needed, as well as for the support team who could be the first front of people who are notified when things get broken and things get broken when you move fast.

    As Rackspace is one of the biggest Openstack consumer out there there are couple of lesson we can learn from them.
    • Continues delivery is an obligatory task if you want to make sure your infrastructure is running the latest code 
    We’re still plugging away on efforts to do continuous delivery from Nova trunk to our production environments. Our deploy pace has slowed as we are learning how to scale this effort and keep up with the pace of commits. [1]
    • You need to have as much automatic testing as possible
    To help facilitate progress, we’ve open-sourced our testing framework, Cloud Cafe, and engaged with the community to talk about how we might move our extensive test suite further upstream in the process.[1]

    Continuous Delivery: While developing our OpenStack-based cloud, we created a Continuous Delivery/Continuous Integration model through which every change to the code goes through automated tests.  This Continuous Delivery pipeline lets us make a lot of changes quickly and gives us the ability to rapidly and confidently deploy new features.[2]
    • Your deployment model can take the benefit of the cloud agility 
    We’re eating our own dog food. We actually use OpenStack to run the control systems for our public cloud.

    If we need more API nodes or additional scheduling capacity, we spin up more instances in our infrastructure cloud, which we call iNova. When upgrading nodes, we spin up new instances with the updated software and roll those into the load balancer. If things work as expected, the old instances are deleted. If there are issues, rollback is as simple as putting the old instances back in rotation. This blue/green deployment approach is a well known pattern. But, doing it in the cloud means we don’t have to have two sets of physical hardware standing around for the task. iNova is now being adopted across our cloud engineering teams for development, testing and production uses.[2]

    References
    1. http://www.rackspace.com/blog/ramping-up-our-openstack-investment-involvement/
    2. http://www.rackspace.com/blog/how-rackspace-re-wrote-the-cloud-with-openstack-continuous-delivery/
    3. https://github.com/stackforge/cloudcafe
    4. http://www.slideshare.net/openstackindia/introduction-to-tempest
    5. https://github.com/openstack/tempest/tree/master/tempest

    Tuesday, August 13, 2013

    Akanda virtual router for virtual networks in Openstack

    The Openstack virtual network, aka Neutron (old name Quantum) is responsible to deliver networking services for other cloud infrastructure projects. There is already quite a lot about it on the Internet and we reviewed couple of  possible deployment scenarios and use cases here (Emerging of the virtual network aka Quantum in Openstack) as well. Going further this route and discussing virtual networking opportunities it is a time to ask another question:

    Does Openstack Virtual Network/Quantum/Neutron support virtual routers?

    I don't have yet a very clear answer to this but found today this interesting project: https://github.com/dreamhost/akanda.

    Looking at DreamHost wiki we can see that they use it to solve a problem of distributing and redirecting traffic in multi tenancy environment to secure, extend and control the networking traffic. As they claim it is being implemented with a help of virtual router, aka akanda.

    http://wiki.dreamhost.com/DreamCompute_Overview

    Network

    Networking services for DreamCompute are delivered through OpenStack's Quantum service, coupled with Nicira's NVP (Network Virtualization Platform), and DreamHost's own Akanda project.
    Nicira initiated and leads the Networking project team within the OpenStack community. Nicira's NVP, in concert with the OpenStack Quantum project, provides the ability to create private networks per tenant with high availability and network hardware independence. Nicira's NVP virtualizes the network; the software does for networks what hypervisors do for servers.
    Another component to DreamCompute networking is the open source Akanda project developed by DreamHost. Akanda serves as a network traffic router (OSI Layer 3) for virtual networks created in DreamCompute. Akanda allows virtual networks to be configured to talk to each other, to LANs, to WANs or to the Internet.
    The combination of OpenStack, Nicira and Akanda bring a level of network flexibility and security that sets DreamCompute apart from the competition.

    Tuesday, July 16, 2013

    Nicira NVP network controller

    Networking is evolution and often going toward automation and virtual networking. Looking at the available resources we can say that there is a lot of stuff to read about Cisco Nexus platform and the VXLAN protocol. Comparing this with the SDN vendors like Nicira or Midokura the situation is not the same. There isn't available open documentation for the products. There is only a few videos on YouTube that demonstrate what the products do and how they look like.

    Below are some of the videos I came across that gives you a glitch into the Nicira NVP cluster.
    • VMware Nicira Network Virtualization and VPLEX
    • OpenStack with VMware vSphere

    Monday, June 10, 2013

    Remote execution and configuration management framework written in Python

    There are number of tools out there that you can use for automation and/or configuration management. Many of these tools are flexible and offer a range of command and GUI applications interfaces as well as programming libraries and API. If you are paranoid or have very unique requisitions you can writhe with a help of them the automaton scripts yourself. This is a simple example code to automate a task using the paramiko Python library.

    Likely, the most popular one are puppet and chef. Puppet is written in Ruby where Chef is in Ruby and Erlang (the newest version).

    Problem

    Is there any automation and configuration framework that is natively written in Python.

    Analisis and results description

    Automation tool called Salt or Saltstack is written in Python and provide a native support for modules written in Python.

    The tool is rapidly evolving and getting popular. On the main project page we can find number of resources like email group, wiki and IRC (http://saltstack.com/community). There is even a YouTube channel. Below is an introduction and example how salt works.

    http://docs.saltstack.com/topics/tutorials/walkthrough.html
    http://www.linuxjournal.com/content/getting-started-salt-stack-other-configuration-management-system-built-python


    Monday, May 27, 2013

    How does the VXLAN protocol work

    The VXLAN is one of the overlay network tunneling protocols that is used to built network infrastructure for cloud environment. Below are some details about the operation and specification.
    • Frame headers definition

    • The traffic between VMs is encapsulated in IP/UDP packets
    • Logical isolation is implemented in a form of logical overlay where the traffic is exchanged between encryption tunnels endpoints
    • VXLAN ID is used to identify the specify isolated L2 cloud network that belongs to a tenant
    • The tunnel endpoints represent the edge of the cloud network infrastructure
    • The tunnel endpoints perform encapsulation and decapsulation
    • It is there where all the logic is implemented to find out where to sent next a packet or to witch VM the packet should be delivered after decapsulation



    • A comprehensive summary and operational features can be found under the links in reference section, below are few of the main characteristics and benefits:
      • It operates over IP and used UDP to carry payload 
      • Multicast support is the only other requirement for switches and routers to support VXLAN 
      • Multicast is used to handle L2 broadcast traffic (like ARP requests)
      • Logical networks can be extended among virtual machines placed in different Layer 2 domains
    References
    1. http://www.definethecloud.net/vxlan-deep-dive
    2. http://www.definethecloud.net/vxlan-deep-divepart-2
    3. http://www.cisco.com/en/US/prod/collateral/switches/ps9441/ps9902/white_paper_c11-685115.html
    4. http://www.emulex.com/artifacts/d658610a-d3b6-457c-bf2d-bf8d476c6a98/elx_wp_all_VXLAN.pdf
    5. http://blogs.cisco.com/datacenter/more-vxlan-qa/
    6. http://blog.scottlowe.org/2011/12/07/revisiting-vxlan-and-layer-3-connectivity/
    7. http://blog.scottlowe.org/2011/12/22/otv-and-vxlan-layer-3-connectivity-compared/

    Sunday, May 26, 2013

    Can Cloud load balancer use other than ServiceNet network

    The are couple of network that you are going to hear when hosting physical servers and using cloud product at Rackspace. The most important one are:
    • Public network
    • ServiceNet network
    • Private cloud network
    Public network is a conventional large scale, 3 tier network (built using access, distribution, core routers and switches) that forward all public traffic. There is cloud public network and there is a dedicated public network.

    ServiceNet is internal network. We use the same name either you have a cloud server or a dedicated server. All your internal communication between cloud-cloud and dedicated-dedicated can or is happening over it. The dedicated ServiceNet is separated from the cloud ServiceNet even though the names are the same.

    Private cloud network is an enhancement on the cloud site that allows you to create your own isolated network for the cloud tenant you created. It can be used for any  traffic you want and you can assign any subneting you wish. This network is implemented in a form of an network overlay with a help of SDN technology (Nicira).

    By default all cloud servers and other cloud products will be connected to cloud ServiceNet network. That means every cloud load balancer at Rackspace can communicate with cloud server using internal network.

    At the moment (May 2013) there is  no support for cloud load balancer (CLB) to sent or receive traffic from private cloud network. If you want to load balance traffic across cloud servers using CLB you have only 2 options:
    • user ServiceNet network interface (10.*)
    • use public network interface (cloud server additional bandwidth costs will be added)
    The public network is not free and you need to pay for every MB you sent out (pricing).

    References
    1. http://support.rightscale.com/06-FAQs/What_is_Rackspace_ServiceNet_and_how_can_I_use_it%3F
    2. http://www.rackspace.com/knowledge_center/article/rackconnect-with-cloud-networks-faq


    Thursday, May 23, 2013

    Midokura Midonet software

    The network virtualization movement is getting bigger and stronger. Looking how serious the VMware is looking into SDN concept ( VMware NSX Network Virtualization ) is is only a matter of time before we start seeing this on a regular basis in data centers.

    With this message in mind let's take a look at one of the SDN vendors like Midokura and its software Midonet.
    1. General availability 

    2. During Openstack Summit in April 2013 Midokura sent a strong message that its Midonet software is available to download. Midonet is Midokura the SDN implementation of the cloud network virtualization concept.

    3. Software

    4. At the moment there is only a little bit of information what it is.

      MidoNet pushes intelligence to the edge of the network, as it takes an overlay-based approach to network virtualization and sits on top of any IP-connected network. [1]

      Its technology is functionally similar to VMware-backed Nicira's, but the approach is different: Midokura has a Level 3 network gateway, whereas Nicira is Level 2. Both companies offer distributed switching at Level 2 [2]

      Midokura uses a continuous-licensing basis for its network virtualization software. The technology is a 5-to-10MB download that runs on top of a JVM on standard server hardware. [2]

    5. Midonet network architecture
    6. Below, on the left we see the logical and on the right the physical topology.

      The main logical concept is based on having a virtual router that your VMs are talking to. As Midonet software is using/is based on Open vSwitch that pacifically means that there is going to be a virtual port attached to Virtual router and VM in Open vSwitch. That way the VM is directly (virtualy) connected to its virtual router / default gateway. The building layout of the physical topology seems to confirms that assumptions.

      The physical topology help us better understanding the Midonet distributed model as well: Midonet software architecture.

      The backend network is a standard IP base physical network infrastructure that aims to provide IP connectivity between the Midonet nodes. The power of Midonet comes from the way it manages its distributed NW State DB (network state data base). This is the critical part and place where is decided what to do with an Ethernet frame/IP packet from or to VM when there is no flow entry on the routers or hypervisor virtual switches.

      This is the link for a full Midokura Midonet presentation (in Japan).


    References
    1. http://www.midokura.com/press-releases/midonet_launch_at_openstacksummit2013/
    2. http://www.theregister.co.uk/2013/04/02/midokura_sdn_funding/

    Tuesday, May 21, 2013

    Architecture frameworks

    Did you see big systems implementations?
    Did you participated in a system implementation or deployment?
    What is your current role?
    What is your carrier goal?

    Every company is organised in a different way. Every company is build in a different way. But it all starts from a vision that is going to be mapped into an architecture. Once the concept or the first sketches of the architecture are established the implementation process is going to follow.

    It is an example how it can look like. If you are interested in other options it is worth to take a look at some of the established (enterprise) architecture frameworks. Generally speaking, it is rather a quite difficult document to read as it tries to address the issue of an enterprise company. But even though its complexities it still gives a good overview how complex some processes, implementations or deployments can be. Below is a table from FEAF framework that list key roles and teams for an organisation.



    This one below shows the concept of an architecture from a high level point of view.


    The more detailed description of the architecture domains can be found here. Personally, I'm specializing in the area of technical infrastructure architecture. With the specialization in virtual networking and cloud network in IaaS cloud.

    Technical architecture or infrastructure architecture: The structure and behaviour of the technology infrastructure. Covers the client and server nodes of the hardware configuration, the infrastructure applications that run on them, the infrastructure services they offer to applications, the protocols and networks that connect applications and nodes.

    References
    1. http://en.wikipedia.org/wiki/Solution_architecture
    2. http://en.wikipedia.org/wiki/Architecture_domain
    3. http://en.wikipedia.org/wiki/TOGAF
    4. http://pubs.opengroup.org/architecture/togaf8-doc/arch/
    5. http://en.wikipedia.org/wiki/Multitier_architecture
    6. http://en.wikipedia.org/wiki/Peer-to-peer