Search This Blog

Friday, March 22, 2013

Code changes in Openstack Grizzly release

There is almost a new Openstack version called Grizzly around the corner. Some stats what can we expect base on [1] and [2].

nova
added lines 281036
removed lines 214574
total lines added 66462
commits 1889

https://github.com/openstack/nova/graphs/code-frequency
Quantum
added lines 92376
removed lines 41864
total lines added 50512
commits 602

https://github.com/openstack/quantum/graphs/code-frequency
Keystone
added lines 28488
removed lines 17265
total lines added 11223
commits 289
Glance
added lines 15717
removed lines 13163
total lines added 2554
commits 253
Cinder
added lines 73344
removed lines 72825
total lines added 519
commits 306

https://github.com/openstack/cinder/graphs/code-frequency
Horizon
added lines 187351
removed lines 125838
total lines added 61513
commits 160
Swift
added lines 17270
removed lines 6313
total lines added 10957
commits 222

https://github.com/openstack/swift/graphs/code-frequency
References
  1. http://www.slideshare.net/enovance/meetup-open-stackgrizzly-17372714
  2. https://github.com/openstack

Openstack deployment options

Openstack wants to be as flexible as possible. It means that each service that is developed under the umbrella of Openstack has to be written in a modular way to accept different backend system depending on user preferences. From a high level point of view it means that a service need to have a well defined internal and external API, and the more specific technical implementation details are left for backend systems.

The Openstack specific public (external) API is one milestone for every project. If the API is not reach, flexible and useful enough it will not empower uses when consuming the service. On the other side, if the internal service level API is badly design it may cause issues for example when integrating with other components, exchanging messages, causing bottlenecks or hinder vertical scalability to expand system capacity.

Below is a list of deployment options for Folsom and Grizzly Openstack release.
  • External API
    •  XML
    •  JSON
  • Possible hypervisors for OpenStack Compute
    • KVM
    • Xen
    • Citrix XenServer
    • Microsoft HyperV
    • VMware ESX
    • LXC
  • Possible OpenStack Block Storage drivers 
    • Coraid
    • EMC
    • GlusterFS
    • Huawei
    • LVM
    • NetApp
    • Nexenta
    • NFS
    • Ceph RBD
    • SAN/HP
    • SAN/Solaris
    • Scality
    • Sheepdog
    • SolidFire
    • Storwize
    • Windows
    • Xenapi
    • XIV
    • Zadara
  • Possible OpenStack Network backends 
    • Big Switch
    • Brocade
    • Cisco
    • Hyper-V
    • Linux Bridge
    • MidoNet
    • NEC
    • Nicira
    • Open vSwitch
    • PLUMgrid
    • Ryu
  • OpenStack Identity drivers
    • LDAP
    • SQL
    • PAM
    • KVS

How to manually create shadow password for a user in Linux

On Linux systems users passwords are stored in /etc/shadow file. An example line showing a password and account details for a 'demo' user on my system looks like this:
 
$ grep demo /etc/shadow
demo:$6$DdiZmmSe$eSXGHIB2gx.cHY.PR.Tfz8l00iStSgea0o7glv2ptBq8FpfSjz5XVU2GgCVzr72zAx4wG4gfYXucgoOGb3Rb7/:15786:0:99999:7:::

Problem

How to compute and generate a user password so it can be copied into the shadow file directly.

Solution and results description

The description of how the password is created can be found here:
 
$ man shadow
encrypted password
           Refer to crypt(3) for details on how this string is interpreted.


$ man 3 crypt
Glibc Notes
       The glibc2 version of this function supports additional encryption algorithms.

       If salt is a character string starting with the characters "$id$" followed by a string terminated by "$":

              $id$salt$encrypted

       then instead of using the DES machine, id identifies the encryption method used and this then determines how the rest of the password string is  interpreted.   The  following
       values of id are supported:

              ID  | Method
              ─────────────────────────────────────────────────────────

              1   | MD5
              2a  | Blowfish (not in mainline glibc; added in some
                  | Linux distributions)
              5   | SHA-256 (since glibc 2.7)
              6   | SHA-512 (since glibc 2.7)

       So $5$salt$encrypted is an SHA-256 encoded password and $6$salt$encrypted is an SHA-512 encoded one.

       "salt"  stands  for the up to 16 characters following "$id$" in the salt.  The encrypted part of the password string is the actual computed password.  The size of this string
       is fixed:

       MD5     | 22 characters
       SHA-256 | 43 characters
       SHA-512 | 86 characters


Analyzing the shadow line for the demo user we can see that his password:
  • uses SHA512 algorithm
  • it was generated with a salt string DdiZmmSe
  • it is 86 char long
  •  
    $ python -c "print len('eSXGHIB2gx.cHY.PR.Tfz8l00iStSgea0o7glv2ptBq8FpfSjz5XVU2GgCVzr72zAx4wG4gfYXucgoOGb3Rb7/')"
    86
    
The first impression that we could simply use a tool to generate an SHA digest isn't going to work unfortunately. The reason is that SHA512 generates only a 512 bit long message digest (that is 64 char string) and the password in shadow file is 86 char long.

Further researching found out that even though the 'crypt' function uses the standard SHA crypto function it varies in a number of ways to produce the 86 char long string. An interesting blog describing the algorithm can be found here: http://www.vidarholen.net/contents/blog/?p=33.

There are number of ways you generate our password:
  • we can use a bash script 
https://github.com/rtomaszewski/experiments/blob/master/shadow_pass.sh
 
$ ./shadow_pass.sh demo DdiZmmSe
$6$DdiZmmSe$eSXGHIB2gx.cHY.PR.Tfz8l00iStSgea0o7glv2ptBq8FpfSjz5XVU2GgCVzr72zAx4wG4gfYXucgoOGb3Rb7/
  • we can write a little script and call the crypt function directly to generate the password
http://serverfault.com/questions/330069/how-to-create-an-sha-512-hashed-password-for-shadow
 
$ python -c "import crypt, getpass, pwd; print crypt.crypt('demo', '\$6\$DdiZmmSe\$')"
$6$DdiZmmSe$eSXGHIB2gx.cHY.PR.Tfz8l00iStSgea0o7glv2ptBq8FpfSjz5XVU2GgCVzr72zAx4wG4gfYXucgoOGb3Rb7/

Further reading


Tuesday, March 12, 2013

Rackspace Private Cloud v3 available

In my previous post How to install Rackspace Private Cloud (Alamo) on a single physical server I described how to install Alamo version 2. Since then the software has evolved and a new versions 3 with new features is available for download and testing.

Instalation

Full instruction how to install it can be found here: http://www.rackspace.com/cloud/private/script/

A compact version looks like:

curl -L "http://sh.opencenter.rackspace.com/install.sh" | bash -s server
curl -L "http://sh.opencenter.rackspace.com/install.sh" | bash -s dashboard
curl -L "http://sh.opencenter.rackspace.com/install.sh" | bash -s agent

Architecture has changed

Alamo version 3 comes with a modified and extended architecture to support new features. A high level diagram can be found here: http://www.rackspace.com/cloud/private/openstack_software/

More documentation

Knowledge centre articles:
http://www.rackspace.com/knowledge_center/getting-started/rackspace-private-cloud

More instruction how to install and access to dedicated form for Rackspace Private Cloud
https://privatecloudforums.rackspace.com/viewtopic.php?f=4&t=345&start=0

Opencenter, installer and more about source code is here:
https://github.com/rcbops

Monday, March 11, 2013

ASA ssh login problem

Working for ISP is big fun. From all the work you do there is one routine like swapping of network devices (for example Cisco ASA firewall) that you are going to do. Not going into too much details the process is straight forward and requires:
  • copy the config to new device
  • rack the new device
  • make sure that the switches and VLANs are configured properly
  • change routing info if needed 
  Problem

After putting new ASA FW into rack you can connect using serial line but you can't access it over SSH. You getting this error message.
 
$ ssh 1.1.1.77
ssh_exchange_identification: Connection closed by remote host

Troubleshooting and solution

From serial console access enable debugging:
 
# debug ssh

Connect over ssh. You are going to see this logs on console:
 
Device ssh opened successfully.
SSH0: SSH client: IP = '212.100.225.42'  interface # = 2
SSH: unable to retrieve default host public key.  Please create a defauth RSA key pair before using SSH
SSH0: Session disconnected by SSH server - error 0x00 "Internal error"

Searching for 'unable to retrieve default host public key' finds the links in reference sections.  To fix this we need:
 
fw-asa(config)# crypto key generate rsa
INFO: The name for the keys will be: 
Keypair generation process begin. Please wait...

Once ASA has its own RSA key to use for SSH handshaking the logs from a sucessful SSH session looks like:
 
fw-asa# 
Device ssh opened successfully.
SSH0: SSH client: IP = '212.100.225.42'  interface # = 2
SSH: host key initialised
SSH: license supports 3DES: 2
SSH: license supports DES: 2
SSH0: starting SSH control process
SSH0: Exchanging versions - SSH-2.0-Cisco-1.25
SSH0: send SSH message: outdata is NULL
server version string:SSH-2.0-Cisco-1.25SSH0: receive SSH message: 83 (83)
SSH0: client version is - SSH-2.0-OpenSSH_4.3
client version string:SSH-2.0-OpenSSH_4.3SSH0: begin server key generation
SSH0: complete server key generation, elapsed time = 1830 ms
SSH2 0: SSH2_MSG_KEXINIT sent
SSH2 0: SSH2_MSG_KEXINIT received
SSH2: kex: client->server aes128-cbc hmac-md5 none
SSH2: kex: server->client aes128-cbc hmac-md5 none
SSH2 0: expecting SSH2_MSG_KEXDH_INIT
SSH2 0: SSH2_MSG_KEXDH_INIT received
SSH2 0: signature length 143
SSH2: kex_derive_keys complete
SSH2 0: newkeys: mode 1
SSH2 0: SSH2_MSG_NEWKEYS sent
SSH2 0: waiting for SSH2_MSG_NEWKEYSSSH0: TCP read failed, error code = 0x86300003 "TCP connection closed"
SSH0: receive SSH message: [no message ID: variable *data is NULL]

SSH2 0: Unexpected mesg type receivedSSH0: Session disconnected by SSH server - error 0x00 "Internal error"

References
  1. http://www.myteneo.net/blog/-/blogs/accessing-cisco-asa-using-ssh/
  2. http://ciscotalk.wordpress.com/2011/08/31/enabling-ssh-on-a-cisco-asa/

Sunday, March 10, 2013

Openstack Gerrit code review process details

Are you looking to contribute code to one of the Openstack projects (example include Nova, Quantum or Glance and many more). If the answer is yes, you definitely need to get familiar with the code review that Openstack enforces. A good overview of what this is can be found here Code review process in Openstack uses Zuul.

On a technical site the process has been implemented with a help of Gerrit system. From end user perspective you get access to a nice and good looking page in a browser that helps you to review, comment and approve code changes that should be committed to master repository. A quick and good introduction of what Gerrit do this and how it helps can be found here: Gerrit Code Review - A Quick Introduction.

Example how this interface looks like:

Code Review: https://review.openstack.org

When you select a link you dig into more details: (example) https://review.openstack.org/#/c/23878/


In this screen we can find:

  • linked Bug id if exists (https://code.launchpad.net/bugs/1131759
  • linked Blueprints if exists
  • reviewer list
  • one or more patches with proposed code changes
  • comments and suggestions before the code can be accepted in the mainstream repository
But the best part of Gerrit is its capability to pull the original file and create a diff to visual the changes. An example from the review #23878 is seen below.

https://review.openstack.org/#/c/23878/4/heat/tests/test_api_openstack_v1.py

Saturday, March 2, 2013

Text developer editor with Python API

I often need to work between Linux and Windows systems. Under every OS I have my favorite tools I like that help me to get the job done. But there has always been one tool that I wasn't very happy with: a good text editor.

Problem

What is a good cross platform editor with development features that is written and integrated with Python.

Analisis and discussion 

When I code I always like to know the editor so I can quickly and comfortably navigate in a single or multiple files at the same time. I've found recently one that I tend to use more: Sublime. The other one I was using for a long while was Notepad++ but it was only limited to Windows.

Why Sublime works for me:
My simple config

You can view and change all global settings under Menu - Preferences - Settings - Default but a better way is to create a local customized user preferences file.

To modify user settings open the following file under Menu - Preferences - Settings - User and copy or modify these options:
 
"fade_fold_buttons"        : false,
"highlight_line"           : true,
"auto_complete_size_limit" : 44194304,
"tree_animation_enabled"   : true,

My packages

Below are some of my packages I'm using

How to highlight a whole line in Sublime like in Notepad++ 

In notepad++ when editing your can enable whole like to be highlighted (here are some example screenshots). To achieve the same effect in Sublime you need to enable the highlight_line: true.

References and documentation

http://www.sublimetext.com/2
http://docs.sublimetext.info/en/latest/index.html