How to avoid Rackspace cloud API limits when creating cloud servers (cloudservers.exceptions.OverLimit)

The Rackspace Cloud API has number of limitations. One of them is how many cloud server you can create in 1 minute. The current limitations [1] looks like:

Verb	URI		RegEx		Default
POST	*		.*		10/min
POST	*/servers		^/servers		50/

When your program [2] tries to create more than 10 cloud servers a minute you are going to run into error and start seeing this cloudservers.exceptions.OverLimit exception.

$ python performance-single-cs.py -v -t 1 -s 13 -u user -k key  run | tee log.$(date +%s).txt

Traceback (most recent call last):
  File "performance-single-cs.py", line 349, in module
    Main().run()
  File "performance-single-cs.py", line 346, in run
    self.test_performance(user,key, sample, cs_count, timeout)
  File "performance-single-cs.py", line 298, in test_performance
    t.test_multi_cs_perf(sample, cs_count, timeout)
  File "performance-single-cs.py", line 250, in test_multi_cs_perf
    cs_records=self.cs_create_all(cs_count, i+1)
  File "performance-single-cs.py", line 225, in cs_create_all
    server=self.cs_create(k, sample_nr)
  File "performance-single-cs.py", line 178, in cs_create
    server=sm.create(name, image, flavor)
  File "/usr/lib/pymodules/python2.7/cloudservers/servers.py", line 172, in create
    return self._create("/servers", body, "server")
  File "/usr/lib/pymodules/python2.7/cloudservers/base.py", line 26, in _create
    resp, body = self.api.client.post(url, body=body)
  File "/usr/lib/pymodules/python2.7/cloudservers/client.py", line 69, in post
    return self._cs_request(url, 'POST', **kwargs)
  File "/usr/lib/pymodules/python2.7/cloudservers/client.py", line 50, in _cs_request
    resp, body = self.request(self.management_url + url, method, **kwargs)
  File "/usr/lib/pymodules/python2.7/cloudservers/client.py", line 37, in request
    raise exceptions.from_response(resp, body)
cloudservers.exceptions.OverLimit: Too many requests... (HTTP 413)

A naive code that was used in the first version of my program was simply trying to sent as many create requests as possible. This of course is not going to work if we increase servers above the allowed limit.


Code demonstration

An exaple of the naive code.

	def cs_create(self, count, sample_nr):
	name='test' + str(int(time.time()))
	image=112
	flavor=1
	log("[%2d][%2d] creating image: " % (sample_nr, count) + pformat( {'name': name, 'image' : image, 'flavor' : flavor } ) )
	sm=self.cs.servers
	server=sm.create(name, image, flavor)
	return server
	def cs_create_all(self, cs_count, sample_nr):
	log("[%2d][ ] starting test nr %d, creating %d cloud server, please wait ..." % (sample_nr, sample_nr, cs_count) )
	cs_records = []
	for k in range(1, cs_count+1):
	date_start=datetime.datetime.now()
	server=self.cs_create(k, sample_nr)
	cs_records.append(cs_record)
	return cs_records

view raw naive_cloud_api_limits.py hosted with ❤ by GitHub

A code that is more inteligent looks like this.

	# example code that creates cloud server and try not to hit the cloud API climits
	cs_count=13 # number of cs to build
	build_nr=1
	delayed_10s=False
	while build_nr <= cs_count :
	if 0==build_nr % 11 and not delayed_10s:
	time.sleep(60+10)
	try:
	server = self.cs_create(build_nr, sample_nr)
	delayed_10s=False
	except exceptions.OverLimit:
	time.sleep(10)
	delayed_10s=True
	continue
	cs_records.append(cs_record)
	build_nr+=1

view raw cloud_api_limits.py hosted with ❤ by GitHub

References

1. http://docs.rackspace.com/servers/api/v1.0/cs-devguide/content/Rate_Limits-d1e1017.html
2. https://github.com/rtomaszewski/cloud-performance