Archive

Archive for May, 2009

Python HTTPConnection bound to network interface

May 27th, 2009 1 comment

The web server I use at work are multi homed with the default route being the internal management network. We came across an issue where we wanted make a XMLHTTPRequest for a data feed from another company into our web app. We all know due to cross-site scripting attacks this is no longer possible so we had to write a little proxy script to pull the data and serve it from our own site. The standard python httplib doesn’t have the ability to bind to a specific interface so I have done a little sub-classing and now have a HTTPConnection which allows me to bind to a specific interface. Hope this helps someone as from my searching it seems to be a common request. You will meed to change the IP address to match your setup :-)

import httplib
import socket
 
 
 
class HTTPConnectionInterfaceBound(httplib.HTTPConnection):
    """This class allows communication via a bound interface for 
       multi network interface machines."""
 
    def __init__(self, host, port=None, strict=None, bindip=None):
        httplib.HTTPConnection.__init__(self, host, port, strict)
        self.bindip = bindip
 
 
    def connect(self):
        """Connect to the host and port specified in __init__."""
        msg = "getaddrinfo returns an empty list"
        for res in socket.getaddrinfo(self.host, self.port, 0,
                                      socket.SOCK_STREAM):
            af, socktype, proto, canonname, sa = res
            try:
                self.sock = socket.socket(af, socktype, proto)
                if self.debuglevel > 0:
                    print "connect: (%s, %s)" % (self.host, self.port)
                if self.bindip != None :
                    self.sock.bind ((self.bindip, 0))
                self.sock.connect(sa)
            except socket.error, msg:
                if self.debuglevel > 0:
                    print 'connect fail:', (self.host, self.port)
                if self.sock:
                    self.sock.close()
                self.sock = None
                continue
            break
        if not self.sock:
            raise socket.error, msg
 
 
 
conn = HTTPConnectionInterfaceBound('www.thegoldfish.org', 80, bindip='192.168.56.83')
conn.request("GET", "/")
r1 = conn.getresponse()
print r1.status, r1.reason
print r1.read()
Categories: Python Tags: ,

Using external scripts with django models

May 5th, 2009 No comments

I have used a few web frameworks over the years but I think I have finally found the one that suits my particular needs. I have played with RoR, Turbo Gears, Catalyst and a couple of others but none have actually made me want to write code instead of hoping that it allows me to write less. That was until I discovered Django. A friend of mine had said he was using for his website it but for some reason I managed to get it stuck in my head that he was using Mambo CMS so I never really paid it much attention.

Something that I usually need in a web application is some way to interact with it from the command line, I am a systems administrator – CLI is what I do. In the past I have just connected directly to the database from my scripts but it has always bothered my that I have already done all this business logic in my models and I end up having to repeat it in a limited fashion. I have had a brief look around a couple of times but never found anything that works to my satisfaction until I found the following code on this site by James Bennett http://www.b-list.org/weblog/2007/sep/22/standalone-django-scripts/. James has several methods on that page but I think that this one will be best if you plan on distributing your application.

import os
from optparse import OptionParser
 
usage = "usage: %prog -s SETTINGS | --settings=SETTINGS"
parser = OptionParser(usage)
parser.add_option('-s', '--settings', dest='settings', metavar='SETTINGS',
                          help="The Django settings module to use")
(options, args) = parser.parse_args()
if not options.settings:
    parser.error("You must specify a settings module")
 
    os.environ['DJANGO_SETTINGS_MODULE'] = options.settings
 
LOG_FILENAME = 'logging.log'
logging.basicConfig(filename=LOG_FILENAME,level=logging.INFO,)

The issue with this was after a while I realised that it didn’t work unless you were in the base directory of your application, this may be because I was in doing something wrong – I am relatively new to python. Then I thought how does Django do it ? The answer to this lived in the core.management.__init__ method setup_environ(). Anyway I ended up using a mashup of the two concepts to arrive at the following code. It is a bit long winded but it works from any directory so the script can be anywhere. It sets the environment up based on the settings file being in the normal place – in the base directory of the application.

import os
import sys
from optparse import OptionParser
 
# Parse the command line options
usage = "usage: %prog -s SETTINGS | --settings=SETTINGS"
parser = OptionParser(usage)
parser.add_option('-s', '--settings', dest='settings', metavar='SETTINGS',
                          help="The Django settings module to use")
(options, args) = parser.parse_args()
if not options.settings:
    parser.error("You must specify a settings module")
 
# set
project_directory, settings_filename = os.path.split(options.settings)
if project_directory == os.curdir or not project_directory:
    project_directory = os.getcwd()
project_name = os.path.basename(project_directory)
settings_name = os.path.splitext(settings_filename)[0]
sys.path.append(os.path.join(project_directory, os.pardir))
project_module = __import__(project_name, {}, {}, [''])
sys.path.pop()
 
os.environ['DJANGO_SETTINGS_MODULE'] = '%s.%s' % (project_name, settings_name)

With that added at the top of my scripts I can keep may maintenance scripts in a place more logical like /usr/local/bin instead of the project root and run them like this

/usr/local/bin/myscript.py --settings=/path/to/project/settings.py
Categories: Django, Python Tags: ,

libvirt-qpid and python

May 2nd, 2009 No comments

Libvirt is fast becoming the standard tool for managing virtual machines on Linux and Qpid is the Apache foundations new implementation of AMQP which is the first open standard for Enterprise Messaging. These two technologies have the potential to work in well together for large virtualization installations and luckily for us the good guys in the libvirt team have done just that http://libvirt.org/qpid/ but there are currently very few examples on how to use it. I am putting this brief tutorial in their wiki as a starting point for others but will continue to publish my experiences here.

Installation

libvirt-qpid is currently available in Fedora 10 repositories so you can install it using yum

yum -y install libvirt-qpid qpidd python-qpid
chkconfig libvirt-qpid on
chkconfig qpidd on
service libvirt-qpid start
service qpidd start

Testing that it is running

We can check that it is running using ”qpid-tool” and the list command

# qpid-tool
Management Tool for QPID
qpid: list
Management Object Types:
ObjectType                 Active  Deleted
============================================
com.redhat.libvirt:domain  6       0
com.redhat.libvirt:node    1       0
com.redhat.libvirt:pool    1       0

Simple client in python

Now that we have it running lets make a simple client to get information from it. To do this I use python. The following is a simple script that does some of the basics

#!/usr/bin/env python
 
from qmf.console import Session
from yaml import dump
 
sess = Session() # defaults to synchronous-only operation. It also defaults to user-management of connections.
 
# attempt to connect to a broker
try:
    broker = sess.addBroker('amqp://localhost:5672')
    print "Connection Success"
except:
    print "Connection Failed"
 
domains = sess.getObjects(_class='domain', _package='com.redhat.libvirt.domain')
 
# Print a list of the domains
for d in domains:
    print d
 
# Select the first domain
domain = domains[0]
 
# Print a list of the properties of the domain
print 'Properties:'
props = domain.getProperties()
for prop in props:
    print "\t",prop
 
# Access a value of a property and print it
print domain.name
 
# Print a list of the methods of the domain
print 'Methods:'
meths = domain.getMethods()
for meth in meths:
    print "\t",meth
 
# Ca method of the domain and print it
xmldesc =  domain.getXMLDesc()
 
# Call another method of the domain and print the result
if domain.state == 'running':
    result = domain.shutdown()
    print result
else:
    result = domain.create()
    print result
 
# Disconnect from the broker (otherwise we hang the terminal
sess.delBroker(broker)

Links

http://qpid.apache.org/qmf-python-console-tutorial.html

Categories: AMQP, Libvirt, Python Tags: