Planet Cixar

lhc11.jpg 990×640 pixels

2008-10-06T23:53:57+00:00

That's not an LHC. It's a portal to other worlds!

kriskowal: This tweet was made by a Python program.

2008-10-06T23:33:27+00:00

kriskowal: This tweet was made by a Python program.

First look: latest Fedora and Ubuntu betas really shine

2008-10-06T07:00:00+00:00

Ars takes a look at the latest beta releases of Ubuntu 8.10 and Fedora 10. These betas include impressive new features and reflect the growing maturity of the Linux desktop.

War of the Worlds 2.0 Update

2008-10-06T06:21:00+00:00

It turns out I write a lot of words and say very little. Here's what you need to know about the upcoming alien invasion:

Follow @wotw2
On Halloween, tweet in earnest about the unfolding alien invasion. Make your own story. Keep in touch with your friends like you would in any other disaster situation. Try to imagine what you would tweet as the aliens land, emerge, and lay waste to every metropolis around the world.
Email wotw2@cixar.com if you want to help plan the story or provide technical assistance.

kriskowal: @segphault also bought Anathem. It's now high on my pile of books to read.

2008-10-06T05:09:59+00:00

kriskowal: @segphault also bought Anathem. It's now high on my pile of books to read.

kriskowal: Readers Digest version: follow @wotw2 (gasp I said "it") and tweet about your "experiences" as the alien invasion unfolds.

2008-10-06T05:04:57+00:00

kriskowal: Readers Digest version: follow @wotw2 (*gasp* I said "it") and tweet about your "experiences" as the alien invasion unfolds.

kriskowal: War of the Worlds 2.0: http://tinyurl.com/4zm922 -- Halloween Reenactment on Twitter. Spread the word.

2008-10-06T05:01:21+00:00

kriskowal: War of the Worlds 2.0: http://tinyurl.com/4zm922 -- Halloween Reenactment on Twitter. Spread the word.

Ask a Wizard: War of the Worlds 2.0

2008-10-06T04:54:30+00:00

Let's reenact War of the Worlds on Twitter, Delicious, and in our Blogs for Halloween. Here's the plan.

War of the Worlds 2.0

2008-10-06T04:51:38+00:00

Last week, Josh Lewis; friend, former Apple coworker, and Lex Luthor hairstyle enthusiast; got me thinking about fiction on Twitter. Then, Ryan Paul pointed out that he had written an article about Twitter fiction already. Here's the idea. Let's (and by "us", I mean everyone) reenact The War of the Worlds, on the Internet, for Halloween!

CBS ran the Orsen Welles War of the Worlds radio program in the tense times leading up to the second World War. The aforementioned Wikipedia article claims that Adolf Hitler denounced the program as a sign of democratic decadence. Without commercial breaks and interpolated among real news, the program was broadcast in earnest and public panic ensued. War of the Worlds was the Ultimate Prank, never to be reproduced. As an homage, let's take Halloween to tweet, share bookmarks and status messages, and blog in earnest about the alien invasion in progress. Everyone, tell your story.

The War of the Worlds has several phases that I believe have strong analogs in the tense times leading to World Web II (point oh).

a noteworthy astronomer speculates on the possibility of an alien invasion. This would be a good time to talk about the Drake equation in your blog, especially if astronomy is your hobby. Send an email to wotw@cixar.com with your blog so we can proliferate it on the @wotw2 Twitter account.
The alien invasion occurs. Follow @wotw2 to keep in sync with the progress of the invasion. This Twitter feed will automatically update, in general terms, the unfolding of the alien invasion like clockwork throughout the world. Coordinate with Tweeters in your area to tell local stories.

cylinders fall from the sky. Tweet about where you are. Ask your friends where they are. Form posses. Skip town or take a closer look.
tripods emerge. Flee, get stuck in traffic, or take refuge and tell us what you see.
Martians begin obliterating every Terran metropolitan area with heat rays. Don't call them heat-rays; that would be a dead giveaway. Describe what they do and come up with your own name! Do you work in a public service like hospitals or fire? What's your job and what do you do? Do you organize your coworkers and flee? Do you head for the hills with your go-bag?
Military, local militia, and national guard units get organized and attack the alien invaders. Do you serve in the military? This is your last chance to tell us where you're headed. Do you have family in a militia? Try to keep in touch and let us know how and where they valiantly fought and lost.
The invasion spreads from cities to countryside.
Tripods begin to shut down an malfunction. Are you near one? Do you take a closer look?

After the threat dies down, people begin to blog and speculate about what happened, and every topic near and dear to them.
The curtain rises. Blog, link, and tweet about the experience.

The "War of the Worlds 2.0" event will be synchronized, on Halloween Friday, with the http://twitter.com/wotw2 account and we're writing automation with Ryan Paul's Gwibber tool to automatically post Tweets to various services on that day. If you would like to make additional accounts to synchronize local events, please send an email to wotw2@cixar.com with a Twitter account name and password and we'll hook you up with edit privileges for the tweet plan on Google Docs. Tweets (rows) will be deployed for each column (account).

Let's tell a story!

I started reading Anathem today and I'm really enjoying it so far.

2008-10-06T04:36:54+00:00

I started reading Anathem today and I'm really enjoying it so far.

By segphault 41 minutes ago.

Multi-Dimensional Analog Literals in C++

2008-10-05T19:44:46+00:00

"Have you ever felt that integer literals like "4" don't convey the true size of the value they denote? If so, use an analog integer literal instead."

JSON Basics

2008-10-05T07:38:58+00:00

JSON is a strict subset of JavaScript, particularly a subset of its object-literal notation, discovered and specified by Doug Crockford. The subset is sufficient to make the creation of parsers and formatters in various languages nearly trivial, while completely trivializing the process of parsing the notation in a web browser.

Object-literals in JavaScript are very similar to their analogs in many other languages, like Perl, Python, and even AppleScript. The notation provisions text strings, numbers, booleans, and the literal null for all elemental (scalar) data. Then the notation provides Arrays for ordered values and Objects for unordered, unique, string-to-anything key-value-pair mappings. With these types, you can express most hierarchical data easily.

10
3.1415
"a"
true
false
null
[1, 2, 3]
{"a": 10, "b": 20}
[{"a": 10}, {"a": 20}]

JSON makes a couple simplifications even on JavaScript's object literal grammar. These make it even easier to write parsers and formatters.

a JSON expression contains no new-line characters.
all strings, including keys in objects, are enquoted between double quotes.

JSON joins the ranks of many other markup languages that we can use to easily transfer data among web servers, but its primary function is as a common data interchange language between web browser clients and web servers hosted in the numerous languages of the web. JSON is well adapted for this space because it can take advantage, through various channels, of every web browser's fast, built-in JavaScript interpreter.

There are two flavors of JSON services with subtle differences for both clients and servers. One is JSON proper, which I will call XHR JSON because it uses an XML HTTP Request. The other is called JSONP or "JSON with Padding". You would use one, the other, or neither based on security and performance concerns.

XHR JSON

XHR JSON uses a feature that exists in one form or another in all modern web-browsers, called an XML HTTP Request, and a function in JavaScript that lets you evaluate arbitrary text strings as JavaScript programs, called eval. With an XML HTTP Request, the client can make an HTTP Request to any URL on the same domain as the hosting page. This constraint is called the "Same Origin Policy". Unfortunately, the policy is in many cases not sufficient to isolate vulnerability to paths of a particular domain, nor to prevent cross site scripts from using the data (more on that later). Whether the security mechanism is effective or not is a longer and later discussion. The point being that this mechanism is the crux of your choice between XHR JSON and JSONP.

With an XHR, you request plain text from a URL on the same domain as your page, then you evaluate it as a JavaScript program. Performing a cross-browser compatible XML HTTP Request isn't trivial in itself, so let's assume that I'm using a library that provides an xhr function that synchronously (blocks) until an HTTP request is complete and returns the content of the HTTP response as a String. There are other variations on xhr for asynchronous requests and grabbing XML.

var text = xhr("/some.json");
var json = eval(text);

In this case, the "/some.json" contains unadulterated JSON. However, because your JSON is likely to be an Object like {"a": 10}, we need to make a special arrangement. A JavaScript program that is merely an Object literal would be mis-parsed initially as a code block. For this reason, we must force the JavaScript parser into expression context instead of statement context. For this reason, we wrap the JSON string in parentheses or assign it to a variable.

var text = xhr("/some.json");
var json = eval("(" + text + ")");

I prefer parentheses because I haven't, in my fallible memory, encountered a browser that supported XHR but wasn't standards compliant enough for the eval function to return the value of the last evaluated expression.

When you use an XHR to grab JSON, you are vulnerable to the host, depending on it to not give you an alternate JavaScript program that insidiously evaluates to the same data. For that reason, a lot of JSON libraries engage in the slower and dubious attempt to validate the JSON before evaluating it with a regular expression. The bottom line is that you're vulnerable to your server when you use XHR and eval.

That being said, it's better to get an exception early than an untraceable error later. For that reason, attempting to validate JSON is a good idea. There's another one: variable laundering. The eval function inherits the calling context's scope chain. This means that the server can read and alter any variables on your scope chain. I use an evalGlobal routine to launder my scope chain. This doesn't give security all by itself, but it could help lead to a secure system down the way and can turn a bunch of silent name resolution errors or data leaks into thrown NameErrors.

(function (evalGlobal) {
	var text = xhr("/some.json");
	validateJson(text);
	var json = evalGlobal("(" + text + ")");
	...
})(function () {
	return eval(arguments[0]);
})

I'll give a detailed explanation of evalGlobal in another article. For now, suffice it to say, this is much more elegant than using a variable to capture the JSON value, although it is possible:

var text = xhr("/some.json");
var json;
eval("json = " + text);

JSONP

JSONP is different than XHR JSON in both the way the server hosts the data and in the way that the client consumes it. On the client side, the user adds a <script> tag to their own page. The source of the script is the URL of a server side program with the name of a callback function that the client places in global scope sent as a query argument.

<script src="http://other.com/some.jsonp?callback=foo">

The client script arranges to receive parsed and evaluated JSON data asynchronously by adding a foo method to the global scope, the window object.

window.foo = function (json) {
	...
	delete window.foo;
};
var script = document.createElement('script');
script.src = "http://other.com/some.jsonp?callback=foo";
document.getElementsByTagName('head').appendChild(script);

When you add a script tag to your document, browsers are kind enough to notice and automatically send an HTTP request to fetch the request JavaScript program. In this case, your page trusts the script on other.com to return a JavaScript program that will call your "foo" function and pass it some JSON data.

foo({"a": 10});

With JSONP, the client is vulnerable to the remote server because it can opt to write an arbitrary JavaScript program before calling foo, if at all. So, you should only use JSONP to request data from domains that you trust.

Also JSONP is slower and less responsive to errors than XHR JSON can be. The only signal that you receive as to the progress or status of a JSONP request is whether your callback gets called in a timely fashion. XHR JSON is preferable in all situations where it is possible and it may even make sense to create a proxy to the other domain from your same domain server. Using a proxy also gives you an opportunity to validate the JSON on the server-side, where computation time is cheap.

Same Origin XHR JSON

There's another trick with XHR JSON, and I'm not sure what scenarios require it. Some people can use the JSONP technique to fetch JSON from a foreign domain. These folks can't send HTTP headers or cookies, and they never get an opportunity to alter the text of the HTTP request before it's evaluated as JavaScript in their browser. I would think that it would be sufficient to prevent an other domain client from intercepting sensitive data for the server to require an authenticated token in the HTTP request, as can be provided by an XML HTTP Request but not a JSONP request. However, some crackers can attempt to get data from your service by using JSON data in a cross site script, much like JSONP. However, instead of using a callback, these crackers arrange to override the Array or Object constructors and thus can monitor and transmit snippets of JSON constructed by simply evaluating JavaScript object literal notation. This technique can be prevented by padding your XHR JSON service with an infinite loop.

while (1);
{"a": 10}

In this case, your client conspires with the server, and since it does get a chance to intercept and modify the text of the JSON, it strips the known number of characters for the while loop before evaluating it.

var text = xhr("/some.json");
var json = evalGlobal("(" + text.substring("while (1);".length) + ")");

What baffles me is that, unless you've secured your JSON with an authentication token, any server capable of making HTTP requests and parsing JSON could do the same thing. Malicious clients are not required to use web browsers. The only situation where this might occur would be if the cracker had compromised your client already, had your authentication token, and needed your browser to make a cross domain JSONP request on its behalf. That could not be the case since the server would have no reason to give an XHR JSON authentication token to a client that could only fetch data with JSONP. That point is even moot since, going back, the cracker has already compromised your client and might as well use XHR JSON with all the same rights as you on even the same origin.

That being said, I noticed that GMail used this technique, so I assume there's substance to it. Perhaps someone will clarify in the comments.

kriskowal: @wynk on cixar.com, "jolly boots of doom" is most frequently sought.

2008-10-04T19:34:45+00:00

kriskowal: @wynk on cixar.com, "jolly boots of doom" is most frequently sought.

Jonathan Haidt on the moral roots of liberals and conservatives | Video on TED.com

2008-10-04T01:05:46+00:00

A really neat piece on moral psychology, how it polarizes people along the red/blue axis, and how we can become purple.

Presidential Election 2008 FAQ

2008-10-03T23:45:56+00:00

Designing Django's Object-Relational-Model - The Python Saga - Part 6

2008-10-03T22:13:11+00:00

Django is a web application framework in the Python language. One of the advantages that Django has over other libraries is that it was written and designed by Python experts. That is to say, they knew about variadic arguments, properties, and metaclasses. Furthermore, they knew how to cleverly use these ideas to sweep a lot of complexity under the hood (or bonnet if you will; some of them appear to be British) so that common developers, or uncommonly good developers who want to think about other things most of the time, can gracefully suspend disbelief that anything complicated is going on when they design their database in pure Python. This article will illustrate how Django uses metaclasses and properties to present an abstraction layer where you can specify a database schema with Python classes. For simplicity, the "database" backend will be plain Python primitive objects—tables will be dictionaries of dictionaries.

In the end, we want to be able to write code that looks a whole lot like it's using Django's Object-Relational-Model:

class Cow(Model):
	id = PrimaryKey()
	name = ModelProperty()

cow = Cow(id = 0, name = 'Moolius')
cow.save()

cow = Cow.objects.get(0)

The easiest part (for the purpose of this exercise) is Django's concept of an "object manager". In Django, every model has an object manager that provides a query API and, depending on the backend, might cache instances of Model objects. Conveniently, a very narrow subset of the object manager API is almost exactly the same as a dictionary. Conceptually, the object manager boils down to a dictionary proxy for the database where you can use the get function to retrieve records from the database. For simplicity, our ObjectManager is just going to be a dictionary.

class ObjectManager(dict):
	pass

Beyond the scope of this article, the ObjectManager should be handy for grabbing lots of objects from the database at once. Django provides a very thorough and relatively well-optimized lazy query system with its object managers. The ObjectManager has get, and filter methods which, instead of simply accepting the primary key, accept keyword arguments that translate to predicate logic rules. In particular, the filter function is lazy, so you can chain filter commands to construct complex queries and Django only goes to the database once.

While it would be super-cool to model all of this with native Python, it actually is a lot of code, so that's a topic for maybe later. We'll just use the built in dict.get.

We'll also need all of the code from Part 5 since models will be another application of the ordered property pattern. This is how Django creates SQL tables with fields in the same order as the Python properties.

from ordered_class import \
	OrderedMetaclass,\
	OrderedClass,\
	OrderedProperty

We use the OrderedMetaclass to make a ModelMetaclass. The model metaclass will have all the same responsibilities as our StructMetaclass, including "dubbing" the properties so that they know their own names. The model metaclass will also create an ObjectManager for the class. This isn't the complete ModelMetaclass; we'll come back to it.

class ModelMetaclass(OrderedMetaclass):
	def __init__(self, name, bases, attys):
		super(ModelMetaclass, self).__init__(name, bases, attys)
		if '_abstract' not in attys:
			self.objects = ObjectManager()
			for name, property in self._ordered_properties:
				property.dub(name, self)

The next step is to create a ModelProperty base class. This class will be an OrderedProperty so it's sortable. It will also need to implement the dub method so it can figure out its name. Other than that, it'll be just like the StructProperty from the previous section: it will get and set its corresponding item in the given object.

class ModelProperty(OrderedProperty):
	def __get__(self, objekt, klass):
		return objekt[self.item_name]
	def __set__(self, objekt, value):
		objekt[self.item_name] = value
	def dub(self, name):
		self.item_name = name
		return self

There is a distinction in the refinement of ModelProperty from StructProperty: ModelProperty objects will eventually need to distinguish the value stored in the dictionary from the value returned when you access an attribute. In the primitive case, they're the same, but for ForeignKey objects, down the road, you'll store the primary key for the foreign model instead of the actual object. This is the same as the behavior in an underlying database backend.

class ModelProperty(OrderedProperty):
	def __get__(self, objekt, klass):
		return objekt[self.item_name]
	def __set__(self, objekt, value):
		objekt[self.item_name] = value
	def dub(self, name):
		self.attr_name = name
		self.item_name = name
		return self

Let's consider a PrimaryKey ModelProperty. The purpose of a PrimaryKey is to designate a property of a model that will be used as the index in its object manager dictionary. In Django, this can be an implicit id field at the beginning of the table. For simplicity in this exercise, we'll require every model to explicitly declare a PrimaryKey. The ModelMetaclass will identify which of its ordered properties is the primary key by observing its type. Other than their type, a primary key's behavior is the same as a normal ModelProperty, so it's a really easy declaration:

class PrimaryKey(ModelProperty):
	pass

Now we can go back to our ModelMetaclass and add the code we need for every class to know the name of its primary key. I create a list of PrimaryKey objects from my _ordered_properties and pop off the last one, leaving error checking as an exercise for a more rigorous implementation. There should be only one primary key.

class ModelMetaclass(OrderedMetaclass):
	def __init__(self, name, bases, attys):
		super(ModelMetaclass, self).__init__(name, bases, attys)
		if '_abstract' not in attys:
			self.objects = ObjectManager()
			for name, property in self._ordered_properties:
				property.dub(name)
			self._pk_name = [
				name
				for name, property in self._ordered_properties
				if isinstance(property, PrimaryKey)
			].pop()

Now all we need is a Model base class. The model base class will just be a dictionary with the model metaclass and a note that it's abstract: that is, it does not have properties so the metaclass better not treat it as a normal model.

class Model(OrderedClass, dict):
	__metaclass__ = ModelMetaclass
	_abstract = True

The model will also have a special pk attribute for accessing the primary key and a save method for committing a model to the ObjectManager.

class Model(OrderedClass, dict):
	__metaclass__ = ModelMetaclass
	_abstract = True

	def save(self):
		self.objects[self.pk] = self

	@property
	def pk(self):
		return getattr(self, self._pk_name)

Now we have all the pieces we need to begin using the API. Let's look at that cow model.

class Cow(Model):
	id = PrimaryKey()
	name = ModelProperty()

cow = Cow(id = 0, name = 'Moolius')
cow.save()

cow = Cow.objects.get(0)

All of this works now. You make a cow model; that invokes the model metaclass that sets up Cow._pk_name to be "id" and tacks on a Cow.objects object manager. Then we make a cow and put it in Cow.objects with the save method. This is analogous to committing it to the database backend. From that point, we can use the object manager to retrieve it again.

We can refine the Model base class to take advantage of the fact that it's not just a dictionary anymore: it's an ordered dictionary. We create a better __init__ method that will let us assign the attributes of our Cow either positionally or with keywords. That makes our cow more like a hybrid of a list and a dictionary. Also, since our model instances aren't merely dictionaries, we create a new __repr__ method that will note that cows are cows and moose are moooose. The new __repr__ method also takes the liberty to write the items in the order in which their properties were declared.

class Model(OrderedClass, dict):

	…

	def __init__(self, *values, **kws):
		super(Model, self).__init__()
		found = set()
		for (name, property), value in zip(
			self._ordered_properties,
			values,
		):
			setattr(self, name, value)
			found.add(name)
		for name, value in kws.items():
			if name in found:
				raise TypeError("Multiple values for argument %s." % repr(name))
			setattr(self, name, value)

	…

	def __repr__(self):
		return '%s %s>' % (
			self.__class__.__name__,
			" ".join(
				"%s:%s" % (
					property.item_name,
					repr(self[property.item_name])
				)
				for name, property in self._ordered_properties
			)
		)

Now we can make a cow model with positional and keyword arguments, and print it out nice and fancy-like:

>>> Cow(0, name = 'Moolius')
<Cow id:0 name:"Moolius">

The next step is to introduce ForeignKey model properties. These are properties that will refer, via a relation on a primary key, to an object in another model. So, the ForeignKey class will accept a Model for the foreign model. Its dub method will override the item_name (preserving the attr_name) provided by it's super-class's dub method. The new item_name with be the attr_name and the name of the primary key from the foreign table, delimited by an underbar. This will let the foreign key property hide the fact that it does not contain a direct reference to the foreign object; it just keeps the foreign object's primary key. However, if you access the foreign key property on a model instance, it will go off and diligently fetch the corresponding model instance. If you assign to the foreign key property, it'll tolerate either a primary key or an actual instance.

class ForeignKey(ModelProperty):
	def __init__(self, model, *args, **kws):
		super(ForeignKey, self).__init__(*args, **kws)
		self.foreign_model = model
	def __get__(self, objekt, klass):
		return self.foreign_model.objects.get(objekt[self.item_name])
	def __set__(self, objekt, value):
		if isinstance(value, self.foreign_model):
			objekt[self.item_name] = value.pk
		else:
			objekt[self.item_name] = value
	def dub(self, name):
		super(ForeignKey, self).dub(name)
		self.item_name = '%s_%s' % (
			name,
			self.foreign_model._pk_name,
		)

Now we can write code with more than one model using relationships. Let's give our cow a bell.

class Bell(Model):
	id = PrimaryKey()

class Cow(Model):
	id = PrimaryKey()
	name = ModelProperty()
	bell = ForeignKey(Bell)

bell = Bell(0)
bell.save()
cow = Cow(0, 'Moolius', bell)
cow.save()

Note that you must save the bell so that when you construct the cow, it can fetch the bell from Bell.objects.

There's more to Django's ORM, of course. This article doesn't cover parsing and validation, which are both assisted by the ORM. Nor does it cover queries, query sets, the related_name for ForeignKey properties on foreign models, Django's ability to use strings for forward references to models that have not yet been declared, or many of the other really neat features.

What this article does cover though, is that you can create a powerful abstraction of a proxied database with pure-Python in less than 200 lines of code. This means that you could create a light-weight proxy over HTTP to a Django database that exposes itself with a REST API. You could also create an abstraction layer that would allow you to pump Django ORM duck-types back into Django to use pure Python objects in addition to or in stead of a database backend.

But, if this article does nothing else, I hope it communicates that Django is cool. I have read a whole lot of code from every dark corner of the web and I have liked very little of it; people I've worked with will testify that I've regularly "hated on" every library or framework I've ever seen. I've never met Simon Willson and the growing developer community around Django. However, I've read their code and now I can tell you, over the course of several articles, that they're really smart and you should use their code.

The new user switching applet in Ubuntu 8.10 feels a little bit awkward, but it packs in lots of functionality. I think I like it.

2008-10-03T19:29:20+00:00

The new user switching applet in Ubuntu 8.10 feels a little bit awkward, but it packs in lots of functionality. I think I like it.

By segphault 2 days, 9 hours ago.

Everybody seems to think that the Ubuntu Dust theme is awesome, but I'm not really seeing the appeal. I <3 GNOME-Colors: http://is.gd/3u99

2008-10-03T17:27:39+00:00

Everybody seems to think that the Ubuntu Dust theme is awesome, but I'm not really seeing the appeal. I <3 GNOME-Colors: http://is.gd/3u99

By segphault 2 days, 11 hours ago.

It looks like AOL's instant messaging servers are messed up right now. If you need to chat, I'm still on GTalk and IRC.

2008-10-03T07:17:08+00:00

It looks like AOL's instant messaging servers are messed up right now. If you need to chat, I'm still on GTalk and IRC.

By segphault 2 days, 22 hours ago.

Norwegian standards body implodes over OOXML controversy

2008-10-03T07:00:00+00:00

Norway's national standards body lost 13 key members last week. The technical committee members resigned in protest of procedural irregularities in the OOXML approval process.

Python language slithers into the future with 2.6 release

2008-10-03T07:00:00+00:00

Developers of the open source Python programming language have announced the official release of version 2.6. This version makes a nice stepping stone, helping third-party developers migrate their code to the upcoming 3.0 release.

I'm not really enthusiastic about Fedora's new Echo icon theme. I wish that distros would all just conform with the Tango palette and style.

2008-10-02T21:23:34+00:00

I'm not really enthusiastic about Fedora's new Echo icon theme. I wish that distros would all just conform with the Tango palette and style.

By segphault 3 days, 7 hours ago.

There are some cool things in Fedora 10. I like Plymouth, the new graphical boot system. Also, glitch-free PulseAudio FTW!

2008-10-02T21:22:26+00:00

There are some cool things in Fedora 10. I like Plymouth, the new graphical boot system. Also, glitch-free PulseAudio FTW!

By segphault 3 days, 7 hours ago.

This overview of the iPhone SDK is one of the best technical articles on Ars in recent memory: http://is.gd/3q6k

2008-10-02T19:44:29+00:00

This overview of the iPhone SDK is one of the best technical articles on Ars in recent memory: http://is.gd/3q6k

By segphault 3 days, 9 hours ago.

Ugh. This bug really sucks: http://is.gd/3rsl Now I have to install the craptastic VMWare server so I can test Fedora 10. :-(

2008-10-02T19:26:27+00:00

Ugh. This bug really sucks: http://is.gd/3rsl Now I have to install the craptastic VMWare server so I can test Fedora 10. :-(

By segphault 3 days, 9 hours ago.

I can't seem to get the Fedora 10 beta to boot in VirtualBox. :-(

2008-10-02T19:18:18+00:00

I can't seem to get the Fedora 10 beta to boot in VirtualBox. :-(

By segphault 3 days, 9 hours ago.

Ordered Properties - The Python Saga - Part 5

2008-10-02T03:38:42+00:00

In C and SQL, structs and records have properties that appear in a particular order. In Python, objects use hash-tables to store their attributes, so the order is not deterministic nor relevant. That's no consolation if you're trying to model C structs or SQL tables in Python though. Reading though the Django code, I discovered that those wily coders had synthesized a technique that combines the virtues of properties and metaclasses that we can generally model the order in which fields of a struct or SQL table appear.

The trick is to provide an API that allows your users to write code like:

from time import gmtime
from cstruct import Struct, IntegerProperty

class EpochTime(Struct):
	seconds = IntegerProperty()
	microseconds = IntegerProperty()

epoch_time = EpochTime()
timestruct = gmtime(epoch_time.seconds)

In this case, the Struct type cares about the size, order, and disposition of the C structure it models so that it can unpack a byte buffer presumably retrieved from a C-type library.

There are two components to the general solution. First, there's an OrderedProperty base type. Ordered properties track the order in which they were initialized. For this purpose, ordered properties use a global counter. The absolute value of a property's creation counter is irrelevant. The only requirement is that they monotonically increase as each property is declared, and that classes declared concurrently do not interfere.

from itertools import count
next_counter = count().next

I learned about itertools from Brett Cannon who was preparing his thesis defense when I was at Cal Poly. In another piece of code, which I would cite if I could recall, I learned the trick of using the itertools.count function to atomize a global counter. It takes advantage of Python's dubious global interpreter lock.

Then the OrderedProperty base type just stores a creation counter upon initialization. Keep in mind that all derived types must trickle their __init__ call—your base types are not always what they seem and there are use cases you can not foresee where you might be required to receive and pass your initialization arguments to an unknown super-type. That's a side-effect of working in a language with mix-ins, and it's a "good thing".

class OrderedProperty(object):
	def __init__(self, *args, **kws):
		self._creation_counter = next_counter()
		super(OrderedProperty, self).__init__(*args, **kws)

The next part of the puzzle is inspecting your property order. We use a metaclass to note when new types are created and we give them an _ordered_properties attribute with the ordered-property items in their attributes. Keep in mind that base types might have properties too. __mro__ is an attribute of all classes that is a linearized list of a classes inheritance hierarchy. It's effectively the result of a dynamic topological sort algorithm for the closure of your base types. We traverse it backwards so that if you create a dictionary via dict(_ordered_properties), name collisions are resolved chronologically.

class OrderedMetaclass(type):
	def __init__(self, name, bases, attys):
		super(OrderedMetaclass, self).__init__(name, bases, attys)
		self._ordered_properties = sorted(
			(
				(name, value)
				for base in reversed(self.__mro__)
				for name, value in base.__dict__.items()
				if isinstance(value, OrderedProperty)
			),
			key = lambda (name, property): property._creation_counter,
		)

Then, we create our OrderedClass that we can inherit to get free _ordered_properties attributes.

class OrderedClass(object):
	__metaclass__ = OrderedMetaclass

Let's see it in action:

class Foo(OrderedClass):
	bar = OrderedProperty()
	baz = OrderedProperty()
Foo._ordered_properties == [
	('bar', <Ordered Property instance somewhere>),
	('baz', <Ordered Property instance somewhere>),
]

With a small modification, we can track ordered inner classes too.

class OrderedMetaclass(type):
	def __init__(self, name, bases, attys):
		super(OrderedMetaclass, self).__init__(name, bases, attys)
		self._creation_counter = next_counter()
		self._ordered_properties = sorted(
			(
				(name, value)
				for base in reversed(self.__mro__)
				for name, value in base.__dict__.items()
				if isinstance(value, OrderedProperty)
				or isinstance(value, OrderedMetaclass)
			),
			key = lambda (name, property): property._creation_counter,
		)

So, we put all of those order classes in our generic ordered properties module. Now we can delve into our particular Struct example.

First, we need field types. We will need a base type and derivative types for all of the field types we want to model from C, like IntegerField. Bear in mind that OrderedProperty is just a mix-in; it doesn't actually define __get__ and its ilk because those are all specific to the derivative implementation. All struct fields are going to store their actual values in a special _value dictionary on their corresponding Struct instance.

class StructProperty(OrderedProperty):
	def __get__(self, objekt, klass):
		return objekt._values[self.name]
	def __set__(self, objekt, value):
		objekt._values[self.name] = value
	def dub(self, name):
		self.name = name
		return self

You'll notice that instances of StructProperty need to know their name, the key in their corresponding instance's _values dictionary. The struct property instances don't get this from their initializer. Instead, each property will get "dubbed", given a name, when the StructMetaclass visits its _ordered_properties.

Let's just look at an integer.

class IntegerProperty(StructProperty):
	_format = 'i'

The only thing special about an integer is that it's format specifier for the pack routine is "i".

We're going to want to use a metaclass again, this time inheriting from our OrderedMetaclass. The purpose of this metaclass will be to analyze the ordered properties, construct an aggregate format specifier for pack and unpack methods, and to dub each of its properties with their name.

class StructMetaclass(OrderedMetaclass):
	def __init__(self, name, bases, attys):
		super(StructMetaclass, self).__init__(name, bases, attys)
		for name, property in self._ordered_properties:
			property.dub(name)
		self._format = "".join(
			property._format
			for name, property in self._ordered_properties
		)

All that remains is to create our API base class, Struct. This class initializes its values and declares its metaclass.

from struct import unpack

class Struct(OrderedClass):
	__metaclass__ = StructMetaclass

	def __init__(self, *args, **kws):
		super(Struct, self).__init__(*args, **kws)
		self._values = {}

	def unpack(self, value, prefix = None):
		if prefix is None: prefix = ""
		for (name, property), value in zip(
			self._ordered_properties,
			unpack(prefix + self._format, value)
		):
			self._values[name] = value

So, now our epoch time example is possible using Struct and IntegerProperty, completely oblivious to the machinations behind the scenes.

from time import gmtime
from datetime import datetime

class EpochTime(Struct):
	seconds = IntegerProperty()
	microseconds = IntegerProperty()
epoch_time = EpochTime()

timestruct = gmtime(epoch_time.seconds)
datestruct = (timestruct[:6] + (epoch_time.microseconds,))
print datetime(*datestruct)

Duct tape and paperclips. I'd be totally lost without them.

2008-10-02T02:47:37+00:00

Duct tape and paperclips. I'd be totally lost without them.

By segphault 4 days, 2 hours ago.

Wow. My monitor just came back to life. The source button was stuck. I got it loose with a paperclip.

2008-10-02T02:40:45+00:00

Wow. My monitor just came back to life. The source button was stuck. I got it loose with a paperclip.

By segphault 4 days, 2 hours ago.

One of my monitors died yesterday. I'm starting to feel a little bit of claustrophobia. I think I need to get a new one.

2008-10-02T00:59:41+00:00

One of my monitors died yesterday. I'm starting to feel a little bit of claustrophobia. I think I need to get a new one.

By segphault 4 days, 4 hours ago.

The GNU Netcat -- Official homepage

2008-10-01T23:21:54+00:00

Netcat has a build flag called -DGAPING_SECURITY_FLAG that permits you to run "nc" with the "-e" arg which permits the first person to connect to your server to run the specified command, albeit a shell. Anybody else know some bizarrely communicative flags? I figure that GCC at least should, in addition to "-ansi" and "-pedantic", accept "-yesimeanit" for particularly strong compliance, like only features that are available in both VisualC++ and G++, that admittedly would make the language completely useless.

GIMP 2.6 is the first release to include GEGL! Congratulations to the GIMP developers! :-)

2008-10-01T20:08:55+00:00

GIMP 2.6 is the first release to include GEGL! Congratulations to the GIMP developers! :-)

By segphault 4 days, 9 hours ago.

GIMP 2.6 released, one step closer to taking on Photoshop

2008-10-01T07:00:00+00:00

GIMP 2.6 has been officially released. The new version is the first to include the Generic Graphics Library, a powerful graph-based image editing framework that enables the GIMP to support higher color depth. This takes the open source graphics program one step closer to being ready for the professional graphics market.

"Orphan Works" copyright reform fails in wake of bailout bid

2008-10-01T07:00:00+00:00

The Orphan Works Act of 2008 was passed by the Senate after "diligent search" was clarified with a handful of specific guidelines. With the controversy over the $700 billion economic bailout package, House failed to act upon it, making its prospects for passage dim this year.

Metaclasses - The Python Saga - Part 4

2008-10-01T05:45:45+00:00

The original type function, whose behavior is preserved in modern Python 2.5, accepts an object and returns the class, albeit the type, that would emit it. It's like the typeof operator in JavaScript that returns the String name of the primitive type of an object, or the C++ function that returns a pointer to an object's virtual function table. They're all sufficient for comparing apples to oranges, but all of them are also insufficient for the more interesting comparison of apples to the idea of a Fiji apple: the question, "Does your type inherit from this?", that can be accomplished with Python's isinstance, JavaScript's instanceof, or C++'s infernal dynamic_cast. So, type's single argument behavior is effectively retired.

At some transcendental moment, somebody deeply involved in the Python project must have been thinking, "Well, if functions and classes return objects, what returns a class? Could a class, like a property, be syntactic sugar for some deeply metaphysical latent behavior in pure Python?". I figure this is how the type function grew its new wings.

So consider a class declaration:

class Foo(object):
	bar = 10
	def __init__(self, bar = None):
		if bar is not None:
			self.bar = bar

This is what is actually happening behind the curtains:

name = 'Foo'
bases = (object,)
def __init__(self, bar = None):
	if bar is not None:
		self.bar = bar
attys = {'bar': bar, '__init__': __init__}
Foo = type(name, bases, attys)

That is to say, there is no magic in the syntax. Ultimately all of the magic happens when you call type. By "magic" I mean functionality that cannot be replicated in pure Python without the interpreter's intervention.

The type function returns a type: a function that returns new instances. It's also called a "metaclass". type just happens to also be the implied metaclass of object. That is to say, you can create your own metaclasses.

The big question about metaclasses is, "Why on earth would you want to define a metaclass?". David Mertz from IBM wrote that you would simply know when you needed them. Since I read that article, I've wracked my mind for a reason to use metaclasses to no avail. At some point, I was reading Django's ORM code and it occurred to me that the reason you would want to define a metaclass is to provide a class in your API that, when subclassed by unsuspecting users, would invoke certain preparations without their knowledge or consent. Here's how:

Define a metaclass. The best way to define a metaclass is to inherit type and override its __init__ method.

class FooType(type):
	def __init__(self, name, bases, attys):
		super(FooType, self).__init__(name, bases, attys)
		print '%s was declared!' % name

Define a base class for your API. The trick here is that you can override its metaclass. Let's look at this one in an interactive interpreter:

>>> class Foo(object):
...     __metaclass__ = FooType
...
Foo was declared!
>>>

Whoa! You didn't call anything. Not true. Here's what actually happened:

name = 'Foo'
bases = (object,)
attys = {}
attys['__metaclass__'] = FooType
Foo = attys.get('__metaclass__', type)(name, bases, attys)

Python checks your attributes for a metaclass before defaulting to type.

That means that your FooType.__init__ got called. Hot damn. I wonder what happens if you create a subclass.

>>> class Bar(Foo):
...     pass
...
Bar was declared!
>>>

Whoa! I totally inherited a metaclass.

So, the reason for writing a metaclass is that metaclasses give you an opportunity to get and manipulate your derived class objects before anyone instantiates them. You get to do this once, right after the class dictionary is fully populated. You can take this opportunity to monitor class declarations, to prepare additional attributes, or to interpolate additional base types.

Keep in mind that metaclasses are jealous. If you create a metaclass for a type that inherits from base classes in someone else's API, your metaclass must inherit from their metaclass. I suspect that it's best not to assume that your base types use a particular metaclass. Thankfully, you can use an expression for your base type.

class FooType(getattr(Bar, '__metaclass__', type)):
	pass
class Foo(Bar):
	__metaclass__ = FooType

This takes advantage of the Python idiom of accessor methods like dict.get and getattr that accept a default-if-none-exists argument. Unfortunately, Python's object doesn't explicitly state that type is its metaclass. Otherwise, you could safely say:

class FooType(Bar.__metaclass__):
	pass

Such things are to be looked for in Python 3. I find that the Python developers have either, after considerable review and debate, already accepted or rejected most of my ideas before I even consider them, so I'm not even going to check for a PEP on this one.

HELLO, I’M FAMOUS STARRING JOHN HODGMAN: GQ Features on men.style.com

2008-10-01T05:04:31+00:00

This article made me LLOL a couple times (*literally* laughed out loud. I was alone, so I know it was sincere). It also made me want to cry a little, and made me wish that I could be John Hodgman's long-time-friend who could always tell him when he made mistakes or looked stupid. But then I remembered that he had Johnathan Coulton for that, so I didn't feel bad anymore.

I appreciate the feedback. Thanks for the responses!

2008-10-01T01:24:20+00:00

I appreciate the feedback. Thanks for the responses!

By segphault 5 days, 3 hours ago.

My technical Linux coverage at Ars has mostly been for app developers who want platform insight, particularly in mobile. Is that good?

2008-10-01T01:10:28+00:00

My technical Linux coverage at Ars has mostly been for app developers who want platform insight, particularly in mobile. Is that good?

By segphault 5 days, 4 hours ago.

A quick Twitter poll: how many Ars readers out there want to see more kernel coverage rather than higher-level coverage for Linux stuff?

2008-10-01T01:08:56+00:00

A quick Twitter poll: how many Ars readers out there want to see more kernel coverage rather than higher-level coverage for Linux stuff?

By segphault 5 days, 4 hours ago.

Thanks everyone, for the positive comments about my cloud computing article! :-)

2008-09-30T18:39:03+00:00

Thanks everyone, for the positive comments about my cloud computing article! :-)

By segphault 5 days, 10 hours ago.

Check out this awesome Qt demo with custom SVG/CSS widgets designed for touchscreen interaction: http://is.gd/3l75

2008-09-30T18:25:30+00:00

Check out this awesome Qt demo with custom SVG/CSS widgets designed for touchscreen interaction: http://is.gd/3l75

By segphault 5 days, 10 hours ago.

Commentary: Bankruptcy, not bailout, is the right answer - CNN.com

2008-09-30T10:36:18+00:00

Jeff Miron on the bailout plan.

Nokia launches Linux-based Qt Extended mobile platform

2008-09-30T07:00:00+00:00

Nokia has launched Qt Extended 4.4, a Linux-based mobile platform that leverages the Qt toolkit. Ars takes a look at some of the components of the platform and what it offers third-party application developers.

Why Stallman is wrong when he calls cloud computing stupid

2008-09-30T07:00:00+00:00

Richard Stallman, one of the fathers of the software freedom movement, has declared that cloud computing is stupid in calling for users to reject web applications. Ars tells Stallman why we think he is wrong and takes a close look at emerging initiatives that aim to bring software freedom to the web.

Properties - The Python Saga - Part 3

2008-09-30T06:34:40+00:00

Properties come out of a tired programming language genesis. In the beginning, there were structs. The trouble with structs was that an opaque data structure could not programmatically monitor or intercept access and mutation of its member data.

So that's not a big deal; we could solve the problem with classes. The best practice to avoid programming yourself into a corner was to never expose a datum; you would write accessor and mutator functions, whether you needed them at the moment or not. Thus, as your design grew, you could eventually do nice things like validation, observation, or proxying. The trouble with this approach was that you had to write six times as much code on the off chance you'd need to extend it some day. But it was worth it.

The idea of managed properties came along eventually in various languages (Python, C#, some implementations of JavaScript, and recent versions of [C]). The notion is that you would initially write all of your classes like structs with member data camped in public view. You would encourage your API consumers to interact with those members directly. Then, as need arose, you would subvert the member variables with property objects. These objects would intercept accesses and mutations with functions that you could write at any time of your design process.

Lets observe this design shift in Python. Here's a class with unmanaged data:

class Foo(object):
	def __init__(self);
		self.bar = 10

Here's some other fellow's code that uses your class:

foo = Foo()
foo.bar = 20
print foo.bar
del foo.bar

So there you have it. Just to keep on the same page, the idea at this point is to add a feature to Python that permits both of those code samples to work and, in-fact, be perfectly cromulent. However, we also want to eventually add features to Foo such that its bar attribute can be managed, validated, proxied, secured, or outright lied about. Enter property. property is a function that accepts an accessor function and optional mutator and deleter functions. The property must be a class attribute to work. Here's how you would use a property:

class Foo(object):
	def __init__(self):
		self.bar = 10
	def get_bar(self, objekt, klass):
		return self.baz / 2
	def set_bar(self, objekt, value):
		self.baz = value * 2
	def del_bar(self, objekt):
		del self.baz
	bar = property(get_bar, set_bar, del_bar)

Now we have a Foo class that transparently maintains the invariant that "bar" will always be half of "baz".

Sometimes you don't need to have a setter for a property, and you almost never need a deleter. For the common case, you can use the property function as a decorator.

class Foo(object):
	def __init__(self):
		self.baz = 20
	@property
	def bar(self):
		return self.baz / 2

Creating the `property` function.

So, it's easy to assume that the property function does all the magic behind the scenes, setting up traps in your class's accessor and mutator paths. There's actually another layer of code that can be done entirely in Python. That is, we can implement the property function in pure Python. The trick is that the property function is actually a type or factory method (who cares which) that returns a Python duck-type: a property object. A property object is any object that implements __get__, __set__, or __del__. These are special magic Python functions that intercept access, mutation, and deletion on members. All you have to do is install an object on a class with one of methods defined, with the name of the member you want to manage. The property function just handles the common cases. Let's redefine the property function in Python, as the Property class.

class Property(object):
	def __init__(self, fget):
		self.fget = fget
	def __get__(self, objekt, klass):
		return self.fget(objekt)

This defines enough of the Property object to decorate an accessor function.

class Foo(object):
	def __init__(self):
		self.baz = 20
	@Property
	def bar(self):
		return self.baz / 2

Here's a full implementation of Property. You will note that, in order to exactly emulate the property object, the __init__ method has the same argument names as the internal property so that code that uses keyword arguments will function in perfect ambivalence.

class Property(object):
	def __init__(
		self,
		fget,
		fset = None,
		fdel = None,
		doc = None,
	):
		self.fget = fget
		self.fset = fset
		self.fdel = fdel
		self.__doc__ = doc
	def __get__(self, objekt, klass):
		return self.fget(objekt)
	def __set__(self, objekt, value):
		self.fset(objekt, value)
	def __del__(self, objekt):
		self.fdel(objekt)

YouTube - Peter Schiff - (Former Ron Paul Economic Advisor) Versus Art Laffer (Former Ronald McReagan Economic Advisor) - August 28, 2006 - Peter Schiff [Pimp]

2008-09-30T03:53:45+00:00

Want to see something scary? If I were a betting man…

Ask a Wizard

2008-09-29T22:53:18+00:00

My blog's new home. The Python Saga continues for the rest of the week.

kriskowal: I've moved my blog to http://askawizard.blogspot.com/ so you can comment and I can track. Still uses SWL though :-)

2008-09-29T22:48:37+00:00

kriskowal: I've moved my blog to http://askawizard.blogspot.com/ so you can comment and I can track. Still uses SWL though :-)

Migrating to Blogger

2008-09-29T19:01:26+00:00

Heij, folks. I'm migrating my blog from my homegrown PyBlosxom solution to Blogger. I will continue cross posting here, but if you would like to make comments, please visit my new location, http://askawizard.blogspot.com/.

kriskowal: Want: umbrella scabard.

2008-09-29T18:33:13+00:00

kriskowal: Want: umbrella scabard.

Incomprehensible Vacuum: Penny-wise, Pound-foolish

2008-09-29T08:48:30+00:00

Friend from college, Aaron Stearatte, proposes that the federal budget is like an optimization problem. Frankly, I think that 3m$ a fine price for studying bear DNA.

Classmate PC gets a boost with million-unit Venezuelan order

2008-09-29T07:00:00+00:00

The government of Venezuela has ordered one million low-cost Classmate PC laptops to be shipped to students with Linux preinstalled.

Mandriva developer talks about Linux boot optimization

2008-09-29T07:00:00+00:00

A developer explains some of the optimization techniques that are used to improve boot performance in the upcoming version of the Mandriva Linux distribution.

Decorators - The Python Saga - Part 2

2008-09-29T04:42:06+00:00

Python introduced a short-hand for the adapter pattern on functions. You can "decorate" a function with another function. This is a neat tool you can use to factor out some common code from a bunch of functions. You can fiddle with the arguments, return values, or intercept exceptions thrown by any function you decorate.

The canonical example is a memoize decorator. The idea is to generalize the notion of memoization so you can simply subscribe to it in any function you want to memoize.

def factorial(n):
	if n == 1: return 1
	return n * factorial(n - 1)
factorial = memoize(factorial)

You accomplish this by writing the memoize decorator. A decorator is a function that accepts a function and returns another. Python virtuously provides a shorthand for taking the function, decorating it, and assigning it to a variable with the same name.

@memoize
def factorial(n):
	if n == 1: return 1
	return n * factorial(n - 1)

In the imagined normal case of decorators, the returned function accepts the same arguments and returns the same kinds of values as the accepted function. However, a decorator does have the liberty of extending or restricting that interface, like accepting additional arguments or raising an exception if the arguments are of the wrong type. It might also perform some common computation on the original arguments and pass the result to the original function as an additional argument. In any case, you can use some closures to create a decorator:

def memoize(function):
	cache = {}
	def decorated(*args):
		if args not in cache:
			cache[args] = function(*args)
		return cache[args]
	return decorated

Of course, that's too simple. A lot of things you put after the "@" symbol are just functions that return decorators so that they can be configured with arguments. For example, you probably want to make a memoize decorator that lets you specify your own cache object. So, you need another layer of deference.

def memoize(cache = None):
	if cache is None: cache = {}
	def decorator(function):
		def decorated(*args):
			if args not in cache:
				cache[args] = function(*args)
			return cache[args]
		return decorated
	return decorator

@memoize({})
def factorial(n):
	if n == 1: return 1
	return n * factorial(n - 1)

Since, in Python, functions, objects, and types are indistinguishable to the casual observer, you can do the exact same thing with a class, although I shudder to think that you might want to forgo the simplicity and elegance of closures. After the transform, the previous code might look like this:

class memoize(object):
	def __init__(self, cache = None):
		self.cache = cache
	def __call__(self, function):
		return Memoized(function, self.cache)

class Memoized(object):
	def __init__(self, function, cache = None):
		if cache is None: cache = {}
		self.function = function
		self.cache = cache
	def __call__(self, *args):
		if args not in self.cache:
			self.cache[args] = self.function(*args)
		return self.cache[args]

@memoize()
def factorial(n):
	if n == 1: return 1
	return n * factorial(n - 1)

So now you can use a Least Recently Used Cache, assuming it is a dictionary-like-object (a duck-dict, if you will):

from lru_cache import LruCache
@memoize(LruCache(max_size = 100, cull = .25))
def factorial(n):
	if n == 1: return 1
	return n * factorial(n - 1)

Download decorators.zip.

kriskowal: …you know, as opposed to, "promote international innovation in clean energy."

2008-09-29T02:23:35+00:00

kriskowal: …you know, as opposed to, "promote international innovation in clean energy."

kriskowal: When McCain said, "eliminate our dependence on foreign oil", I heard, "give my buddie$ permi$$ion to rape America'$ natural re$ource$."

2008-09-29T02:22:56+00:00

kriskowal: When McCain said, "eliminate our dependence on foreign oil", I heard, "give my buddie$ permi$$ion to rape America'$ natural re$ource$."

kriskowal: There was a great fireworks show over CalTech last night. Occasion?

2008-09-28T18:08:36+00:00

kriskowal: There was a great fireworks show over CalTech last night. Occasion?

The Sourcerer - Variadic Positional and Keyword Arguments - The Python Saga - Part 1

2008-09-28T08:46:45+00:00

Shameless plug: I'm preparing a series of blog articles on Python topics that I think are important. Stay tuned for more as the week progresses.

kriskowal: I'm preparing a series of blogs on some of Python's most important features. Stay tuned: http://cixar.com/~kris.kowal/blog

2008-09-28T08:17:21+00:00

kriskowal: I'm preparing a series of blogs on some of Python's most important features. Stay tuned: http://cixar.com/~kris.kowal/blog

Variadic Positional and Keyword Arguments - The Python Saga - Part 1

2008-09-28T07:09:21+00:00

Python supports "variadic" arguments. Variadic arguments are the man behind the curtain for C's printf function. The idea is that a function can accept a variable number of positional arguments, the values to put in your format string. In C this is accomplished with an ellipsis, ..., and some VA macro-linked-list-stuff that I always have to look up. Python goes a couple steps further with variadic arguments and the results are stunning, orthogonal, and actually useful almost every day. With Python, you get both "positional" arguments, like C, and keyword arguments: those arguments that conceptually map, in any order, to the names of the arguments in your function's declaration. The magic symbols are "*" and "**" for positional and keyword arguments respectively. With one "*", you can declare a function that accepts any number of arguments as the declared list object:

def foo(*args):
	return args
assert foo(1, 2, 3) == [1, 2, 3]

You can also pass an array of positional arguments to a function with very similar syntax:

def foo(a, b, c):
	return [a, b, c]
assert foo(*[1, 2, 3]) == [1, 2, 3]

And you can do the same thing with keyword arguments except you use dictionaries:

def foo(**kwargs):
	return kwargs
assert foo(a = 10, b = 20, c = 30) == {'a': 10, 'b': 20, 'c': 30}

Likewise, you can pass keyword arguments:

def foo(a, b, c):
	return [a, b, c]
assert foo(**{'a': 10, 'b': 20, 'c': 30}) == [10, 20, 30]

You can use them in combination, along with default arguments to provide beautiful, orthogonal, reusable abstractions:

def foo(a, b = None, c = None, d = None):
	return [a, b, c, d]
assert foo(*[1, 2], **{'c': 3}) == [1, 2, 3, None]
def bar(a, b, c, *args, **kws):
	return [a, b, c], args, kws
assert bar(1, 2, 3, 4, 5, f = 6) == ([1, 2, 3], [4], {'f': 5})

Planet Cixar

lhc11.jpg 990×640 pixels

kriskowal: This tweet was made by a Python program.

First look: latest Fedora and Ubuntu betas really shine

War of the Worlds 2.0 Update

kriskowal: @segphault also bought Anathem. It's now high on my pile of books to read.

kriskowal: Readers Digest version: follow @wotw2 (*gasp* I said "it") and tweet about your "experiences" as the alien invasion unfolds.

kriskowal: War of the Worlds 2.0: http://tinyurl.com/4zm922 -- Halloween Reenactment on Twitter. Spread the word.

Ask a Wizard: War of the Worlds 2.0

War of the Worlds 2.0

I started reading Anathem today and I'm really enjoying it so far.

Multi-Dimensional Analog Literals in C++

JSON Basics

XHR JSON

JSONP

Same Origin XHR JSON

kriskowal: @wynk on cixar.com, "jolly boots of doom" is most frequently sought.

Jonathan Haidt on the moral roots of liberals and conservatives | Video on TED.com

Presidential Election 2008 FAQ

Designing Django's Object-Relational-Model - The Python Saga - Part 6

The new user switching applet in Ubuntu 8.10 feels a little bit awkward, but it packs in lots of functionality. I think I like it.

Everybody seems to think that the Ubuntu Dust theme is awesome, but I'm not really seeing the appeal. I <3 GNOME-Colors: http://is.gd/3u99

It looks like AOL's instant messaging servers are messed up right now. If you need to chat, I'm still on GTalk and IRC.

Norwegian standards body implodes over OOXML controversy

Python language slithers into the future with 2.6 release

I'm not really enthusiastic about Fedora's new Echo icon theme. I wish that distros would all just conform with the Tango palette and style.

There are some cool things in Fedora 10. I like Plymouth, the new graphical boot system. Also, glitch-free PulseAudio FTW!

This overview of the iPhone SDK is one of the best technical articles on Ars in recent memory: http://is.gd/3q6k

Ugh. This bug really sucks: http://is.gd/3rsl Now I have to install the craptastic VMWare server so I can test Fedora 10. :-(

I can't seem to get the Fedora 10 beta to boot in VirtualBox. :-(

Ordered Properties - The Python Saga - Part 5

Duct tape and paperclips. I'd be totally lost without them.

Wow. My monitor just came back to life. The source button was stuck. I got it loose with a paperclip.

One of my monitors died yesterday. I'm starting to feel a little bit of claustrophobia. I think I need to get a new one.

The GNU Netcat -- Official homepage

GIMP 2.6 is the first release to include GEGL! Congratulations to the GIMP developers! :-)

GIMP 2.6 released, one step closer to taking on Photoshop

"Orphan Works" copyright reform fails in wake of bailout bid

Metaclasses - The Python Saga - Part 4

HELLO, I’M FAMOUS STARRING JOHN HODGMAN: GQ Features on men.style.com

I appreciate the feedback. Thanks for the responses!

My technical Linux coverage at Ars has mostly been for app developers who want platform insight, particularly in mobile. Is that good?

A quick Twitter poll: how many Ars readers out there want to see more kernel coverage rather than higher-level coverage for Linux stuff?

Thanks everyone, for the positive comments about my cloud computing article! :-)

Check out this awesome Qt demo with custom SVG/CSS widgets designed for touchscreen interaction: http://is.gd/3l75

Commentary: Bankruptcy, not bailout, is the right answer - CNN.com

Nokia launches Linux-based Qt Extended mobile platform

Why Stallman is wrong when he calls cloud computing stupid

Properties - The Python Saga - Part 3

Creating the property function.

YouTube - Peter Schiff - (Former Ron Paul Economic Advisor) Versus Art Laffer (Former Ronald McReagan Economic Advisor) - August 28, 2006 - Peter Schiff [Pimp]

Ask a Wizard

kriskowal: I've moved my blog to http://askawizard.blogspot.com/ so you can comment and I can track. Still uses SWL though :-)

Migrating to Blogger

kriskowal: Want: umbrella scabard.

Incomprehensible Vacuum: Penny-wise, Pound-foolish

Classmate PC gets a boost with million-unit Venezuelan order

Mandriva developer talks about Linux boot optimization

Decorators - The Python Saga - Part 2

kriskowal: …you know, as opposed to, "promote international innovation in clean energy."

kriskowal: When McCain said, "eliminate our dependence on foreign oil", I heard, "give my buddie$ permi$$ion to rape America'$ natural re$ource$."

kriskowal: There was a great fireworks show over CalTech last night. Occasion?

The Sourcerer - Variadic Positional and Keyword Arguments - The Python Saga - Part 1

kriskowal: I'm preparing a series of blogs on some of Python's most important features. Stay tuned: http://cixar.com/~kris.kowal/blog

Variadic Positional and Keyword Arguments - The Python Saga - Part 1

kriskowal: Readers Digest version: follow @wotw2 (gasp I said "it") and tweet about your "experiences" as the alien invasion unfolds.

Creating the `property` function.