"…mais ce serait peut-être l'une des plus grandes opportunités manquées de notre époque si le logiciel libre ne libérait rien d'autre que du code…"

Archive for avril 2008

Une grande nouvelle pour le monde python et django: « Google app engine »

Posted by patrick sur avril 8, 2008

http://www.techcrunch.com/2008/04/07/google-jumps-head-first-into-web-services-with-google-app-engine/ (« Google isn’t just talking about hosting applications in the cloud any more. Tonight at 9pm PT (7 april 2008) they’re launching Google App Engine (Update: The site is live), an ambitious new project that offers a full-stack, hosted, automatically scalable web application platform. It consists of Python application servers, BigTable database access (anticipated here and here) and GFS data store services. At first blush this is a full on competitor to the suite of web services offered by Amazon, including S3 (storage), EC2 (virtual servers) and SimpleDB (database)« )

A voir


http://code.google.com/appengine/articles/django.html (« Google App Engine and Django both have the ability to use the WSGI standard to run applications. As a result, it is possible to use nearly the entire Django stack on Google App Engine, including middleware. As a developer, the only necessary adjustment is modifying your Django data models to make use of the Google App Engine Datastore API to interface with the fast, scalable Google App Engine datastore. Since both Django and Google App Engine have a similar concept of models, as a Django developer, you can quickly adjust your application to use our datastore. »)


Posted in google, logiciel libre, python, Web Frameworks | Leave a Comment »

Django news, svn, git, mercurial

Posted by patrick sur avril 8, 2008

> Models in Django are conceptually based on tables in a database. I’d like to enhance that with the introduction of resource-based models.The basic idea comes from Ruby on Rails’ ActiveResource, which allows
> for interaction between two sites through a RESTful API. I would like
> to take this a step further and provide an implementation of this type
> of interface not bound specifically to REST. The interface will be
> generic enough to allow for different types of sources.I couldn’t be more interested in a GSoC because that’s exactly what I
miss in Django. Thanks for proposing that project, I really hope it
will be accepted and count me on for extending the base with at least
RDFModel once you setup the base. I’m already dreaming of some kind of: class User(models.RDFModel):
user = whatever(DjangoUser.name)
# mmh, I need more time to think about the way to declare triples

class Meta:
source = sources.FOAFSource()

Good luck!



Of course, this’ll require better tooling — we’re going to need to make sure
these links all work and maintain them correctly. More documents means we need
better indexing, searching, etc. Luckily someone else already wrote the
perfect tool for the job; it’s called Sphinx (
http://sphinx.pocoo.org/). It’ll
do everything I want, and more. I’m not interested in using different tools,
so let’s not discuss that. Suffice to say I’ve sketched out the code needed
and it’s perfect; all the hard work is going to be in actually
rewriting/moving around docs.

Right.How will I do this, you say? Here’s my plan:

* Using the outline from the existing documentation
http://toys.jacobian.org/django/docoutline-r7392/), make a new outline and figure out where each bit fits into the new regime. This doesn’t need to be in any way perfect or 100% complete; just good enough to get started from.

http://agiletesting.blogspot.com/2008/03/links-to-resources-from-pycon-talks.htmlI took some notes at the PyCon talks I’ve been to, and I’m gathering links to resources referenced in these talks. Hopefully they’ll be useful to somebody (I know they will be to me at least.) »MPI Cluster Programming with Python and Amazon EC2 » by Pete Skomoroch

* slides in PDF format
* Message Passing Interface (MPI) modules for Python: mpi4py, pympi
* ElasticWulf project (Beowulf-like setup on Amazon EC2)
* IPython1: parallel computing in Python
* EC2 gotchas

« Like Switching on the Light: Managing an Elastic Compute Cluster with Python » by George Belotsky

* S3FS: mount S3 as a local file system using Fuse (unstable)
* EC2UI: Firefox extension for managing EC2 clusters
* S3 Organizer: Firefox extension for managing S3 storage
* bundling an EC2 AMI and storing it to S3
* the boto library, which allows programmatic manipulation of Amazon Web services such as EC2, S3, SimpleDB etc. (a python-boto package is available for most Linux distributions too; for example ‘yum install python-boto)

« PyTriton: building a petabyte storage system » by Jonathan Ellis

* All this was done at Mozy (online remote backup, now owned by EMC, just like Avamar, the company I used to work for)
* They maxed out Foundry load balancers, so they ended up using LVS + ipvsadm
* They used erasure coding for data integrity — rolled their own algorithm but Jonathan recommended that people use zfec developed by AllMyData
* An alternative to erasure coding would be to use RAID6, which is used by Carbonite

« Use Google Spreadsheets API to create a database in the cloud » by Jeffrey Scudder

* slides online
* APIs and documentation on google code

« Supervisor as a platform » by Chris McDonough and Mike Naberezny

* slides online
* supervisord home page

« Managing complexity (and testing) » by Matt Harrison

* slides online
* PyMetrics module for measuring the McCabe complexity of your code
* coverage module and figleaf module for measuring your code coverage

Resources from lightning talks

* bug.gd — online repository of solutions to bugs, backtraces, exceptions etc (you can easy_install bug.gd, then call error_help() after you get a traceback to try to get a solution)
* geopy — geocode package
* pvote.orgKa-Ping Yee‘s electronic voting software in 460 lines of Python (see also Ping’s PhD dissertation on the topic of Building Reliable Voting Machine Software)
* bitsyblog — a minimalist approach to blog software

http://weblog.rubyonrails.com/2008/4/2/rails-is-moving-from-svn-to-git (We’ve been preparing for Rails to move the official source repository from Subversion to Git for some time now and it seems that it’ll happen over the next week or so. The premiere will happen alongside the official launch of Github.

The move will also mean that we’re going to be switching ticket tracking to Lighthouse. So now both our repository and ticket tracking will be powered by Rails applications, which is a nice bonus treat.When the move happens, we’ll freeze the existing Subversion repository and the Trac installation. Both will live on for a long time to come, but will be entirely deprecated. This means that your existing svn:externals will not break, but if you want the latest edge, you’ll have to get it from the new git repository.

So now is a great time to learn more about Git in anticipation of this move. I recommend starting with the Git for SVN’ers crash course.


Mercurial and even bazaar is quite popular among rubysts than Git…


Why Git and not Mercurial ? What’s all the Git hoopla about ?————————————————Although I do fully agree with punting SVN and moving to a distributed SCM, I don’t think that Git is the best solution for Rails. Git is fantastic no doubt, but its windows support is no where near Mercurial. Considering that Ruby on Rails is a platform independent framework, it would make sense to use something that plays nice for everybody.I’m not a windows user, but I’m sure there are still a few of them out there 🙂Also, the Mercurial command set is much more closer to SVN than Git, so the learning curve should be smaller.

Downsides to Mercurial: No peepcode show. And hghub.com isn’t ready. But guess what, turns out you don’t need either to get this working.

I’d highly recommend reconsidering your decision to move to Git and consider Mercurial as an alternative.


Although I prefer mercurial slightly more, git and mercurial are so alike that trying to figure out which is superior is just a waste of time. It’s better to just quickly choose one (even randomly) and stick with it.


Git is awesome, I’m not saying it isn’t. I still think Hg would fit better in this case, but since windows is in the minority here it really doesn’t matter.Getting a DSCM in there is the most important point, and I’m glad Rails is going in that direction.——————————————————————-

http://www.technobabble.dk/2008/mar/06/exposing-calendar-events-using-icalendar-django/ (« I recently wrote simple abstraction for exposing calendar events in Django as iCalendar feeds. It relies on vobject for managing the formatting of the calendar file, so you will need this if you want to try it out. It is available in Debian in the python-vobject package. »)

http://morethanseven.net/posts/why-the-webstandards-world-appears-to-be-choosing-django/ (« I’ve been noticing an interesting trend recently, not one I have any empirical evidence for mind, but one I though interesting non-the-less. Parts of the webstandards world appear to all be playing with Django. Part of this has been the odd mention down the pub, at barcamps or at SXSW this year. But the main source of information on the topic has been twitter. To name but a few I’ve seen tweets from Steve, Ross and Aral recently and Stuart and Cyril literally wont shut up about it.What’s interesting is that this didn’t happen with Rails, not in the corner of the pub that generally talks more about markup, javascript and CSS anyway. I’ve worked on a couple of Rails projects both personally and commercially, and I’ve just launched a little pet project build with django called doesyourapi. What follows is, to my mind, a few reasons why I think this trend exists and also why I think it will continue, at least for the time being…People: You can’t ignore the personal touch, and in Simon Willison and Stuart Langridge we already have two people who bridge the community and the web standards crowd, at least in the UK. Personal technology choices at least are often driven by personal correspondence. « )

http://fi.am/entry/using-the-geonames-database-with-django/ (« I’ve been using the Geonames database with Django for a while, as part of the ffloat.it project (which will be launched soon, stay tuned ;)). I’ve done some preprocessing to the data, normalizing some bits and denormalizing others, but always aiming for the best speed…Just unzip django-geonames-0.1.zip and drop the geo application inside your project. Add it to your INSTALLED_APPS, syncdb and then go to the geo directory and run the geoimport.py script (you’ll need to specify database credentials and db name, run geoimport.py -h for help). I haven’t included any documentation for the models, but i think, more or less, they should be self explanatory. Don’t hesitate to ask any question, posting a comment to this entry »)

http://code.google.com/p/django-wordpress-admin/ (« For those looking to migrate their self-hosted WordPress site to Django. This app will hook into the existing databases allowing management through Django’s admin interface and easy migration to a Django site. Built using manage.py inspectdb with some custom managers thrown in to separate out the different content that is all thrown into one big table (wp_posts). « )

Posted in 2008, python | Leave a Comment »