"…mais ce serait peut-être l'une des plus grandes opportunités manquées de notre époque si le logiciel libre ne libérait rien d'autre que du code…"

Archive for the ‘tests’ Category

Quelques liens sur des modules python intéressants: plac, tox, shinken, six

Posted by patrick sur juillet 13, 2010

  • http://pypi.python.org/pypi/plac/ (‘…There is no want of command line arguments parsers in the Python world. The standard library alone contains three different modules: getopt (from the stone age), optparse (from Python 2.3) and argparse (from Python 2.7). All of them are quite powerful and especially argparse is an industrial strength solution; unfortunately, all of them feature a non-zero learning curve and a certain verbosity. They do not scale down well, at least in my opinion. It should not be necessary to stress the importance of scaling down; nevertheless, a lot of people are obsessed with features and concerned with the possibility of scaling up, forgetting the equally important issue of scaling down. This is an old meme in the computing world: programs should address the common cases simply and simple things should be kept simple, while at the same keeping difficult things possible. plac adhere as much as possible to this philosophy and it is designed to handle well the simple cases, while retaining the ability to handle complex cases by relying on the underlying power of argparse. Technically plac is just a simple wrapper over argparse which hides most of its complexity by using a declarative interface: the argument parser is inferred rather than written down by imperatively. Still, plac is surprisingly scalable upwards, even without using the underlying argparse. I have been using Python for 8 years and in my experience it is extremely unlikely that you will ever need to go beyond the features provided by the declarative interface of plac: they should be more than enough for 99.9% of the use cases. plac is targetting especially unsophisticated users, programmers, sys-admins, scientists and in general people writing throw-away scripts for themselves, choosing the command line interface because it is the quick and simple. Such users are not interested in features, they are interested in a small learning curve: they just want to be able to write a simple command line tool from a simple specification, not to build a command-line parser by hand. Unfortunately, the modules in the standard library forces them to go the hard way. They are designed to implement power user tools and they have a non-trivial learning curve. On the contrary, plac is designed to be simple to use and extremely concise, as the examples below will show…’)
    • http://micheles.googlecode.com/hg/plac/doc/plac.html#plac-vs-the-rest-of-the-world (‘…Originally plac boasted about being « the easiest command-line arguments parser in the world ». Since then, people started pointing out to me various projects which are based on the same idea (extracting the parser from the main function signature) and are arguably even easier than plac: opterator by Dusty Phillips CLIArgs. Luckily for me none of such projects had the idea of using function annotations and argparse; as a   consequence, they are no match for the capabilities of plac. Of course, there are tons of other libraries to parse the command line. For instance Clap by Matthew Frazier which appeared on PyPI just the day before plac; Clap is fine but it is certainly not easier than placplac can also be used as a replacement of the cmd module in the standard library and as such it shares many features with the module cmd2 by Catherine Devlin. However, this is completely coincidental, since I became aware of the cmd2 module only after writing plac. by Pavel Panchekha ..’)
  • http://codespeak.net/tox/ (‘tox aims to automate state-of-the-art packaging, testing and deployment of Python software right from your console or CI server, invoking your tools of choice‘)
  • http://packages.python.org/six/ (‘Six provides simple utilities for wrapping over differences between Python 2 and Python 3.‘)
  • http://www.shinken-monitoring.org/faq/ (‘Shinken is a new monitoring tool in AGLv3 written in Python compatible with Nagios. The main goal of Shinken is to allow users to have a fully flexible architecture for their monitoring system that can easily scale to large environments. Shinken is backward compatible with the Nagios configuration standard and plugins. It works on any operative system and architecture that supports Python, which includes Windows and Mac OS X/Darwin‘)

Posted in 2 to 3, 2010, Administration, Administration système, Années, Génie logiciel, python, tests, tests | Tagué: , , , , , , , | Leave a Comment »

Python: quelques liens sur pycon 2010 : state of packaging, PEP345, PEP 386, unittest, cmd2, GIL, distributing, etc..

Posted by patrick sur février 28, 2010

pycon 2010 Atlanta

pycon 2010 Atlanta

State of python packaging

  • http://blip.tv/file/3259547 (‘:Deployment, development, packaging, and a little bit of the cloud’ by Ian Bicking)
  • http://blip.tv/file/2061678 (‘Eggs and Buildout Deployment in Python, Puzzled about Python eggs and packages? Wondering how to repeatably pull together collections of packages into standalone development, testing and deployment environments, all while managing inter-dependencies? In this participatory tutorial, we’ll start with distutils, walk through using eggs in the cheeseshop and creating your own eggs, touch a bit on using virtualenv to set up a development environment, and then dig into using zc.buildout to rigorously control assembly specifications, with build recipes, versioning and dependency management. We’ll close by showing how to create your own recipes. Attendees are strongly encouraged to bring a laptop or partner with someone who does.’)
  • http://blip.tv/file/2080263 (‘How I Distribute Python applications on Windows – py2exe & InnoSetup’ by Brian Dorsey There are many deployment options for Python code. I’ll share what has worked well for me on Windows, packaging command line tools and services using py2exe and InnoSetup. I’ll demonstrate a simple build script which creates windows binaries and an InnoSetup installer in one step. In addition, I’ll go over common errors which come up when using py2exe and hints on troubleshooting them. This is a short talk, so there will be a follow-up Open Space session to share experience and help each other solve distribution problems )
  • http://blip.tv/file/2072580 (‘[VIDEO HAS ISSUES: no audio first 1.5m] If you’ve ever created a nifty application that makes people’s lives easier you know the truly hard part is convincing others to use it. One way to increase the number of people installing your software is to convince Linux distributions to package your software so that their end users can install by using the system tools they’re used to. One way of convincing them is by making your application easy to package.’)
  • http://blip.tv/file/3263882 (‘New *and* Improved: Coming changes to unittest, the standard library test framework’ by Michael Foord)
  • http://blip.tv/file/3261338 (‘Easy command-line applications with cmd and cmd2’ by Catherine Devlin)
    • http://catherine.devlin.googlepages.com/cmd2.html (‘cmd2 is a tool for writing command-line interactive applications. It is based on the Python Standard Library’s cmd module, and can be used anyplace cmd is used simply by importing cmd2 instead’)
  • http://blip.tv/file/3254256 (‘Understanding the Python GIL by David Beazley‘)

Posted in 2010, Génie logiciel, python, tests | Tagué: , , , , , | Leave a Comment »

Quelques nouvelles de python: cobra, fusil, django

Posted by patrick sur février 12, 2008

http://www.blueskyonmars.com/2008/02/08/cobra-programming-language/ ( » So, we’ve got Jython and IronPython as Python language reimplementations. There’s also Boo, which is clearly heavily inspired by Python but has some interesting extensions (static typing, for example). I just came across Cobra (http://cobra-language.com/). Cobra, like Boo, is built on the .NET platform. The syntax is clearly inspired by Python, which I consider a good thing. In keeping line noise to a minimum, Cobra even ditches the “:” at the end of the line preceding a block of code. Chuck Esterbrook has also pulled inspiration from a number of other places. I recognize some D and Eiffel in there (it’s got design by contract and unit tests built right into the classes). There’s a comparison to Python available right on the Cobra site. »)

http://www.haypocalc.com/blog/index.php/2008/01/04/112-fusil-version-07 (« L’idée de Fusil est de simplifier l’écriture d’un projet de fuzzing. Il suffit de décrire le scénario pour préparer et surveiller l’environnement, sans avoir à s’occuper des détails techniques (rediriger la sortie du processus, détecter un plantage, etc)« )

http://www.voidspace.org.uk/python/weblog/arch_d7_2008_01_26.shtml#e930 (« This is an early announcement of a new project: Python-System, an implementation of the BCL and other .NET libraries in pure Python. Before you decide I’m completely mad, let me explain my motivation. Smile

The goal here is to provide libraries (the BCL and other .NET libraries) to aid the porting of IronPython code to run on CPython. Specifically, I would like to get Resolver One spreadsheets, exported as code, to run under CPython. I’m opening it up because it may also be useful to other people, and maybe you’ll help me. Laughing

I will be starting with partial implementations of Array, DateTime, Color, Point and friends (the ‘low hanging fruit’). The goal is for ‘compatible but not necessarily complete’ implementations of the classes (etc) that I need. Code that is useful for other people will happily be added of course.« )

http://blog.michaeltrier.com/2008/2/11/this-week-in-django-10-2008-02-10 (« This week we talk about a few source commits, a discussion about unicode usernames, Some cool projects from the community, the Tip of the Week, and a couple of questions from the IRC. »)

http://www.djangoproject.com/weblog/2007/dec/16/book/ («  The Django Book started shipping last week, and we’ve put the full text online for free. We put a draft of the book up about a year ago for comments, and were amazed by the quality (and quantity!) of responses. We read each of the comments (around 2500) as we revised the book towards a final print release. That print release has been available in stores for about a week, and we’ve put the text up for you to read for free. »)

http://www.holovaty.com/blog/archive/2007/12/12/1311 (« …Now that the Django Book is finally in the can, I’m mulling the idea of writing another book — this time, a book about online journalism. In the past two years, I’ve been to (way too) many journalism-related events and conferences trying to spread the good word about « journalism via computer programming, » and I’ve detected a strong, I daresay furious, demand, from journalists at all levels in the org chart, for information about this new form of journalism. Higher-ups want to know why they should employ programmers; middle managers want to know how to find them and how to treat them; and working journalists want to learn these skills and strategies. The problem is that I can’t point them anywhere for in-depth information. This book would attempt to solve that. I want to take a shot at writing a manual, a manifesto, a practical guidebook to this emerging discipline of database-driven Web journalism. It would be a combination of high-level strategy and low-level technique, probably split cleanly into two parts (one for the suits, one for the non-suits). »)

http://www.biologeek.com/journal/index.php/astuces-et-bonnes-pratiques-django (« Développant avec Django depuis maintenant près de deux ans (ça rajeunit pas tout ça…), je suis encore surpris de découvrir de nouvelles possibilités de temps en temps. Dans mon combat pour les bonnes pratiques, je pense qu’il y a quelques bases à avoir pour se lancer dans un projet d’envergure avec Django. Je vais essayer de lister les miennes, n’hésitez pas à ajouter les vôtres pour que ça devienne une ressource collaborative…)

Posted in DotNet, Ironpython, python, tests | Tagué: , , | Leave a Comment »

Les nouveautés dans ASP.NET Extensions: ASP.NET MVC et ASP.NET AJAX

Posted by patrick sur décembre 15, 2007

Comme le précise ce billet http://dosimple.ch/articles/MVC-ASP.NET/ écrit le 2 mai 2006 et celui-ci écrit le 13 mars 2007, le framework ASP.NET n’incite pas particulièrement à la séparation stricte de type Modèle, Vue, Contrôleur. On a vu que Monorail, projet open source, implémente le modèle MVC. Microsoft emboite le pas avec un grand retard en introduisant ASP.NET MVC: voir le billet suivant: http://weblogs.asp.net/scottgu/archive/2007/10/14/asp-net-mvc-framework.aspx (« One of the things that many people have asked for over the years with ASP.NET is built-in support for developing web applications using a model-view-controller (MVC) based architecture. Last weekend at the Alt.NET conference in Austin I gave the first public demonstration of a new ASP.NET MVC framework that my team has been working on. You can watch a video of my presentation about it on Scott Hanselman’s blog here.

What is a Model View Controller (MVC) Framework?

MVC is a framework methodology that divides an application’s implementation into three component roles: models, views, and controllers.

  • « Models » in a MVC based application are the components of the application that are responsible for maintaining state. Often this state is persisted inside a database (for example: we might have a Product class that is used to represent order data from the Products table inside SQL).

  • « Views » in a MVC based application are the components responsible for displaying the application’s user interface. Typically this UI is created off of the model data (for example: we might create an Product « Edit » view that surfaces textboxes, dropdowns and checkboxes based on the current state of a Product object).

  • « Controllers » in a MVC based application are the components responsible for handling end user interaction, manipulating the model, and ultimately choosing a view to render to display UI. In a MVC application the view is only about displaying information – it is the controller that handles and responds to user input and interaction.

One of the benefits of using a MVC methodology is that it helps enforce a clean separation of concerns between the models, views and controllers within an application. Maintaining a clean separation of concerns makes the testing of applications much easier, since the contract between different application components are more clearly defined and articulated.

The MVC pattern can also help enable red/green test driven development (TDD) – where you implement automated unit tests, which define and verify the requirements of new code, first before you actually write the code itself.

A few quick details about the ASP.NET MVC Framework

I’ll be doing some in-depth tutorial posts about the new ASP.NET MVC framework in a few weeks once the bits are available for download (in the meantime the best way to learn more is to watch the video of my Alt.net presentation).

A few quick details to share in the meantime about the ASP.NET MVC framework:

  • It enables clean separation of concerns, testability, and TDD by default. All core contracts within the MVC framework are interface based and easily mockable (it includes interface based IHttpRequest/IHttpResponse intrinsics). You can unit test the application without having to run the Controllers within an ASP.NET process (making unit testing fast). You can use any unit testing framework you want to-do this testing (including NUnit, MBUnit, MS Test, etc).

  • It is highly extensible and pluggable. Everything in the MVC framework is designed so that it can be easily replaced/customized (for example: you can optionally plug-in your own view engine, routing policy, parameter serialization, etc). It also supports using existing dependency injection and IOC container models (Windsor, Spring.Net, NHibernate, etc).

  • It includes a very powerful URL mapping component that enables you to build applications with clean URLs. URLs do not need to have extensions within them, and are designed to easily support SEO and REST-friendly naming patterns. For example, I could easily map the /products/edit/4 URL to the « Edit » action of the ProductsController class in my project above, or map the /Blogs/scottgu/10-10-2007/SomeTopic/ URL to a « DisplayPost » action of a BlogEngineController class.

  • The MVC framework supports using the existing ASP.NET .ASPX, .ASCX, and .Master markup files as « view templates » (meaning you can easily use existing ASP.NET features like nested master pages, <%= %> snippets, declarative server controls, templates, data-binding, localization, etc). It does not, however, use the existing post-back model for interactions back to the server. Instead, you’ll route all end-user interactions to a Controller class instead – which helps ensure clean separation of concerns and testability (it also means no viewstate or page lifecycle with MVC based views).

  • The ASP.NET MVC framework fully supports existing ASP.NET features like forms/windows authentication, URL authorization, membership/roles, output and data caching, session/profile state management, health monitoring, configuration system, the provider architecture, etc. »)

A voir:

  • https://pvergain.wordpress.com/2007/03/13/critique-de-larchitecture-aspnet/ (« Le problème avec ASP. Net est qu’il n’y a qu’un objet qui traite les demandes http, c’est l’objet PAGE. C’est lui qui a le contrôle de tout, et donc il mélange le code dit de «contrôle», et le code qui pilote la «visualisation» des éléments en html. Et bien souvent, on mélange aussi le code qui pilote le «Modèle» c’est à dire l’obtention des données directement depuis la base de données avec Ado.Net (c’est ce qu’on obtient lorsqu’on fait du WYSIWYG dans Visual Studio en choisissant les Sql Data Source et les glissant-déposant sur l’ihm). Cet anti-modèle (ou anti–pattern) a été souvent pointé du doigt par les architectes et développeurs, car en plus de faire produire du code spaghetti (bien que orienté objet), il rend impossible les tests systématisés (automatisés).« )
  • http://dosimple.ch/articles/MVC-ASP.NET/ (« Dans le framework ASP.NET la vue est un fichier HTML agrémenté de balises ASP. Le contrôleur et le modèle sont en général mélangés dans un objet qui dérive de la classe System.Web.UI.Page. »)
  • http://www.castleproject.org/monorail/index.html (« MonoRail is a MVC Web Framework inspired by Action Pack. MonoRail differs from the standard WebForms way of development as it enforces separation of concerns; controllers just handle application flow, models represent the data, and the view is just concerned about presentation logic. Consequently, you write less code and end up with a more maintainable application. Although the project name is MonoRail, we do not have any affiliation with the Mono project. MonoRail runs on Microsoft .Net 1.1, 2.0 and Mono.« )
  • http://en.wikipedia.org/wiki/ASP.NET_MVC_Framework (« ASP.NET MVC Framework is a Model-view-controller framework which Microsoft is adding to ASP.NET. It allows an application to be built as a composition of three roles: Model, View and Controller. A Model represents the state of a particular aspect in the application. Frequently, a model maps to a database table, with the entries in the table representing the state of the table. A Controller handles interactions and updates the model to reflect a change in state of the application. A View ASP.NET MVC Framework couples the models, views and controllers using interface-based contracts, thereby allowing each component to be easily tested independently. The view engine in the MVC framework uses regular .aspx pages to design the layout of the UI pages onto which the data is composed; however any interactions are routed to the controllers rather than using the postback mechanism. Views can be mapped to REST-friendly URLs. ASP.NET MVC Framework has been launched as a Community Technology Preview on December 10, 2007 extracts necessary information from a model and renders a UI to display that.. »)
  • http://www.hanselman.com/blog/ScottGuMVCPresentationAndScottHaScreencastFromALTNETConference.aspx ( » I attended the ALT.NET Conference last weekend in Austin, TX. I personally find the name « Alt » as in « Alternative » too polarizing and prefer terms like « Pragmatic.NET » or « Agile.NET. » At the conference I suggested, partially in jest, that we call it « NIH.NET » as in « Not Invented Here.NET. » 😉 Ultimately this is a group that believes in:
    • Continuous Learning
    • Being Open to Open Source Solutions
    • Challenging the Status Quo
    • Good Software Practices
    • DRY (Don’t Repeat Yourself)
    • Common Sense when possible.

    ScottGu gave an hour long presentation on the upcoming MVC Framework and I took some guerilla video. ScottGu’s presentation is here in Silverlight and it’s about 60 minutes long. Considering it’s so long, the video squished nicely. This was the first time the MVC Framework was shown publicly. Note that this was a Prototype, not the Production code and both ScottGu and I make that point a number of times to drive home that it’s early. Some of the code was written on a plane, just to give you an idea. After The Gu did his piece on the MVC Framework, I showed some prototype hacking that I’d done over the previous few days along with some work Phil Haack did. My presentation is here as Silverlight and it’s about 30 minutes long. I showed the Model View Controller with Controllers in IronPython and an IronPython view using a WebFormViewEngine. Then I talked about the possibilities of alternate ViewEngines and showed Phil Haack’s prototype RubyViewEngine. »)

  • http://weblogs.asp.net/scottgu/archive/2007/11/13/asp-net-mvc-framework-part-1.aspx ( » ASP.NET MVC Framework (Part 1) I’m going to use a simple e-commerce store application to help illustrate how the ASP.NET MVC Framework works. For today’s post I’ll be implementing a product listing/browsing scenario in it. Specifically, we are going to build a store-front that enables end-users to browse a list of product categories when they visit the /Products/Categories URL on the site>>Since it adds a testing project, does this require Team System? No – the good news is that Test Projects with VS 2008 are now supported with the VS 2008 Professional edition – and no longer require team system. You will also be able to use the VS Express and Standard edition products as well, and then use NUnit, MBUnit or some other testing framework with them (we’ll ship project templates for these as well…>> 2. Is it also possible to have this URL mapped: /Products/Beverages/3 instead of /Products/Beverages?page=3 or does it _need_ the parameter name? Yes – this is totally supported. Just set a route rule for /Products/Beverages with the format /Products/<Category>/<Page> – they’ll then get passed in as arguments to the action method:public List(string category, int? page) {}>>Also, some ideas and general remarks: 1. I’d love to see more helper methods for use in the views, I like how Ruby on Rails has many shortcuts for creating forms and formfields etc. Yes – we’ll have a rich library of helper methods for the views. They’ll include helpers to create all the standard forms, formfields and more.

>>2. Migrations! Not a part of MVC but from my experience with Ruby on Rails I would love to see support for this somewhere, anywhere! It would fit perfectly with the more agile way of developing that is possible with ASP.NET MVC. Rob Conery is building .NET Migrations support as part of the SubSonic project, and recently joined Microsoft. You’ll be able to use this with ASP.NET MVC

>>I’m also very keen to get my hands on the CTP. Scott, you mention, using Inversion of Control containers with the MVC framework. I’d be very interested in seeing a blog post on this subject. Also, will there be a sample application (with tests and IoC) available alonside the CTP? We have a IControllerFactory extensiblity point that owns creating Controller instances. You can use this with a IOC container to customize any Controller instance prior to it being called by the MVC framework. We’ll be posting samples of how to use this with ObjectBuilder and Windsor with the first CTP I believe

>> Very cool! One thing I’d like to see guidance on is developing MVC apps with multiple UIs. You say here that it’s best to put your models and controllers in the web app, but say we want a Winforms, WPF, Silverlight, and Web UI all doing similar things. Or a Mobile Web UI and Desktop Web UI… Would these still each need their own Models and Controllers, or does it make sense to have one library that they all share? If so, how is that supported? I’m still new to MVC, so if I’m missing something conceptually here, tell me! That is a good question, and one we’ll need to provide more guidance on in the future. In general it is often hard to share the same controller across both client and web UI, since the way you do data access and the stateful/stateless boundaries end up being very different. It is possible – but some guidance here would ceretainly be useful. My team should hopefully be coming out with this in the future

>>I really appreciate this material. Do you support the MVC pattern over the MVP pattern? Or are there just better scenarios for using each? The above approach I showed uses a MVC based pattern – where the Controller and the View tend to be more separate and more loosly coupled. In a MVP pattern you typically drive the UI via interfaces. This works well with a controls model, and makes a lot of sense with both WinForms and WebForms where you are using a postback model. Both MVC and MVP are perfectly fine approaches to use. We are coming out with the MVC approach for ASP.NET partly because we’ve seen more demand for it recently for web scenarios…

>> When can we expect a similar chapter with SubSonic as the DAL and scaffolding provider? (see http://oakleafblog.blogspot.com/2007/11subsonic-will-be-toolset-for-microsofts.html I’ll be covering scaffolding in a future blog post. LINQ to SQL scaffolding is built-in with ASP.NET MVC and doesn’t require SubSonic. SubSonic will also obviously be adding new ASP.NET MVC features as well.

>> My applications in .NET works with « 3 layers » pattern (business logic and data access in your own dll). How can i use this wonderfull MVC with my Models (data access) and Controllers (B.Logic)?? Because if i’m not reuse this, i’ve repeat code in every layer; then this MVC is not DRY (don’t repeat yourself) and the community don’t accept. There is no need to put your business and data logic in the same assembly as your controllers. You can split them up across multiple class library projects if you prefer

>> Does this mean that with MVC, we no longer use LinqDatasource in the View section? While the LinqDataSource control will technically work in MVC Views, if you are using a MVC model you wouldn’t want to place it there. Instead you want to perform all of your data and application logic in the Controller layer – and then pass the needed data to the view.

>> Perhaps I missed it somehow, but can you explain on which version of asp.net will this ctp run? The MVC framework builds on top of .NET 3.5

>> Scott, this is amazing timing! The URL mapping features inherit in an MVC application are PERFECT for the upgrade to ASP.NET 3.5 I’m making to my spelldamage.com site. Hosting question, will hosts simply need to support the ASP.NET 3.5 framework to allow us to run ASP.NET MVC applications? Your hoster will need to support .NET 3.5 for the MVC support.

>> Is’nt the MVC framework, in fact the Controller, implementation of the Front Controller pattern? Yes – the ASP.NET MVC framework uses a front-controller pattern. »)

  • http://weblogs.asp.net/scottgu/archive/2007/12/03/asp-net-mvc-framework-part-2-url-routing.aspx (« Last month I blogged the first in a series of posts I’m going to write that cover the new ASP.NET MVC Framework we are working on. The first post in this series built a simple e-commerce product listing/browsing scenario. It covered the high-level concepts behind MVC, and demonstrated how to create a new ASP.NET MVC project from scratch to implement and test this e-commerce product listing functionality. In today’s blog post I’m going to drill deeper into the routing architecture of the ASP.NET MVC Framework, and discuss some of the cool ways you can use it for more advanced scenarios in your application… What does the ASP.NET MVC URL Routing System do? The ASP.NET MVC framework includes a flexible URL routing system that enables you to define URL mapping rules within your applications. The routing system has two main purposes:
    • Map incoming URLs to the application and route them so that the right Controller and Action method executes to process them
    • Construct outgoing URLs that can be used to call back to Controllers/Actions (for example: form posts, <a href= » »> links, and AJAX calls)

Having the ability to use URL mapping rules to handle both incoming and outgoing URL scenarios adds a lot of flexibility to application code. It means that if we want to later change the URL structure of our application (for example: rename /Products to /Catalog), we could do so by modifying one set of mapping rules at the application level – and not require changing any code within our Controllers or View templates.

>> BTW, can you explain in short about the Active Record Type support in our MVC.

.NET 3.5 has LINQ to SQL built-in – which is a great ORM. LINQ to Entities will also be built-into .NET 3.5 in the future (it also ships with the MVC setup). The MVC framework doesn’t require LINQ to SQL or LINQ to Entities as the data model – it also works with NHibernate, LLBLGen, SubSonic, DataSets, DataReaders, and/or any other data model with .NET. We will, though, provide out of the box scaffolding support for LINQ to SQL and LINQ to Entities that delivers nice integration with the MVC support.

>> Question : What is this MockHttpContext in the UnitTest class? I mean, i can guess what it is .. but is it new in .NET 3.5 or the MVC framework? The MockHttpContext is an example mock object. It isn’t currently built-in with the MVC framework (with the first preview you’d need to create this yourself). I’m hoping that before the MVC framework ships, though, that there are a built-in set of mock objects and test helpers that you can use out of the box (without having to mock them yourself). Note that all core contracts and types with the MVC framework are mockable (non-sealed and interfaces). So you can also use any existing Mock framework to test it (RhinoMocks, TypeMock, etc).

>> 1) Is it possible to use routes with a file extension if I don’t want to enable wildcard script mappings? For example, /Products/List/Beverages.rails or /Products/Detail/34.rails ? Yes – this is fully supported. We recommend changing your route rule to /[controller].mvc/[action]/[id]. When you install the ASP.NET MVC Framework we automatically register this .mvc extension – although you are free to use any other one you want.

>> Hypothetically, do you think it would be possible to customise the route creation process so that route data could be gathered from attributes on actions? We don’t currently support this built-in, but hypothetically you could load all the controllers at app-startup and use reflection on them to calculate the route rules. This would enable the scenario you are after.

>> Is it possible to use IronRuby as the coding language to target the MVC framework?? Yes – we’ll support both IronRuby and IronPython with the ASP.NET MVC Framework.

>> I am liking this more and more. I can see how the routing configuration code in global.asax.cs could become quite large. In my application, I can see dozens, maybe hundreds of unique routing rules. Is there any way this configuration can be put in a file somewhere? That file would be read on Application start. Seems like that would make deployment of routing changes easier, too. We don’t currently have a pre-built file format for reading in mapping rules. But it should be relatively easy to build (you could even use LINQ to XML to read in and construct the rules).

>> Just out of curiosity where in the HttpApplication cycle are the routing rules evaluated and are they exposed in any way to the rest of the HttpApplication cycle? My current security system grants permissions at what would become the controller-action level so if the route determination is made early enough I’d really like to drive my authorization off of it.
The routing rules are resolved after the OnPostMapRequestRequestHandler event. This means that it will happen *after* your authorization rules evaluate. The good news is that this means you should be able to re-use your existing security system as-is.

>> Will there be any way to use an XML document to create the routing rules outside of the Global.asax code? Yep – this scenario will be supported.

>> I noticed that in Django, you have to repeat yourself kind of often when you have a deep nested hierarchy of pages. Your search & search-results pages seem to be continuing that trend. I’m sure I could come up with some hierarchical data structure which can be serialized into Route objects, but is the ASP.NET team planning anything along those lines that would come stock?

With our first preview the Route type is not extensible (you can’t subclass it) – which means you sometimes need to register multiple routes for a series of things. For the next preview we are looking at enabling Route to be sub-classed – in which case you could easily encapsulate multiple URL mappings into a single registration.

>> Have you thought about being able to define a regular expression for url matching and using backreferences or named captures as the tokenized url? I think this would allow for much more flexibility while keeping the list of routing rules down to a minimum.

Yes – this is something we are looking at. The challange with regular expressions is that only a subset of people really understand them (and importantly – understand how to optimize them). But I agree that having this as an option would be super flexible.

>> I must say that i am still worried about having to leave all the knowledge that we ave until now with webforms and start a new technology and still think that we would need some kind of a bridge to close the gap between today solution of webforms and tomorrow solution of MVC. Although the way you structure application flow will be different, I think you’ll be pleasantly surprised by the amount of knowledge overlap that exists. Authentication, Authorization, Caching, Configuration, Compilation, Session State, Profile Management, Health Monitoring, Administration, Deployment, and many, many other things are exactly the same. MVC views are also .aspx pages (that use .ascx user controls and .master files). So the concept re-use is quite heavy there as well. »)

  • http://weblogs.asp.net/scottgu/archive/2007/12/06/asp-net-mvc-framework-part-3-passing-viewdata-from-controllers-to-views.aspx (« The last few weeks I have been working on a series of blog posts that cover the new ASP.NET MVC Framework we are working on. The ASP.NET MVC Framework is an optional approach you can use to structure your ASP.NET web applications to have a clear separation of concerns, and make it easier to unit test your code and support a TDD workflow. The first post in this series built a simple e-commerce product listing/browsing site. It covered the high-level concepts behind MVC, and demonstrated how to create a new ASP.NET MVC project from scratch to implement and test this e-commerce product listing functionality. The second post in this series drilled deep into the URL routing architecture of the ASP.NET MVC framework, and discussed both how it worked as well as how you can handle more advanced URL routing scenarios with it. In today’s blog post I’m going to discuss how Controllers interact with Views, and specifically cover ways you can pass data from a Controller to a View in order to render a response back to a client. In Part 1 of this series, we created an e-commerce site that implemented basic product listing/browsing support. We implemented this site using the ASP.NET MVC Framework, which led us to naturally structure the code into distinct controller, model and view components. When a browser sends a HTTP request to our web site, the ASP.NET MVC Framework will use its URL routing engine to map the incoming request to an action method on a controller class to process it. Controllers in a MVC based application are responsible for processing incoming requests, handling user input and interactions, and executing application logic based on them (retrieving and updating model data stored in a database, etc). When it comes time to render an HTML response back to the client, controllers typically work with « view » components – which are implemented as separate classes/templates from the controllers, and are intended to be focused entirely on encapsulating presentation logic. Views should not contain any application logic or database retrieval code, instead all application/data logic should only be handled by the controller class. The motivation behind this partitioning is to help enforce a clear separation of your application/data logic from your UI generation code. This makes it easier to unit test your application/data logic in isolation from your UI rendering logic.Views should only render their output using the view-specific data passed to it by the Controller class. In the ASP.NET MVC Framework we call this view-specific data « ViewData ». The rest of this blog post is going to cover some of the different approaches you can use to pass this « ViewData » from the Controller to the View to render.

>> One question Scott: Let say i’m jumpng on MS MVC bandwagon, do i have to abandon asp.net Page Life cycle godddies, should i set up my mind to different approach. When you use the ASP.NET MVC approach you’ll want to have all post in your site go to your Controller. This helps ensure that the application can be easily tested and preserves the separation of concerns. This is different from using ASP.NET server controls which postback to themselves (which is very powerful too – but means that it can sometimes be slightly harder to test). What doesn’t change is that all of other ASP.NET pieces (forms authentication, role management, session state, caching, profiles, configuration, compilation, httprequest/response, health monitoring, etc, etc) works with both web forms based pages and MVC based pages. MVC UI are also built with .aspx, .master, and .ascx files – so there is a high level of skill re-use there as well.

>> Some asp.net controls require <form runat=server>. If we use a asp:dropdownlist for example, we have to place in the asp:form. And this means viewstate comes back! Is there any way to get rid of hidden form fields? Or you suggest that we do must use classic HTML controls ? I’ll cover this more in my next blog in this series. We have helpers that can automate generating things like dropdownlists, etc. Think of them as controls for MVC. These don’t use viewstate and give you a little more control over the output of your HTML.

>> It would be nice if you can just compare a bit our MVC with ROR. Within ROR, we can create tables, Columns and Rows with Ruby, without using a single line of SQL, and all its done through ActiveRecord. In short all CRUD advantages.

>> SubSonic is almost in this line. Can you explain more in this line or how SubSonic can be used to take ActiveRecord Type advantages.

RoR is made up of several components.

« Action Controller » is the name of the MVC framework that Rails uses. That is the closest analogy to the ASP.NET MVC Framework – and at a high-level the ASP.NET MVC framework uses many of the same core concepts (URLs map to controller actions). The ASP.NET MVC Framework has a few additional features (like the ability to map incoming parameters directly to action method parameters). It is also more explicit about calling RenderView within the request.

« Active View » is the name of the View engine that Rails uses. This is analagous to the .aspx/.master/.ascx infrastructure ASP.NET has. Our view engine is IMO a little richer, and supports several additional features (declarative controls/components, templating, multiple regions for nested master pages, WYSIWYG designer, strongly typed ViewData access, pluggable storage provider, declarative localization, etc).

« Active Record » is the name of the ORM (object relational mapper) that RoR uses. This is analagous to an ORM in the .NET world – whether it is LINQ to SQL, LINQ to Entities, LLBLGen, SubSonic, NHibernate, ActiveRecord (the .NET version), etc. All of these ORMs have ways to map rows and columns in a database to objects that a developer then manipulates and works against (and can save changes back). The LINQ to SQL ORM is built-in to .NET 3.5 today, and LINQ to Entities will be built-in with .NET 3.5 SP1.

I think from your follow-up question above you are referring specifically to the « migrations » feature in RoR – which allows you to define table schemas using code, and version the schemas forward/backward. There isn’t a built-in version of this in the core .NET Framework today – although the SubSonic project has recently released a version that allows you to-do this using .NET. You can find out more about it here: www.subsonicproject.com.

>> Superb, …but I still longing for the IOC integration (Spring.NET, Windsor, StructureMap)…maybe in the next post ? I am planning on posting about IOC and dependency injection in the future (the ASP.NET MVC framework fully supports it and can be used with Spring.NET, Windsor, StructureMap, etc). I have a few other more basic MVC tutorials I need to get out of the way first – but testing and dependency injection are on the list for the future.

>> Nice series, eagerly waiting to see AJAX approach in the new ASP.NET MVC Framework, so when shall we expext that.
We will have ASP.NET AJAX support with the ASP.NET MVC Framework. I’ll talk more about this in a future tutorial series post.

>> This is coming together nicely. I was just hoping you might explain why there are 3 different ways to pass the ViewData? Does each method offer something that the others don’t? If not, surely it would be best to choose one method and force all developers to follow the same practice? Conceptually there are two ways to pass viewdata – either as a dictionary, or as a strongly typed object. In general I recommend using the strongly typed dictionary approach – since this gives you better compilation checking, intellisense, and refactoring support. There are, however, some times when having a more latebound dictionary approach ends up being flexible for more dynamic scenarios« )

  • http://blog.wekeroad.com/2007/10/26/microsoft-subsonic-and-me/ (« Rather than try and come up with some lame metaphors and trite pop-culture references, I’ve decided to use some advice from English 101 Professor:

    “Whatever the hell you’re trying to say, just say it”

    So I will: I’m going to work for Microsoft. I just signed the offer letter. I’ll be working with the ASP.NET guys on the new MVC platform as well as some other groovy things like Silverlight. I get to work “across the hall” from one of my very good friends – Phil Haack. I think it’s worth pointing out that SubSonic hasn’t been “bought”. Some might smell a conspiracy here, but I’ll leave that to the X-Files and Cap’n Crunch crowd to drum up all the evil reasons why the mothership has “beamed me up”. SubSonic will remain under the same MPL 1.1 license it always has, and will remain as completely Open Source as it always has – nothing will change at all. I’m just getting paid, essentially, to work on it 🙂...This is crucial to me. I decided to be direct with him and make sure we both understood these important points:

    “I want to be sure I have complete creative control over SubSonic, and that you don’t censor my blog… is that cool?”

    Shawn’s response is why I took the job:

    “Well Duh…” (he added some more things that were a bit more eloquent than “duh” – but I don’t think I was listening).

    I can make jokes about the UAC on my blog? And make up fictional Matrix converstations with ScottGu? Sign me up! I start on November 12th, right after DevConnections in Vegas (come on by if you’re out that way at the DNN Open Force« )

  • http://www.subsonicproject.com/ (« A Super High-fidelity Batman Utility Belt. SubSonic works up your DAL for you, throws in some much-needed utility functions, and generally speeds along your dev cycle. Why SubSonic ? Because you need to spend more time with your friends, family, dog, bird, cat… whatever. You work too much. Coding doesn’t need to be complicated and time-consuming. »)
  • http://en.wikipedia.org/wiki/Test-driven_development (« Test-Driven Development (TDD) is a software development technique consisting of short iterations where new test cases covering the desired improvement or new functionality are written first, then the production code necessary to pass the tests is implemented, and finally the software is refactored to accommodate changes. The availability of tests before actual development ensures rapid feedback after any change. Practitioners emphasize that test-driven development is a method of designing software, not merely a method of testing. Test-Driven Development began to receive publicity in the early twenty-first century as an aspect of Extreme Programming, but more recently is creating more general interest in its own right. Along with other techniques, the concept can also be applied to the improvement and removal of software defects from legacy code that was not developed in this way . »)
  • ss
  • http://weblogs.asp.net/scottgu/archive/2007/12/09/asp-net-mvc-framework-part-4-handling-form-edit-and-post-scenarios.aspx
  • (« ASP.NET MVC Framework (Part 4): Handling Form Edit and Post Scenarios   The last few weeks I have been working on a series of blog posts that cover the new ASP.NET MVC Framework we are working on.  The ASP.NET MVC Framework is an optional approach you can use to structure your ASP.NET web applications to have a clear separation of concerns, and make it easier to unit test your code and support a TDD workflow. The first post in this series built a simple e-commerce product listing/browsing site.  It covered the high-level concepts behind MVC, and demonstrated how to create a new ASP.NET MVC project from scratch to implement and test this e-commerce product listing functionality.  The second post drilled deep into the URL routing architecture of the ASP.NET MVC framework, and discussed both how it worked as well asthird post discussed how Controllers interact with Views, and specifically covered ways you can pass view data from a Controller to a View in order to render a response back to a client.  In today’s blog post I’m going to discuss approaches you can use to handle form input and post scenarios using the MVC framework, as well as talk about some of the HTML Helper extension methods you can use with it to make data editing scenarios easierClick here to download the source code for the completed application we are going to build below to explain these concepts… Our Data Model. We are going to use the SQL Server Northwind Sample Database to store our data.  We’ll then use the LINQ to SQL object relational mapper (ORM) built-into .NET 3.5 to model the Product, Category, and Supplier objects that represent rows in our database tables. We’ll begin by right-clicking on our /Models sub-folder in our ASP.NET MVC project, and select « Add New Item » -> « LINQ to SQL Classes » to bring up the LINQ to SQL ORM designer and model our data objects…To learn more about LINQ and LINQ to SQL, please check out my LINQ to SQL series here The HtmlHelper object (as well as the AjaxHelper object – which we’ll talk about in a later tutorial) have been specifically designed to be easily extended using « Extension Methods » – which is a new language feature of VB and C# in the VS 2008 release.  What this means is that anyone can create their own custom helper methods for these how you can handle more advanced URL routing scenarios with it.  The objects and share them for you to use. We’ll have dozens of built-in HTML and AJAX helper methods in future previews of the ASP.NET MVC Framework.  In the first preview release only the « ActionLink » method is built-into System.Web.Extensions (the assembly where the core ASP.NET MVC framework is currently implemented).  We do, though, also have a separate « MVCToolkit » download that you can add to your project to obtain dozens more helper methods that you can use with the first preview release.
  • >> This is nice. But do you handle subcomponents? subviews? Is view reuse possible inside other view? This is very important for even for modest sized applications? How would you handle posts correcly inside of subviews 😉 ?The first public MVC preview doesn’t have the concept of SubControllers yet (where you can attach them for regions of a larger view).  That is something we have planned for to tackle soon though.

    >> However, liviu raises good questions about Sub-Controllers, Composite Views and AJAX scenarios. How Will the MVC framework address the complex wiring of different parts of a page to different controllers for async updates?? My brain hurts thinking about it, yet I have implement these portal pages all the time cos our customers demand it. The first ASP.NET MVC Preview allows you to call RenderView (for example: RenderView(« Edit »)) and have it apply to either a .aspx viewpage or a .ascx viewusercontrol.  This ends up being really useful for scenarios where you are doing AJAX callbacks and want to refresh a portion of a page.  These callbacks could either be to the same Controller that rendered the origional page, or to another Controller in the project. We don’t have all the AJAX helper methods yet for this – but you could roll them yourself today.  We’ll obviously be releasing an update that does include them in the future.

  • >> Can you provide an example of how we might use a server side control to generate the drop down list or a textbox (a la WebForms) along with the pro’s and con’s of each approach? Personally, I do not like the old ASP <%= … %> syntax; but if there is a good reason to use it then I am willing to change that opinion. The main reason I showed the <% %> syntax in this post was that we don’t have the server-side MVC control equivalents available yet for this dropdown and text scenario.  That is on our roadmap to-do though – at which point you don’t need to necessarily use the <% %> syntax. In general what we’ve found is that some developers hate the <% %> syntax, while others prefer it (and feel that controls get in the way).  We want to make both sets of developers happy, and will have options for both. 🙂
  • >> Great work on the MVC model, it is a clean and intuitive model that appears to scale well. My only recommendation is that your actions (verbs) appear after the ids (nouns), e.g. [Controller]/[Id]/[Action]/  That would keep the architecture noun focused, opposed to verb focused. With this approach you can easily build an administrative interface on a public read-only site by adding verbs such as /edit/ or /delete/ at the end of your URL. A simple security rule can be added to the Controller that says ignore all Verbs after a noun, such as category or product, if user is not part of an administrative group. Thanks for the suggestion Josh.  You can do this today by creating your own custom route registration.  In the next preview I believe Route will be extensible so that you could also create a single resource route that supports these semantics.>> I was wondering if we could get an idea of the official « roadmap » for asp.net mvc?  Specifically, I’d like to know what features the team plans to release on what dates … ultimately, when this thing will RTM.  As I’ll be evaluating mvc and monorail for an upcoming app, this kind of information would really be helpful in determining which direction to go. We don’t have a formally published roadmap just yet – we will hopefully publish one early next year though.« )
  • http://en.wikipedia.org/wiki/ASP.NET_AJAX (« ASP.NET AJAX, formerly code-named Atlas, is a set of extensions to ASP.NET developed by Microsoft for implementing Ajax functionality. Including both client-side and server-side components, ASP.NET AJAX allows the developer to create web applications in ASP.NET 2.0 (and to a limited extent in other environments) which can update data on the web page without a complete reload of the page. The key technology which enables this functionality is the XMLHttpRequest object, along with Javascript and DHTML. ASP.NET AJAX was released as a standalone extension to ASP.NET in January 2007 after a lengthy period of beta-testing. It was subsequently included with version 3.5 of the .NET Framework, which was released alongside Visual Studio 2008 in November 2007.« )

Posted in 2007, Acces aux données, active record, AJAX, Architecture logicielle, ASP.NET, Développement logiciel, design pattern, DotNet, IDE-GUI, Ironpython, javascript, open source, ORM, RIA, ruby, tests, Web Frameworks | Tagué: , , , , , , , | Leave a Comment »

Des nouvelles du monde python: ironpython, windmill, satchmo, grok + le réseau social ohloh.net

Posted by patrick sur novembre 10, 2007

Quelques nouvelles du monde python:

  • différentes mesures de la popularité du langage python sur http://www.langpop.com/. Le langage C vient en première position tandis que Python se situe entre la 6e et la 12e position suivant les outils de mesure.
  • la version 2.06a de Ironpython est sortie (« …this release includes all of the changes we have made in the past month. The most noteworthy differences include:
    • the IronPython project is now distributed under the Microsoft Public License (Ms-PL)
    • improved support of COM events under the –X:PreferComDispatch flag
    • the abstract syntax tree for IronPython is being hardened so to speak
    • we have included an updated DLR sample in the source zip file – ToyScript
    • Internally we have migrated our Visual Studio project and solution files over to a Visual Studio 2008 beta release. While we’ve modified these files in the source distribution of IronPython to be compatible with Visual Studio 2005 and the msbuild tool, it’s possible we may have missed something. If you experience any problems building IronPython from VS tools please let us know.
    • The ToyScript sample is not intended to be distributed with IronPython on a permanent basis and will likely be released in some other fashion in the not too distant future. »)
    • For whatever reasons, IronPython 2.0 Alpha 6 started to invoke delegate to abstract method. (Previous versions didn’t.) This triggered Mono bug 317488 which is apparently hard to fix. https://bugzilla.novell.com/show_bug.cgi?id=317488 I developed completely insane workarounds to avoid this problem. Now you can laugh (or weep). https://fepy.svn.sourceforge.net/svnroot/fepy/trunk/patches/latest/patch-317488
  • Source: http://www.voidspace.org.uk/python/weblog/ (« ...Web application testing with Windmill. A new (Python based) browser testing tool was just announced on the Testing in Python Mailing List. It comes out of the Chandler Project and claims to implement a larger set of a browser testability than Selenium: Windmill.
    • Windmill is an Open Source AJAX Web UI Testing framework written in Python and JavaScript originally for testing the Chandler Server WebInterface at OSAF. After spending time with Selenium we realized we had a variety of needs that weren’t being fulfilled and built Windmill from the ground up. Windmill implements cross browser testing, in-browser recording and playback, and functionality for fast accurate debugging and test environment integration. Support for Firefox, IE6/7, Safari on * *Windows, Linux and Mac OS 10.4 and 10.5.)
  • http://www.ohloh.net/projects/search?page=1&q=django (« Ohloh is an open source network that connects people through the software they create and use. Ici on recense les projets basés sur le framework web python Django. »)
  • http://www.satchmoproject.com/ (« ...Satchmo’s mission is to use Django to create an open source framework for creating unique and robust online stores. To provide maximum flexibility, Satchmo is licensed under the BSD license…. This project was started by a group of individuals that were interested in using the Django framework to create a robust shopping cart solution. After some more discussions, we have decided to focus on building a modular framework of shopping cart or eCommerce packages that can be easily put together to form a full store. Here’s a bunch of questions you might be asking yourself at this point… Why use Django? First off, we wanted to use Python for the project. Many of us have experience with similar PHP based projects and we quickly realized that they tend to break down pretty quickly under the unwieldy syntax, lack of good OO support and hackish nature of many of the projects. Once we decided on Python, there were many frameworks to evaluate. They all have tradeoffs but there are some things we really liked about Django:
    • Robust, scalable system that has been used on major commercial sites
    • Clean separation of program logic from presentation
    • Nice abstraction of SQL (but with the capability to code SQL if required)
    • Solid documentation
    • URL mapping capabilities will be very useful for a shopping cart solution
    • Out of the box admin capabilities making it easy to get up and running & provide enhanced security for multiple users in a store.

This list is not exhaustive and there could be endless debate and flamewars on the choices but we’ve all reached this decision independently and encourage you to investigate & make your own choice….« )

  • http://grok.zope.org/ (« Grok is a web application framework for Python developers. It is aimed at both beginners and very experienced web developers. Grok has an emphasis on agile development. Grok is easy and powerful. You will likely have heard about many different web frameworks for Python as well as other languages. Why should you consider Grok?
    • Grok offers a lot of building blocks for your web application.
    • Grok is informed by a lot of hard-earned wisdom.

Grok accomplishes this by being based on Zope 3, an advanced object-oriented web framework. While Grok is based on Zope 3, and benefits a lot from it, you do not need to know Zope at all in order to get productive with Grok« )

..etc, etc…

Posted in Ironpython, python, REST, tests, Web applications | Tagué: , , , , | Leave a Comment »

Tools for Web testing, Web recording/playback, nose unittest extensions, and code coverage.

Posted by patrick sur mars 17, 2007

Sources

Other web testing resources:

Introduction

Author: Titus Brown

 

Contents

At PyCon ’07, I gave a talk on testing tools in which I « performed » nine live demos of features from twill, scotch, pinocchio, and figleaf. These are tools for Web testing, Web recording/playback, nose unittest extensions, and code coverage.

This is the source code for that talk.

The Python code should work on any UNIX-like machine; I gave my talk with my MacBook, running Python 2.3.

Note that the only demo that didn’t work during the talk was the first one, in which I used ‘subprocess’ to start a CherryPy server. Since this is contraindicated IMO (I suggest using wsgi_intercept; see Demo 2) I was more or less happy that it failed. (It failed because I made a last minute adjustment to the command that ran app.py, and didn’t test it before the talk… ;(

 

Files

You can get the entire source code (including this text) at

http://darcs.idyll.org/~t/projects/pycon-07-talk-source-latest.tar.gz

The latest version of this README should always be accessible at

http://darcs.idyll.org/~t/projects/pycon-07-talk-source/README.html

 

Warning

This was billed as an « intermediate » talk at PyCon, and I jumped right into code rather than giving a detailed introduction. This document follows that strategy — so go look at the code and read what I have to say afterwards!

 

Feedback

I’m interested in your comments. Please either drop me an e-mail directly or go comment on the associated blog post.

 

Requirements

You will need twill 0.9b1 (the latest available through easy_install), nose 0.9.1 (the latest available through easy_install), scotch (latest), figleaf (latest), and pinocchio (latest). You will also need to have CherryPy 3.x and Django 0.95 installed.

You may need to adjust your PYTHONPATH to get everything working. Check out env.sh to see what I put in my path before running everything.

 

Demo 1: Testing CherryPy

Purpose: show how to use twill to test a (very simple!) CherryPy site.

twill is a simple Web scripting language that lets you automate Web browsing (and Web testing!) Here I’ll show you how to use it to run a few simple tests on a simple CherryPy site.

The CherryPy application under test is in cherrypy/app.py. You can check it out for yourself by running python app.py; this will set up a server on http://localhost:8080/.

The URLs to test are: http://localhost:8080/, which contains a form; http://localhost:8080/form/, which processes the form submit; and http://localhost:8080/page2/, which is just a simple « static » page.

The twill test script for this app is in cherrypy/simple-test.twill. All it does is go to the main page, confirm that it loaded successfully and contains the words « Type something », fills in the form with the string « python is great », and submits it. The final command verifies that the output is as expected.

If you wanted to run all of this stuff manually, you would type the following (in UNIX):

python app.py &twill-sh -u http://localhost:8080/ simple-test.twill

So, how do you do it with twill and nose?

Take a look at the unit test source code in cherrypy/demo1/tests.py. This is a nose test that you can run by typing

nosetests -w demo1/

from within the cherrypy/ subdirectory.

Try running it.

You should see a single ‘.’, indicating success:

.----------------------------------------------------------------------Ran 1 test in 2.069sOK

Now try

nosetests -w demo1/ -v

You should see

tests.test ... ok----------------------------------------------------------------------Ran 1 test in 2.069sOK

So yes, it all works!

Briefly, what is happening here is that:

  1. tests.py is discovered and imported by nose (because it has the magic ‘test’ in its name); then setup(), test(), and teardown() are run (in that order) because they are names understood by nose.
  2. setup executes the application app.py, capturing its stdout and stderr into a file-like object (which is accessible as pipe.stdout). setup has to wait a second for app.py to bind the port, and then sets the URL of the Web server appropriately.
  3. test then runs the twill script via twill.execute_file, passing it the initial URL to go to.
  4. teardown calls a special URL, exit, on the Web app; this causes the app to shut down (by raising SystemExit). It then waits for the app to exit.

A few notes:

  1. setup and teardown are each run once, before and after any test functions. If you added in another test function — e.g. test2 — it would have access to url and pipe and an established Web server.

  2. Note that url is not a hardcoded string in the test; it’s available as a global variable. This lets any function in this module (and any module that can import tests) adjust to a new URL easily.

  3. Also note that url is not hardcoded into the twill script, for the same reason. In fact, because this twill script doesn’t alter anything on the server (mainly because the server is incredibly dumb 😉 you could imagine using this twill script as a lifebeat detection for the site, too, i.e. to check if the site is minimally alive and processing Web stuff properly.

  4. What if the Web server is already running, or something else is running on the port?

  5. More generally, what happens when the Popen call goes awry? How do you debug it?

    (Answer: you’ve got to figure out how to get ahold of the stdout/stderr and print it out to the environment, which can be a bit ugly.)

  6. What happens if /exit doesn’t work, in teardown?

    (Answer: the unit tests hang.)

Notes 4-6 are the reasons why you should think about using the wsgi_intercept module (discussed in Demo 2) to test your Web apps.

 

Demo 2: Testing CherryPy without exec’ing a process

Purpose: demonstrate the use of wsgi_intercept.

The use of subprocess in Demo 1 was a big ugly problem: once you shell out to a command, doing good error handling is difficult, and you’re at the mercy of the environment. But you needed to do this to run the Web server, right?

Well, yes and no. If your goal was to test the entire Web stack — from your OS socket recv, through the CherryPy Web server, down to your application — then you really need to do things this way.

But that’s silly. In general, your unit and functional tests should be testing your code, not CherryPy and your OS; the time for testing that everything works together is later, during your staging and end-to-end testing phase(s). Generally speaking, though, your OS and Web server are not going to be simple things to test and you’re better off worrying about them separately from your code. So let’s focus on your code.

Back to the basic question: how do you test your app? Well, there’s a nifty new Python standard for Web app/server interaction called WSGI. WSGI lets you establish a nicely wrapped application object that you can serve in a bunch of ways. Conveniently, twill understands how to talk directly to WSGI apps. This is easier to show than it is to explain: take a look at cherrypy/demo2/tests.py. The two critical lines are in setup(),

wsgi_app = cherrypy.tree.mount(app.HelloWorld(), '/')twill.add_wsgi_intercept('localhost', 80, lambda: wsgi_app)

The first line asks CherryPy to convert your application into a WSGI application object, wsgi_app. The second line tells twill to talk directly to wsgi_app whenever a twill function asks for localhost:80.

Does it work?

Well, you can try it easily enough:

nosetests -w demo2/ -v

and you should see

tests.test ... ok----------------------------------------------------------------------Ran 1 test in 0.827sOK

So, yes, it does work!

Note that the test itself is the same, so you can actually use the test script simple-test.twill to do tests however you want — you just need to change the test fixtures (the setup and teardown code).

Note also that it’s quite a bit faster than demo1, because it doesn’t need to wait for the server to start up.

And, finally, it’s much less error prone. There’s really no way for any other process to interfere with the one running the test, and no network port is bound; wsgi_intercept completely shunts the networking code through to the WSGI app.

(For those of you who unwisely use your own Web testing frameworks, wsgi_intercept is a generic library that acts at the level of httplib, and it can work with every Python Web testing library known to mankind, or at least to me. See the wsgi_intercept page for more information.)

 

Demo 3: Basic code coverage analysis with figleaf

Purpose: demonstrate simple code-coverage analysis with figleaf.

Let’s move on to something else — code coverage analysis!

The basic idea behind code coverage analysis is to figure out what lines of code are (and more importantly aren’t) being executed under test. This can help you figure out what portions of your code need to be tested (because they’re not being tested at all).

figleaf does this by hooking into the CPython interpreter and recording which lines of code are executed. Then you can use figleaf’s utilities to do things like output an HTML page showing which lines were and weren’t executed.

Again, it’s easier to show than it is to explain, so read on!

First, start the app with figleaf coverage:

figleaf app.py

Now, run the twill script (in other window):

twill-sh -u http://localhost:8080/ simple-test.twill

Then CTRL-C out of the app.py Web server, and run

figleaf2html

This will create a directory html/; open html/app.py.html in a Web browser. You should see a bunch of green lines (indicating that these lines of code were executed) and two red lines (the code for page2 and exit). There’s your basic coverage analysis!

Note that class and function definitions are executed on import, which is why def page2(self): is green; it’s just the contents of the functions themselves that aren’t executed.

If you open html/index.html you’ll see a general summary of code files executed by the Python command you ran.

 

Demo 4: More interesting code coverage analysis with nose and figleafsections

Purpose: demonstrate the figleafsections plugin that’s part of pinocchio.

The figleafsections plugin to pinocchio lets you do a slightly more sophisticated kind of code analysis. Suppose you want to know which of your tests runs what lines of code? (This could be of interest for several reasons, but for now let’s just say « it’s neat », OK?)

For this demo, I’ve constructed a new pair of unit tests: take a look at cherrypy/demo3/tests.py. The first test function (test()) is identical to Demo 2, but now there’s a new test function — test2(). All that this function does is exercise the page2 code in the CherryPy app.

Now run the following commands in the cherrypy/ directory:

rm .figleafnosetests -v --with-figleafsections -w demo3annotate-sections ./app.py

This runs the tests with a nose plugin that keeps track of which tests are executing what sections of app.py, and then annotates app.py with the results. The annotated file is app.py.sections; take a look at it!

When you look at app.py.sections you should see something like this:

-- all coverage --| tests.test2| | tests.test

| | |        | #! /usr/bin/env python

| import cherrypy

|

| class HelloWorld(object):

|     def index(self):

+   |         return """<form method='POST' action='/form'>

|                   Type something: <input type='text' name='inp'>

|                   <input type='submit'>

|                   </form>"""

|     index.exposed = True

|

|     def form(self, inp=None, **kw):

+   |         return "You typed: \"%s\"" % (inp,)

|     form.exposed = True

|

|     def page2(self):

+     |         return "This is page2."

|     page2.exposed = True

|

|     def exit(self):

|         raise SystemExit

|     exit.exposed = True

|

| if __name__ == '__main__':

|     cherrypy.quickstart(HelloWorld())

What this output shows is that tests.test executed the index() and form() functions, while tests.test2 executed the page2() function only — just as you know from having read cherrypy/demo3/tests.py. Neat, eh?

See my blog post on the subject for some more discussion of how this can be useful.

 

Demo 5: Writing a simple twill extension to do form « fuzz » testing

Purpose: show how easy it is to write twill extensions.

Since twill is written in Python, it’s very easy to extend with Python. All you need to do is write a Python module containing the function(s) that you want to use within twill, and then call extend_with <module>. From that point on, those functions will be accessible from within twill. (Note that extension functions need to take string arguments, because the twill mini-language only operates on strings.)

For example, take a look at cherrypy/demo4/randomform.py. This is a simple extension module that lets you fill in form text fields with random values; the function fuzzfill takes a form name, a min/max length for the values, and an optional alphabet from which to build the values. You can call it like this:

extend_with randomformfuzzfill <form> 5 15 [ <alphabet> ]

If you look at the randomform.py script, the only real trickiness in the script is where it uses the twill browser API to retrieve the form fields and fill them with text. Conveniently, this entire API is available to twill extension modules.

Let’s try running it! The twill script cherrypy/fuzz-test.twill is a simple script that takes the CherryPy HelloWorld application and fills in the main page form field with a random alphanumeric string. As in Demo 2, we can put this all together in a simple unit test framework; see cherrypy/demo4/tests.py for the actual code.

You can run the demo code in the usual way:

nosetests -w demo4/ -v

If you run it without output capture, you’ll even see the random text we inserted:

% nosetests -w demo4/ -v -stests.test ... 127.0.0.1 - - [15/Mar/2007:19:08:14] "GET / HTTP/1.1" 200 166 "" ""closing...

==> at http://localhost/

Imported extension module 'randomform'.

(at /Users/t/iorich-dev/talk-stuff/cherrypy/demo4/randomform.pyc)

127.0.0.1 - - [15/Mar/2007:19:08:14] "GET / HTTP/1.1

" 200 166 "" ""

closing...

==> at http://localhost/

fuzzfill: widget "inp" using value "0jX0vUXye0"

Note: submit is using submit button: name="None", value=""

127.0.0.1 - - [15/Mar/2007:19:08:14] "POST /form HTTP/1.1

" 200 23 "" ""

ok

closing...

You typed: "0jX0vUXye0"

[15/Mar/2007:19:08:14] ENGINE CherryPy shut down

----------------------------------------------------------------------

Ran 1 test in 0.617s

OK

(Look for the text after « You typed »…)

 

Demo 6: Django fixtures for twill/wsgi_intercept

Purpose: show how to use wsgi_intercept and twill to test a simple Django app.

OK, I’ve shown you how to write automated tests for CherryPy Web apps. Let’s try it out for Django, now!

Since I don’t actually know any Django, let’s just try the Django intro app, a simple poll site that lets users select choices in a poll. The admin interface is a bit tough to test with twill, because it uses JavaScript, but we can test the main poll site easily enough.

Take a look at django/demo/tests.py for some code.

The first function you should look at is actually the last function in the file: TestDjangoPollSite.test. This function goes to « /polls », clicks on the « pycon » choice in the poll, submits it, and verifies that « pycon » has received 1 vote. (Unlike the CherryPy demos, here we’re using the twill Python API, rather than the scripting language.)

Behind this fairly simple looking test function lies two layers of fixtures.

The TestDjangoPollSite.setup() function is run before the test() function, and it serves to reset the vote count in the database; it’s very much like a unittest fixture, in that it’s run prior to each test* function in TestDjangoPollSite. (If there were a teardown() function in the class, it would be run after each test* function.)

The tests.setup() and tests.teardown() serve the same purpose as their CherryPy analogs in Demo 2: setup() initializes Django and sets up the wsgi_intercept shunt mechanism so that twill can talk to the Django app directly through WSGI. In turn, teardown cleans up the WSGI shunt.

Demos 1/2 and Demo 6 collectively demonstrate (hah!) how easy it is to use twill to start testing your Django and CherryPy apps. Even the simple level of testing demonstrated here serves an important purpose: you can be sure that, at a minimum, your application is configured properly and handling basic HTTP traffic. (More complicated tests will depend on your application, of course.)

(Thanks to Mick for his post — I swiped his code!)

 

Demo 7: Recording and examining a Django session with scotch

Purpose: show how to use scotch to record a Django test.

(For this demo, you’re going to need an extra shell window, e.g. an xterm or another ssh session.)

Make sure you have scotch installed, and then run run-recording-proxy. This sets up an HTTP proxy server on port 8000 that records traffic into memory (and saves into a file when done). You should see

** scotch proxy server running on 127.0.0.1 port 8000 ...** RECORDING to filename 'recording.pickle'

OK, now, in another shell, go into django/mysite/ and run python manage.py runserver localhost:8001. This runs the simple Django polling application on port 8001. You should see

Validating models...0 errors found.Django version 0.95.1, using settings 'mysite.settings'Development server is running at http://localhost:8001/

Quit the server with CONTROL-C.

Now go to your Web browser and open the URL http://localhost:8001/polls/. You should see a page with a link containing the link text « what’s up? » This tells you that the Django app is running.

Set your Web browser’s HTTP proxy to ‘localhost’, ‘8000’. Make sure that your proxy settings forward ‘localhost’ (by default, Firefox does not send localhost requests through the proxy mechanism).

All right, now hit reload! If everything is working right, you should see the same « polls » page, but this time you’ll be going through the scotch proxy server. Check out the window in which you ran scotch — it should say something like

REQUEST ==> http://localhost:8001/polls/++ RESPONSE: 200 OK++ (77 bytes of content returned)++ (response is text/html; charset=utf-8)

(# 1)

If so, great! It’s all working! (If not, well… one of us did something wrong ;).

OK, now go back to your Web browser and click through the poll (select « what’s up? », and choose « pycon », and then hit « submit »).

You should see a bunch more output on the proxy screen, including something like this (after the form submit):

REQUEST ==> http://localhost:8001/polls/1/vote/(post form)choice:  "3"++ RESPONSE: 302 FOUND

++ (0 bytes of content returned)

++ (response is text/html; charset=utf-8)

(# 4)

REQUEST ==> http://localhost:8001/polls/1/results/

Already you can see that this is moderately useful for « watching » HTTP sessions, right? (It gets better!)

OK, now hit CTRL-C in the proxy server shell, to cancel. It should say something like « saved 5 records! » These records are saved into the file recording.pickle by default, and you can look at some of the files in the scotch distribution (especially those under the bin/ directory) for some simple ideas of what to do with them.

All right, so you’ve seen that you can record HTTP traffic. But what can you do with the recording?

 

Demo 8: Convert the Django session into a twill script

Purpose: use scotch to generate a twill script from the recording in Demo 7.

Well, one immediately useful thing you can do with the recording is generate a twill script from it! To do that, type

translate-recording-to-twill recording.pickle

in the proxy window. You should get the following output:

# record 0go http://localhost:8001/polls/# record 2#   referer = http://localhost:8001/polls/

go http://localhost:8001/polls/1/

# record 3

#   referer = http://localhost:8001/polls/1/

fv 1 choice '3'

submit

Don’t be shy — save this to a file and run it with twill-sh! It should work.

So that’s pretty convenient, right? It’s not a cure-all — generating tests from recording can get pretty ugly, and with scotch I don’t aim to provide a complete solution, but I do aim to provide you with something you can extend yourself. (There are lots of site-specific issues that make it likely that you’ll need to provide custom translation scripts that understand your URL structure — these aren’t terribly hard to write, but they are site specific.)

 

Demo 9: Replaying the Django session from the recording

Purpose: use scotch to play back the Web traffic directly and compare.

OK, and now for the last demo: the ultimate regression test!

Leave the Django site running (or start it up again) and, in the proxy window, type play-recorded-proxy recording.pickle. This literally replays the recorded session directly to the Django Web app and compares the actual output with the expected output.

You should see something like this:

==> http://localhost:8001/polls/ ...... 200 OK (identical response)==> http://localhost:8001/polls/1 ...

... 301 MOVED PERMANENTLY (identical response)

==> http://localhost:8001/polls/1/ ...

... 200 OK (identical response)

==> http://localhost:8001/polls/1/vote/ ...

... 302 FOUND (identical response)

==> http://localhost:8001/polls/1/results/ ...

... 200 OK

++ OUTPUT DIFFERS

OLD OUTPUT:

====

<h1>what's up?</h1><ul>

<li>the blue sky -- 0 votes</li>

<li>not much -- 0 votes</li>

<li>pycon -- 2 votes</li>

</ul>

====

NEW OUTPUT:

====

<h1>what's up?</h1>

<ul>

<li>the blue sky -- 0 votes</li>

<li>not much -- 0 votes</li>

<li>pycon -- 5 votes</li>

</ul>

====

What’s happening is clear: because we’re not resetting the database to a clean state, the vote counts are being incremented each time we run the recording — after all, in each recording we’re pushing the « submit » button after selecting « pycon ».

Anyway, this is a kind of neat regression test: does your Web site still return the same values it should? Note that it’s very fragile, of course: if your pages have date/time stamps, or other content that changes dynamically due to external conditions, you’re going to have write custom filter routines that ignore that in the comparisons. But it’s at least a neat concept.

(Again, I should note that this is neat, but it’s not clear to me how useful it is. scotch is very much a programmer’s toolkit at the moment, and I’m still feeling my way through its uses. I do have some other ideas that I will reveal by next year’s PyCon…)

 

Conclusions

I hope you enjoyed this romp through a bunch of different testing packages. I find them useful and interesting, and I hope you do, too.

Note that this stuff is my hobby, not my job, and so I tend to develop in response to other people’s requests and neat ideas. Send me suggestions!

Posted in Architecture logicielle, python, tests, Web Frameworks | 1 Comment »