Recent posts

Use Redis as a session backend

Recently I used Redis in one of our projects, while looking for documentation about it I stumbled upon this article about configuring Redis as a LRU cache. I decided to try and write a little session manager for Nagare using Redis as backend.

Requirements

Nagare Installation

Many Nagare mechanisms can be extended through the use of entry points. The Nagare entry point I used is nagare.sessions:

...
[nagare.sessions]
redis = nagare.sessions.redis:Sessions
...

nagare.sessions.redis:Sessions is the path to the Session manager implementation class.

Session Manager API

Redis Session manager has to inherit from nagare.sessions.common.Sessions

from nagare.sessions import common
class Sessions(common.Sessions):
    ...

The following methods must be implemented:

  • Save an application state to your backend:
def serialize(self, data):
    """Serialize application state."""
def _set(self, session_id, state_id, secure_id, use_same_state, session_data, state_data):
    """Store the serialized state and sessions state into our backend"""
  • Retrieve and restore application state from your backend
def deserialize(self, session_data, state_data):
    """Deserialize application serialized state and session"""
def _get(self, session_id, state_id, use_same_state):
    """Retrieve serialized state from our backend"""
  • Create and delete state
def _create(self, session_id, secure_id):
    """Create a new session in our backend"""
def _delete(self, session_id):
    """Delete a session"

The whole code, with handling of locks and other low level stuff can be viewed here.

Use Redis Session Manager

Now, to use the newly created session manager, you have to create a publisher configuration file. The sessions section is configured as this:

...
[sessions]
type = redis
...

Finally I manage to replace seamlessly the standard Nagare session managers by the Redis one I wrote. Hopefully the same technique can be used to try out any other session backend for Nagare.

Edit:

As pointed out by expo on Nagare mailing list version linked in this post does not work with Nagare latest version. The working version he submitted is available.

Profiling the execution time of a Nagare application

Hi,

In this post we'll talk about Nagare profiling. Here is the point: you have an application that's performing well but that's getting slower and slower after adding new features. So it's time for some optimizations!

One strategy would be to put some logs or print into your code to display execution time and find the parts that are too slow. This strategy has some disadvantages:

  • you need to put some debug code all around, launch the app, play the story, watch your logs... and you need to remove all the debug code after the optimization process which can be painful. Repeat this every time you want to optimize your code.
  • you don't really know where the bottleneck is. Code? Database? Filesystem? External system?

Another strategy is to take advantage of a profiling tool. And there's a WSGI middleware out there for that! The benefits are :

  • you can plug/unplug your middleware only by changing your app configuration. So no code change!
  • you can find instantly where is the slowest part of your app!

Great! So let's dive into that latter strategy.

Creating the profiling middleware

Dependencies

Install the library named repoze.profile using easy_install

easy_install repoze.profile

Creating the WSGI pipe

Add the following lines somewhere in your code (ex. app.py)

from repoze.profile import ProfileMiddleware
def profiler(app, options, config_filename, config, error):
    """Creates a WSGI pipe that profiles execution times.
    """
    return ProfileMiddleware(
               app,  # app is your Nagare application (created in the app.py file by default)
               log_filename='/tmp/repoze.profile.log',
               cachegrind_filename='/tmp/repoze.profile.out',
               discard_first_request=True,
               flush_at_shutdown=True,
               path='/__profile__',
               unwind=False,)

Registering the WSGI pipe

Now tell Nagare to wrap the application into the WSGI pipe.

In your application configuration file, simply add the wsgi_pipe directive in the [application] section

[application]
....
wsgi_pipe = myapp.app:profiler

To deactivate profiling, simply comment this line.

Collecting the stats

Launch your app using the standard nagare-admin serve command and play the story you want to optimize.

While browsing into your application, the profiling middleware will automatically gather execution times.

Viewing the stats

By default the profiling middleware will be accessible at http://localhost:8080/myapp/__profile__

The following screenshot shows the result. As you can see, I added an time.sleep() call in my code to simulate a bottleneck.

Further readings

More informations can be found here :

Some recent projects carried out on Nagare

Hi everybody,

A quick recap of the projects we have carried out during the last six months concerning the Nagare Framework.

In the automative training field

On behalf the GNFA, we continued the development of a modular assessment tool. Around a functional basis powered by Nagare, we have implemented different modules dedicated to specific projets for specific use cases.

Among the customers using our software, we can mention: Citroen, Audi, Volkswagen...

Otherwise, this tool is being used in the EVEMA Project, a research project carried out by GNFA and funded by the French Government State.

EVEMA plans to connect a 3D evaluation software with our assessment tool.

We just started a new project INTERLUDE, also funded by the French State, which aims to implement a web portal solution based on OpenSocial standards. We will come back to you regarding this project during the coming months.

On behalf of the PagesJaunes Groupe (The French Yellow Pages)

We carried out interesting projects about the automation of websites design, integrated into the existing production chain. More details to come...

Otherwise, PagesJaunesGroupe provides their employees with a collaborative website called EUREKA to manage collective intelligence. Recently, EUREKA was awarded the "2011 Intranet - Cegos" prize in the Virtual Communities & Knowledge Management category.

We are also working in partnership with start-ups

Modae Technology is an ESL - Electronic System Level - company that offers innovative solutions for the design of hardware-software embedded systems.

This company has developed modeling technology in order to easily capture functional specifications and to automatically generate code for parallel computing systems. We have implemented a web modeling solution with interesting GUIs.

Dolmen Technologies is a company offering a web based solution for collecting prospect data.

An Nagare-based application manages multiple forms, accessible via iPads, available in stores or trade shows. Thanks to this application, the prospect has to enter a correct address, based on a national referential, thus guarantying a reliable database.

Other projects

See you soon for new projects carried out on Nagare!

Nagare Callbacks, Iterations And The Mighty Default Parameter

Problem

Consider the following Python code:

>>> funcs = []
>>> for a in [1,2,3]:
...     funcs.append(lambda:a)
...
>>> print [f() for f in funcs]
[3, 3, 3]

Here, I created three anonymous functions, which return the value of a. Intuitively, we could think that when evaluating these functions, they would return the value a had when it was referenced during the function creation. Instead, all three functions return the last value of a. Why ?

The lambda expression captured a reference to the outside-defined scope that holds the variable a. This reference is used when evaluating the lambda expression. Normally, a wouldn't exist outside the loop, but lambda expressions captured a reference to it. Too bad, it evaluates to the last value it had.

How can we solve this problem ?

In Python, "Default parameter values are evaluated when the function definition is "executed" (see Python reference documentation).

We can use this behaviour to freeze the value of the iteration variable in our lambda expression.

Example:

>>> funcs = []
>>> for a in [1,2,3]:
...     funcs.append(lambda a=a:a)
...
>>> print [f() for f in funcs]
[1, 2, 3]

Application to Nagare

In Nagare, you often create a lambda expression as a callback to an action. For example:

h << h.input(type='submit', value='Cancel').action(lambda: comp.answer())

Here, comp is referenced in the lambda expression. It's ok because comp is not an iteration variable.

But let's take a look at the main view of the "Tic Tac Toe" example:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
@presentation.render_for(TicTacToe)
def default_view(self, h, comp, *args):
    with h.table:
        i = 0
        for row in zip(self._board[::3], self._board[1::3], self._board[2::3]):
            with h.tr:
                for cell in row:
                    with h.td:
                        if cell == 0:
                            h << h.a('-').action(lambda i=i: comp.answer(i))
                        else:
                            h << ('X' if cell==1 else 'O')
                        i += 1
    return h.root

In this view, there is an iteration to create the board, and for each link, we would like the associated callback to answer the value of i, i.e. the id of the clicked cell! To achieve that, you can see that we put a default argument value on the lambda (line 10).

So remember to use the Mighty Default Parameter when defining Callbacks in an Iteration!

An intranet portal for French Yves Rocher’s stores

Yves Rocher French cosmetics company located in over 80 countries, decides in Febuary to launch an Intranet for the 550 stores in France.

Two objectives are identified:

  1. To offer access to a set of features which simplifies the relationship between the group and its franchise network (Tool Control support, registration, reservation of resources, ...)
  2. To offer access to a set of reference content such as news, new products, quality, training, ....

In response to a constraint to put the Intranet on line for July 2010, we chose to conduct a project with agile methods based on Nagare.

The project was implemented in its entirety in six iterations of two weeks. It was an opportunity to develop new features for creating HTML content, navigation of trees and management of access rights for ease of use for end users.

Once again we used the YUI library for the development of the portal presentation layer.

Yves Rocher has also been conducting a security audit on the application implementation to control the privacy of published data and the tightness of the different profiles of users. This audit has highlighted the great qualities of Nagare in terms of securing access to an application built on this framework.

This project, initially for France is expected to manage the problems of multi-lingual, and typology for an international deployment.

Some screenshots :

How to create your own validator

With web forms you always have to validate the data sent by the users of your application. Nagare offers you two simple validators: IntegerValidator and StringValidator. You can find them in the nagare.validator module. The point of this article is to show you how to easily make your own validators.

First, you need to know what an editor property is. A property is normally used in nagare to manage value in forms. When a form is submitted, the properties receive all the values. See the documention CallbacksAndForms#the-editor-property-objects.

Those properties also the offers the validate() method to set a validation function. The nagare documentation CallbacksAndForms#the-validator-module shows you a simple usage of a IntegerValidator

Let's start with a simple example of a StringValidator:

from nagare.validator import StringValidator
from nagare import editor
my_value = editor.Property()
my_value.validate(lambda v:  StringValidator(v).shorter_than(20).not_empty())

Once the property is created, we link a validator to it. Here we want the string to be shorter than 20 characters but not empty.

We can see the validate() method of a property takes a validating function in parameter. This function is called with the value to validate, raises the exception ValueError if the value is invalid or returns the value to be set (i.e a validation function can do conversions too). Then, we can try to put a value in our property:

>>> my_value(u'nagare.org')
u'nagare.org'
>>> my_value.input
u'nagare.org'
>>> my_value.value
u'nagare.org'
>>> my_value.error
>>> my_value(u'')
u''
>>> my_value.input
u''
>>> my_value.value
u'nagare.org'
>>> my_value.error
u'Can't be empty'

Now we know how to use a simple validator, let us see how to make a custom validator. To explain this part we choose to make an EmailValidator:

class EmailValidator(StringValidator):
    def __init__(self, value, **kw):
        super(EmailValidator, self).__init__(value, **kw)
        self.match('[0-9A-Za-z_\.-]+@[0-9A-Za-z_.-]+\.[A-Za-z]{2,4}', msg=u'Invalid email')

The first thing to do is to create a class which inherits from an existing validator: here we choose to inherit from StringValidator. Then, we can add the functionnality we want to offer or overload an existing function. For our example, we overload the __init__() method and we use the match() method inherited from the StringValidator class to check the email on a regular expression. The msg parameter is the message passed if a ValueError exception is raised.

>>> my_value.validate(EmailValidator)
>>> my_value(u'contact@nagare.org')
u'contact@nagare.org'
>>> my_value.input
u'contact@nagare.org'
>>> my_value.value
u'contact@nagare.org'
>>> my_value.error
>>> my_value(u'contact@')
u'contact@'
>>> my_value.input
u'contact@'
>>> my_value.value
u'contact@nagare.org'
>>> my_value.error
u'Invalid email'

For this example we use the match() method but you can also use all the already existing methods. For that, see directly the code of IntegerValidator and Stringvalidator in the nagare.validator module core/nagare/validator.py.

6000!

This impressive number states how many evaluations have been realised since April 2008 by CITROEN employees on a web skill assessment software, developed by Net-ng.

The project is conducted by the GNFA, national association in charge of automotive training.

Implemented in 2008, the application is based on a set of modules, that have been for the most developed under Nagare. The application interacts with business management systems through webservices.

Among the set of modules, the reporting functionality (700 Mb of data for 6 million data units) has been fully developed with Nagare and widely uses YUI library for presentation (dynamic arrays, filters, ...), reportlab - the OpenSource PDF library for dynamic PDF generation with high quality and complex formatting (contents, inserting graphics) - and xlwt to generate multi-tabs Excel workbooks.

Other projects in skill assessment in the automotive sector are in progress ....To be continued!

More information on skills assessment tools developed by Net-ng: here (in french)

Some screenshots:

Project Internationalization with Nagare (message extraction)

Goals

Nagare doesn't have tight integration with gettext or Babel, here we're going to see a way to use those tools in a nagare project.

In the first part, we will integrate Babel into the project, in order to manage catalogs and extract messages. In the second part, we will set up effective message translation inside the project.

Requirements

We need a working Nagare Installation.

We will use Babel to extract the template file and manage our catalog files, so let's install it:

$easy_install Babel
...

Then, we will create and register the example application this way

$ nagare-admin create-app example_i18n
Application 'example_i18n' created.
...
$ cd example_i18n
$ python setup.py develop
...
Finished processing dependencies for example-i18n==0.0.1

Set some defaults for Babel commands

Create a setup.cfg in your newly created app folder containing the following

[extract_messages]
keywords = _ , ugettext , lazy_ugettext:1 , N_:1,2 , ungettext:1,2 , lazy_ungettext:1,2
output_file = data/locale/messages.pot
[init_catalog]
input_file = data/locale/messages.pot
output_dir = data/locale
domain = messages
[update_catalog]
input_file = data/locale/messages.pot
output_dir = data/locale
domain = messages
[compile_catalog]
directory = data/locale
domain = messages

Here, we decide to put translation files into data/locale, but this folder is not created by create-app so we have to create it manually:

$ mkdir data/locale

And finally, we add message extractors information into out setup.py file:

VERSION = '0.0.1'
from setuptools import setup, find_packages
setup(
      name = 'example_i18n',
      version = VERSION,
      author = '',
      author_email = '',
      description = '',
      license = '',
      keywords = '',
      url = '',
      packages = find_packages(),
      include_package_data = True,
      package_data = {'' : ['*.cfg']},
      zip_safe = False,
      install_requires = ('nagare', 'Babel'),
      entry_points = """
      [nagare.applications]
      example_i18n = example_i18n.app:app
      """,
      message_extractors = {'example_i18n': [
                ('**.py', 'python', None)]}
     )
Note:
Babel has been added as a dependency for the project, this way typing 'python setup.py develop' should install Babel if it is missing.

Prepare application for internationalization

Edit example_i18n/app.py, put every strings between _() and define a fake _ function:

from __future__ import with_statement
import os
from nagare import presentation
_ = lambda x: x

class Example_i18n(object):
    pass
@presentation.render_for(Example_i18n)
def render(self, h, *args):
    this_file = __file__
    if this_file.endswith('.pyc'):
        this_file = __file__[:-1]
    models_file = os.path.join(os.path.dirname(__file__), 'models.py')
    h.head.css_url('/static/nagare/application.css')
    h.head << h.head.title(_('Up and Running !'))
    with h.div(class_='mybody'):
        with h.div(id='myheader'):
            h << h.a(h.img(src='/static/nagare/img/logo.gif'), id='logo', href='http://www.nagare.org/', title='Nagare home')
            h << h.span(_('Congratulations !'), id='title')
        with h.div(id='main'):
            h << h.h1(_('Your application is running'))
            with h.p:
                h << _('You can now:')
                with h.ul:
                    h << h.li(_('If your application uses a database, add your database entities into '), h.i(models_file))
                    h << h.li(_('Add your application components into '), h.i(this_file), _(' or create new files'))
            h << h.p(_('To learn more, go to the '), h.a(_('official website'), href='http://www.nagare.org/'))
            h << _("Have fun !")
    h << h.div(class_='footer')
    return h.root
# ---------------------------------------------------------------
app = Example_i18n

Message extraction and catalog manipulation

Messages can be extracted:

$ python setup.py extract_messages
running extract_messages
extracting messages from example_i18n/__init__.py
extracting messages from example_i18n/app.py
extracting messages from example_i18n/models.py
writing PO template file to data/locale/messages.pot

A messages.pot file is now created into data/locale

For the next step, we have to initialize a catalog for your preferred locale (fr in this example):

$ python setup.py init_catalog -l fr
running init_catalog
creating catalog 'data/locale/fr/LC_MESSAGES/messages.po' based on 'data/locale/messages.pot'

The catalog is now added and is ready to be edited.

Once edited and filled in with the right translations, we can compile our catalog:

$ python setup.py compile_catalog
running compile_catalog
compiling catalog 'data/locale/fr/LC_MESSAGES/messages.po' to 'data/locale/fr/LC_MESSAGES/messages.mo'

This way we can prepare as many catalogs as locales needed.

Note:
when changing messages in a project we have to do a extract_messages followed by a update_catalog, and add missing translation in every modified catalog.

Next Step

We are using a fake _ function, so nothing is translated at all, the next step is to plug gettext functions into example_i18n application, in order to get effective message translation.

A simple tabbed navigation with Nagare

Tabs are a widely used mean to navigate between categories or sections of a website. There are lots of howtos and tutorials to create clean tab based navigation with HTML and CSS.

Let's see how to handle such a navigation with Nagare. We are going to create a tabbed navigation between a TicTacToe and a Counter.

Requirements

We use TicTacToe and Counter examples from our Demonstration Portal. Hence nagare examples have to be installed with following command:

easy_install nagare.examples

To run our small example we use serve-module:

nagare-admin serve-module ./menu_example.py:TabNav menu_example

Create Menu component

The Menu has to meet the following needs:

  1. store the selected entry
  2. display the list of clickable entries
  3. notify when an entry is clicked

A fairly simple implementation in Nagare could be:

from nagare import presentation, var
class Menu:
    def __init__(self, menu):
        self.menu = menu
        self.selected = var.Var(0)
    def display(self, i, comp):
        comp.answer(i)

@presentation.render_for(Menu)
def render(self, h, comp, *args):
    with h.ul(class_='menu'):
        for index, entry in enumerate(self.menu):
            h << h.li(h.a(entry).action(lambda index=index: self.display(index, comp)),
                      class_='selected-entry' if self.selected() == index else '')
    return h.root

Notification of the clicked entry is done through the use of answer() method of the Menu component.

Put it together in TabNav component

TabNav is composed of two parts, menu and content. TabNav is expected to work the following way:

  1. When an entry is clicked in menu, select_tab is called
  2. select_tab change the displayed content and then sets the selected entry in menu
from nagare.examples import counter, tictactoe
from nagare import presentation, component
class TabNav:
    tab_factories = [('Counter', counter.Counter1),
                     ('TicTacToe', tictactoe.Task)]
    def __init__(self):
        self.menu = component.Component(Menu([label for label, factory in self.tab_factories]))
        self.menu.on_answer(self.select_tab)

        self.content = component.Component(None)
        self.select_tab(0)
    def select_tab(self, index):
        self.content.becomes(self.tab_factories[index][1]())
        self.menu().selected(index)
@presentation.render_for(TabNav)
def render(self, h, *args):
    h << self.menu
    with h.div(class_='content'):
        h << self.content
    return h.root

on_answer() method of the Menu component let us bind answer() method of the Menu component with select_tab() method of TabNav component. This way select_tab is called with the index of the clicked entry in the menu as parameter.

Some styling

Finally a bit of CSS can be added to get the tabbed rendering effect.

@presentation.render_for(TabNav)
def render(self, h, *args):
    css = """
body {
    font-family: verdana,'Bitstream Vera Sans',helvetica,arial,sans-serif;
}
ul.menu {
    height: 2em;
    list-style: none;
    margin: 0;
    padding: 0;
}
ul.menu li {
    float: left;
    margin: 0 1px 0 0;
    padding-left: 10px;
    border: 1px solid #555555;
    border-bottom: none;
    color: #555555;
}
ul.menu a {
    color: #555555;
    display: block;
    float: left;
    height: 2em;
    line-height: 2em;
    padding-right: 10px;
    text-decoration: none;
}
ul.menu li.selected-entry {
    background-color: #555555;
}
ul.menu li.selected-entry a {
    color: #FFFFFF;
    font-weight: bold;
}
div.content {
    border: #555555 solid 2px;
    clear: left;
    padding: 1em;
}
"""
    h << h.style(css, type="text/css")
    h << self.menu
    with h.div(class_='content'):
        h << self.content
    return h.root

This renders as following:

/trac/raw-attachment/blog/tabbed-navigation-with-nagare/tabbed_navigation.2.png
Rq:
As we use serve-module, CSS has to be embedded into HTML but it is usually put in the static folder of the application skeleton.

Nagare at PyCon 2010

Net-ng attended the recent PyCon in Atlanta, Georgia, US (February 17-25 2010). PyCon is the largest (Over 1,000 Python programmers) worldwide annuel conference for the community using and developing the open-source Python programming language. It mixed three events such as Tutorials, Conference days (talks, open space sessions, lightning talks, poster session and expo hall) and development sprints.

As a sponsor of this event, next to Google, Oracle, WaltDisney, Microsoft or CCP Games, Net-ng was promoting Nagare at the expo hall (see picture below). We really enjoyed deep discussions with python web developers, who understood the added-values Nagare brings to their web applications developments (A new way to build web applications). This was an extraordinary opportunity for us to present the latest Nagare release (0.3) and its brand new dedicated web IDE.

More about Nagare http://www.nagare.org

More about PyCon http://us.pycon.org/2010/about/

More about Python http://python.org

SQLAlchemy's MapperExtension

Problem

You use SQLAlchemy and you want to compute some models' attributes just before they are flushed to the database by SQLAlchemy (an SQL trigger, but on Python's side).

Solution

SqlAlchemy provides the MapperExtension interface with several methods to be implemented (eg.: before_insert, before_update, after_insert, after_update). You can see a full list of possible hooks in SqlAlchemy's documentation.

Let's see how to always capitalize the name of our users:

class User(object):
    def __init__(self, name):
        self.name = name
from sqlalchemy import Table, Column, Integer, Text
users_table = Table('users',
                     Column('id', Integer, primary_key=True),
                     Column('name', Text))

We want to modify the name before every SQL insert and every SQL update statement. The corresponding MapperExtension could be:

from sqlalchemy.orm import MapperExtension
class CapitalizeNameMapperExtension(MapperExtension):
    def _capitalize_name(self, instance):
        if instance.name is not None:
            instance.name = instance.name.capitalize()
    def before_insert(self, mapper, connection, instance):
        self._capitalize_name(instance)
    def before_update(self, mapper, connection, instance):
        self._capitalize_name(instance)

We can now map the model and the table using this extension:

from sqlalchemy.orm import mapper
user_mapper = mapper(User, users_table,
                     extension=CapitalizeNameMapperExtension())

So now, let's see it in action:

>>> user1 = User("mynameis")
>>> print user1.name
mynameis
>>> from nagare import database
>>> database.session.add(user1)
>>> print user1.name
mynameis
>>> database.session.flush()
>>> print user1.name
Mynameis

Note: this works on column-based attributes. At the call of the MapperExtension's functions, the mapper has already made the graph of objects to be updated in the database. This implies that if you make any change on collection attribute, it will not be stored in the database before the next flush call.

Use Nagare namespaces to create an rss feed

In Nagare HTML and XML generation is done through Renderers. Thus, to generate RSS feeds it is possible to use Nagare XML Renderer, which accept any tag or attribute. Another way is to create a RSS Renderer for Nagare which matches RSS namespace.

To do so we are extending Nagare XmlRender, which enforces possible xml tags and attributes (e.g. namespace).

Simple RSSRenderer

RSS specification, leads to the following namespace definition in Nagare:

class RssRenderer(xml.XmlRenderer):
    """RSS renderer"""
    # RSS tags
    # ------------
    rss = TagProp('rss', set(('version', )))
    channel = TagProp('channel', set())
    title = TagProp('title', set())
    link = TagProp('link', set())
    description = TagProp('description', set())
    language = TagProp('language', set())
    copyright = TagProp('copyright', set())
    managingEditor = TagProp('managingEditor', set())
    webMaster = TagProp('webMaster', set())
    pubDate = TagProp('pubDate', set())
    lastBuildDate = TagProp('lastBuildDate', set())
    category = TagProp('category', set())
    generator = TagProp('generator', set())
    docs = TagProp('docs', set())
    cloud = TagProp('cloud', set(('domain', 'port',
        'path', 'registerProcedure', 'protocol')))
    ttl = TagProp('ttl', set())
    image = TagProp('image', set())
    url = TagProp('url', set())
    width = TagProp('width', set())
    height = TagProp('height', set())
    rating = TagProp('rating', set())
    textInput = TagProp('textInput', set())
    name = TagProp('name', set())
    skipHours = TagProp('skipHours', set())
    skipDays = TagProp('skipDays', set())
    item = TagProp('item', set())
    title = TagProp('title', set())
    link = TagProp('link', set())
    description = TagProp('description', set())
    author = TagProp('author', set())
    category = TagProp('category', set(('domain', )))
    comments = TagProp('comments', set())
    enclosure = TagProp('enclosure', set(('url', 'length', 'type')))
    guid = TagProp('guid', set(('isPermaLink', )))
    pubDate = TagProp('pubDate', set())
    source = TagProp('source', set(('url', )))

Note

Notes:

  • RssRenderer checks if tags and attibutes exist in namespace
  • RssRenderer does not validate against any DTD or schema

Let's look at the following example:

<?xml version="1.0"?>
<rss version="2.0">
    <channel>
        <title>Some Blog Title</title>
        <link>http://someblog.example.com</link>
        <description>Description</description>
        <language>en-us</language>
        <pubDate>Tue, 25 Jan 2010 14:00:00 GMT</pubDate>
        <lastBuildDate>Tue, 25 Jan 2010 14:00:00 GMT</lastBuildDate>
        <docs>http://blogs.law.harvard.edu/tech/rss</docs>
        <generator>Some Generator</generator>
        <managingEditor>editor@example.com</managingEditor>
        <webMaster>webmaster@example.com</webMaster>
        <item>
            <title>Some Post Title</title>
            <link>http://someblog.example.com/post1</link>
            <description>Some Post Description</description>
            <pubDate>Tue, 25 Jan 2010 14:00:00 GMT</pubDate>
            <guid>Some Global Unique Identifier</guid>
        </item>
    </channel>
</rss>

With RssRenderer, this feed is generated this way:

r = rss.RssRenderer()
with r.rss(version="2.0"):
    with r.channel:
        r << r.title('Some Blog Title')
        r << r.link('http://someblog.example.com')
        r << r.description('Description')
        r << r.language('en-us')
        r << r.pubDate('Tue, 25 Jan 2010 14:00:00 GMT')
        r << r.lastBuildDate('Tue, 25 Jan 2010 14:00:00 GMT')
        r << r.docs('http://blogs.law.harvard.edu/tech/rss')
        r << r.generator('Some Generator')
        r << r.managingEditor('editor@example.com')
        r << r.webMaster('webmaster@example.com')
        with r.item:
            r << r.title('Some Post Title')
            r << r.link('http://someblog.example.com/post1')
            r << r.description('Some Post Description')
            r << r.pubDate('Tue, 25 Jan 2010 14:00:00 GMT')
            r << r.guid('Some Global Unique Identifier')

Now RSSRenderer can be used to generate any RSS feed.

Mixing multiple namespaces

There's several extensions to RSS, actually extensions are just XML namespaces added in the feed. One simple extension is blogChannel namespace.

First create a BlogChannelRenderer:

class BlogChannelRenderer(xml.XmlRenderer):
    blogRoll = TagProp('blogRoll', set())
    mySubscriptions = TagProp('mySubscriptions', set())
    blink = TagProp('blink', set())
    changes = TagProp('changes', set())

Now we can mix RssRenderer and BlogChannelRenderer to create one feed:

r = rss.RssRenderer()
r.namespaces = {'blogChannel': 'http://backend.userland.com/blogChannelModule'}
b = rss.BlogChannelRenderer(r)
b.default_namespace = 'blogChannel'
with r.rss(version="2.0"):
    with r.channel:
        r << r.title('Some Blog Title')
        r << r.link('http://someblog.example.com')
        r << r.description('Description')
        r << r.language('en-us')
        r << b.blogRoll('http://someblog.example.com/blogroll')
        r << b.mySubscriptions('http://someblog.example.com/subscriptions')
        r << b.blink('http://someblog.example.com/backlink')
        r << b.changes('http://someblog.example.com/changes')
        r << r.pubDate('Tue, 25 Jan 2010 14:00:00 GMT')
        r << r.lastBuildDate('Tue, 25 Jan 2010 14:00:00 GMT')
        r << r.docs('http://blogs.law.harvard.edu/tech/rss')
        r << r.generator('Some Generator')
        r << r.managingEditor('editor@example.com')
        r << r.webMaster('webmaster@example.com')
        with r.item:
            r << r.title('Some Post Title')
            r << r.link('http://someblog.example.com/post1')
            r << r.description('Some Post Description')
            r << r.pubDate('Tue, 25 Jan 2010 14:00:00 GMT')
            r << r.guid('Some Global Unique Identifier')

This produces the following RSS feed:

<rss xmlns:blogChannel="http://backend.userland.com/blogChannelModule" version="2.0">
  <channel>
    <title>Some Blog Title</title>
    <link>http://someblog.example.com</link>
    <description>Description</description>
    <language>en-us</language>
    <blogChannel:blogRoll>http://someblog.example.com/blogroll</blogChannel:blogRoll>
    <blogChannel:mySubscriptions>http://someblog.example.com/subscriptions</blogChannel:mySubscriptions>
    <blogChannel:blink>http://someblog.example.com/backlink</blogChannel:blink>
    <blogChannel:changes>http://someblog.example.com/changes</blogChannel:changes>
    <pubDate>Tue, 25 Jan 2010 14:00:00 GMT</pubDate>
    <lastBuildDate>Tue, 25 Jan 2010 14:00:00 GMT</lastBuildDate>
    <docs>http://blogs.law.harvard.edu/tech/rss</docs>
    <generator>Some Generator</generator>
    <managingEditor>editor@example.com</managingEditor>
    <webMaster>webmaster@example.com</webMaster>
    <item>
      <title>Some Post Title</title>
      <link>http://someblog.example.com/post1</link>
      <description>Some Post Description</description>
      <pubDate>Tue, 25 Jan 2010 14:00:00 GMT</pubDate>
      <guid>Some Global Unique Identifier</guid>
    </item>
  </channel>
</rss>

With this process any XML namespace can be added to Nagare. For example, Nagare XHTML namespace is implemented as a Renderer.

Nagare installation on Mac OS X (Snow Leopard)

Snow Leopard has been released this summer.

The 64-bits Mac OS X version is installed by default on new Mac. Currently, the use of Nagare with the 64-bits stackless Mac OS X version leads to a Bus Error .

To get around this, we need to install a 32-bits stackless python, thus changing slightly the install procedure for Mac OS X described previously .

Install MacPorts

Install instructions here.

Configure MacPorts

MacPorts has to use universal variant (64 + 32 bits compilation).

In order to do so, you add +universal into variants.conf file, i.e. for default macports installation:

echo +universal | sudo tee -a /opt/local/etc/macports/variants.conf

Install Stackless

Now we have to run configure for 32-bits architecture:

cd /tmp
wget http://www.stackless.com/binaries/stackless-264-export.tar.bz2
tar jxvf stackless-264-export.tar.bz2
cd stackless-2.6.4*
CC="gcc -arch i386" ./configure --enable-stacklessfewerregisters --prefix=<STACKLESS_HOME>
make all
sudo make install

Once Stackless Python installed, Nagare installation goes on as usual.

Installing apache2 mod_fastcgi

Proper production configuration with Apache2 web server of a Nagare application involves using mod_fastcgi.

Here is a brief mod_fastcgi installation howto based on a Redhat system. This should easily be ported to any other nix plateform.

First of all, you'll need some basic development tools like gcc, gcc-c++, make , autoconf, automake ...

Then you should check you've required headers packages for Apache2 installed:

yum install httpd-devel libtools (for RHEL, centos, fedora based system)
up2date install httpd-devel libtools (for older RHEL systems)

OK, now let's install mod_fastcgi:

wget http://www.fastcgi.com/dist/mod_fastcgi-2.4.6.tar.gz
tar xzvf mod_fastcgi-2.4.6.tar.gz
cd mod_fastcgi-2.4.6
cp Makefile.AP2 Makefile
make top_dir=/usr/lib/httpd
make top_dir=/usr/lib/httpd install

Done

SAVEPOINT transactions

The problem

In a function, you want to perform some operations on a database using sqlalchemy's session. If one of the operations fails, you want the session to go back to its state when the function was called.

The solution

Use a SAVEPOINT transaction:

from nagare import database
def my_func():
    try:
        with database.session.begin(nested=True):
            # let the job be done in a SAVEPOINT transaction
            # if an exception is raised in this block, session's state is restored
            ...
        return True
    except:
        return False

Nagare installation on Mac OS X (Leopard)

As Mac OS X is a unix based system, the Linux section of NagareInstallation#linux-installation applies, but a few tweaks are needed to setup a fully working nagare environment.

We'll need the following external tools:

Install Xcode

Xcode should be available from your Applications Install DVD or can be downloaded from here.

Install MacPorts

Install instructions can be found here.

Install Stackless

Here we have to give some specific options to configure:

cd /tmp
wget http://www.stackless.com/binaries/stackless-252-export.tar.bz2
tar jxvf stackless-252-export.tar.bz2
cd stackless-2.5.2*
./configure --enable-stacklessfewerregisters --enable-universalsdk --prefix=<STACKLESS_HOME>
make all
sudo make install

--enable-stacklessfewerregisters and --enable-universalsdk are the key options here. Also note how Stackless Python is installed in its own <STACKLESS_HOME> directory, in order not to override the Python system.

Install libxml2 and libxslt

As the libxml2 version given with Xcode prevents lxml from installing, we'll use MacPorts to install the right one:

sudo port install libxslt

Finish nagare installation

From now on, you can follow the vanilla install guide, starting with section 2.

Dynamic content from actions

In the previous post (blog:dynamic_images) we saw how to use the action() method of the img tag to put dynamic images in a view.

Let's see now how to serve any other dynamic content by raising a HTTPOk exception:

import time
from webob.exc import HTTPOk
class MyApp(object):
    def get_text(self, *args):
        data = time.asctime()
        e = HTTPOk()
        e.body = data
        e.content_type = 'text/plain'
        raise e
from nagare import presentation
@presentation.render_for(MyApp)
def render(self, h, *args):
    return h.a('Click me').action(self.get_text)

Nagare catches such webob.exc exceptions; as they inherit from webob.Response they are immediatly returned in the WSGI pipe.

Dynamic images in your component views

Dynamic images are often used in web apps, for example to create charts from database data. You create your image in a function and then return the content of the image, with the appropriate response headers.

In many frameworks, you must create a method, bind it to an URL and include this URL in the src property of an img tag. In Nagare you directly associate the method to the img tag.

In this example I've taken the Nagare logo and wrote the current timestamp on it using PIL.

Here's the class definition:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
import cStringIO
import time
import pkg_resources
from PIL import Image, ImageDraw
from nagare import presentation
class MyApp(object):
    def get_image(self, h):
        """Returns the modified Nagare logo
        In:
          - ``h`` -- the renderer (not used)
        """
        # Get the Nagare logo
        requirement = pkg_resources.Requirement.parse('nagare')
        img = pkg_resources.resource_stream(requirement, '/static/img/logo.gif')
        # Open the image
        im = Image.open(img)
        draw = ImageDraw.Draw(im)
        # Draw timestamp on it
        draw.text((15, 10), time.asctime(), fill=0xffffff)
        # Save it into a StringIO and return the content
        imgdata = cStringIO.StringIO()
        im.save(imgdata, 'PNG')
        return imgdata.getvalue()

Nothing special here. The interesting part is the rendering method for this component:

1
2
3
 @presentation.render_for(MyApp)
 def render(self, h, *args):
     return h.img.action(self.get_image)

You may have already used the action() method on HTML tags such as a or input. The img tags also have one.

All you have to do is to give to action() a callback, that will take a renderer as argument and will have to return the image data. So I passed to action() the get_image() method of MyApp class. As the function returns the image stream, without explicitly setting the content-type of the HTTP response, Nagare auto discovers the content type and set it by itself.

Common HTML structuration for different components

The problem

We want to obtain the same HTML structure when rendering different components.

Templating using inheritance and named views

The principle is the same than the template method design pattern but applied to components views.

Setting the base class

Let's declare an empty base class:

class BaseEntry(object):
    pass

Its default view sets the common HTML structure and embeds the title and properties named views:

from nagare.presentation import render_for
@render_for(BaseEntry)
def render_base_entry(self, h, comp, *args):
    """All the views of entries will have this HTML structure"""
    with h.div(class_='entry'):
        with h.div(class_='title'):
            h << comp.render(h, model='title')
        with h.div(class_='properties'):
            h << comp.render(h, model='properties')
    return h.root

The BaseEntry's title and properties views are empty. They will be redefined for the derived classes:

@render_for(BaseEntry, model='title')
def render_base_entry__header(self, h, *args):
    return h.root
@render_for(BaseEntry, model='properties')
def render_base_entry__properties(self, h, *args):
    return h.root

We can now verify that the basic HTML structure is generated:

>>> from nagare import component
>>> from nagare.namespaces import xhtml
>>>
>>> entry = component.Component(BaseEntry())
>>> h = xhtml.Renderer()
>>> print entry.render(h).write_htmlstring(pretty_print=True)
<div class="entry">
<div class="title"></div>
<div class="properties"></div>
</div>

Using the base class

Now, let's create a derived class:

class SimpleEntry(BaseEntry):
    def __init__(self, name, firstname, age):
        self.name = name
        self.firstname = firstname
        self.age = age

We only have to define its title and properties views:

@render_for(SimpleEntry, model='title')
def render_simple_entry__title(self, h, *args):
    return '%s %s' % (self.firstname, self.name)
@render_for(SimpleEntry, model='properties')
def render_simple_entry__properties(self, h, *args):
    return h.div('Age: %d' % self.age)

for the component data to be correctly inserted into the right HTML structure:

>>> from nagare import component
>>> from nagare.namespaces import xhtml
>>>
>>> entry = component.Component(SimpleEntry('John', 'Doe', 20))
>>> h = xhtml.Renderer()
>>> print entry.render(h).write_htmlstring(pretty_print=True)
<div class="entry">
<div class="title">Doe John</div>
<div class="properties"><div>Age: 20</div></div>
</div>

Templating using aggregation and named views

When inheritance is not wanted, we can define a component whose job is to do the rendering:

class EntryTemplate(object):
    def __init__(self, entry):
        self.entry = entry
@render_for(EntryTemplate)
def render_entry_template(self, h, *args):
    """
    Same code that we used in ``render_BaseEntry``
    but using ``self.entry`` instead of ``self``.
    """
    with h.div(class_='entry'):
        with h.div(class_='title'):
            h << self.entry.render(h, model='title')
        with h.div(class_='properties'):
            h << self.entry.render(h, model='properties')
    return h.root

Let's use it with our SimpleEntry class:

class SimpleEntry(object): # no inheritance anymore
    def __init__(self, name, firstname, age):
        self.name = name
        self.firstname = firstname
        self.age = age

The title and properties views are still the same, but we must now declare a default view to wrap our SimpleEntry component into a EntryTemplate component and render it:

from nagare.component import Component
@render_for(SimpleEntry)
def render_simple_entry(self, h, comp, *args):
    return Component(EntryTemplate(comp)).render(h)

Then we can use the SimpleEntry components exactly like in the previous chapter:

>>> from nagare import component
>>> from nagare.namespaces import xhtml
>>>
>>> entry = component.Component(SimpleEntry('John', 'Doe', 20))
>>> h = xhtml.Renderer()
>>> print entry.render(h).write_htmlstring(pretty_print=True)
<div class="entry">
<div class="title">Doe John</div>
<div class="properties"><div>Age: 20</div></div>
</div>

Welcome !

Welcome to the blogs about the Nagare web framework and applications.

Be our guest,

The Nagare bloggers