How to write antialiased canvas items

The antialiased rendering back-end for the Gnome canvas is known far and wide for its consistently high frame rate and smooth-as-silk dynamics. This response is largely due to careful programming of the canvas items. It's a bit different than programming normal canvas widgets. The purpose of this document is to give advice on how to get it right.

The interaction loop

There are two main performance goals: consistently high frame rates and low latency. Since the render and repaint time can be longer than the space between events on the input, this is not entirely trivial.

Your application is a process that communicates with the X server through sockets. The communication from the X server is mostly in the form of events, including mouse and keyboard events, as well as expose events and other miscellaneous admin stuff. Generally, this channel is low bandwidth (several K per second max). In the canvas, the communication in the other direction is mostly XShmPutImage requests. This is a low bandwidth channel as well; the high bandwidth part of the interaction is through the shared memory channels. Other Gtk+ widgets may have a higher bandwidth of X drawing primitives.

In general, your application will follow an "interaction loop." It begins with the application taking input events out of its queue, and processing them one at a time. In the case of mouse and keyboard input events, this processing should take a very small amount of time. Generally, the point here is to cause state changes to the items. After all events have been processed, Gtk+ enters the "idle loop," and executes any idle handlers that have been registered. In the case of the canvas, the state changes to the items will cause an "update and repaint" handler to be invoked at this point. This handler first calls the ::update() method of all items that have requested it, then re-renders the repaint area, sending XShmPutImage requests to the X server to repaint this area of the window.

Things are a bit different when it comes to expose events. These are processed immediately, and cause update and repaint even when there are other events waiting in the queue. The rationale here is to do the exposes with absolutely minimum latency. It's also because expose compression is hard to do well. This situation may change in the canvas soon.

This interaction loop works well as long as input event processing is quick. Any time that processing of input events becomes time consuming, there is a risk of a race condition where more input events come in during the processing of existing ones. It's possible to enter a lag swamp in which the queue of events grows longer and longer. This can happen, for example, when expose and mouse events are mixed in the input queue (work it out for yourself).

The thing to keep in mind is that the update and repaint process may cover a number of input events. The longer that re-rendering takes, the more input events. If this processing time is constant for each input event, then you can't win. However, most of the time it's possible to aggregate some of this work, i.e. processing 10 mouse events can be done in considerably less time than ten times the time to process one.

How to accomplish this aggregation is the subject of most of this document. Getting it right depends on the correct use of the ::update () method, which is new to the antialiased back-end in the Gnome canvas.

Rendering

Rendering in the Gnome Canvas is done in phases. The first phase is to determine the repaint area. This is generally done in the input event handlers and in the ::update () method. Then, the canvas decomposes this repaint area into (potentially lots of) small rectangles. For each of these rectangles, the canvas calls the ::render () method of each item whose bbox intersects the rectangle, in bottom-to-top order. The ::render () methods composite the item data over the rectangle. After the entire stack has been rendered, a call to GdkRgb displays the rendered results in the window.

Thus, a canvas item can expect a number of ::render () method calls. The microtile array approach guarantees that the number of ::render () calls is no more than one per 32x32 pixel tile of the repaint area, but this can still be a fairly large number. It's also common for ::render () to be called on items that have not themselves gone through a state change, because they overlap with items that do.

The ::update () method

A quick recap of the interaction loop so far: a number of input events, each of which calls a handler, a single call to the ::update () method of each item that requests it, and a number of calls to the ::render () methods of any items that are in the repaint area.

The ::update () method has no actual responsibilities of its own. In fact, it's quite possible to write canvas items that don't use it at all. It is placed there by the canvas for the convenience of the items.

However, careful use of ::update () is the key to getting top performance from the canvas. It is the only method guaranteed to be called only once per iteration of the interaction loop. Therefore, any computation that's best done once per interaction loop is best placed there.

The main thing that should go in an ::update () method is calculation of the repaint area. The repaint area needs to be computed before any of the rendering can begin, so the only other place is in the input event handlers. However, since repaint area computation can be time consuming, it shouldn't go there. Usually, computing the repaint area for 10 events is not much more work than computing the repaint area for one.

You can also push things both forward from the input handlers and backwards from the renderer. For example, in a drawing program, it may be desirable to queue the mouse events and do the actual drawing in the update method. This doesn't necessarily cut the total amount of time to draw, but does avoid the "lag swamp" (noticeable, by the way, in the current version of Gimp when the "perfect tracking" option is enabled).

Also, many of the antialiased rendering operations in the Gnome canvas are split into two phases - a precomputation phase and a rendering phase. For example, in the vector path renderer, the precomputation phase consists of sorting the vector path into segments monotonic in the y direction, and calculating a bounding box for each one. The rendering step then has the benefit of working with pre-sorted paths, and is also able to do aggressive bounding box culling - especially important when rendering into lots of little rectangles, as is often the case because of the microtiles.

In general, whenever there is a split rendering process like this, you'll want to do the precomputation in the update step, store the precomputed results in the item's private structure, and just do the second phase in the ::render () method.

Computing the repaint area

The canvas has fancy microtile logic for incremental update. However, none of it is of any use if canvas items compute their repaint areas crudely, i.e. by just reporting the bounding box.

The simplest way to compute the delta is to keep the old state, and run a "diff" (or comparison) between the old state and new. If the state changes are queued, then the old state need not be stored permanently in the item's data structure. Rather, one consistent state is stored there, and the update computes a new state by applying the queued state changes to the old one. Then, the diff is computed. Finally, the old state is replaced with the new one.

Other alternatives can work too. You may be able to compute the repaint area directly from the state changes. For example, in a polygon, if only one of the points changes, then the repaint area is the quadrilateral formed by the old and new positions of that point, along with its two immediate neighbors.

Lastly, you might store "changed" flags in the item's data structure. For example, let's say the item is a collection of things that can be toggled on or off. Every time an event handler toggles the thing, it also toggles the thing's changed bit. Then, the ::update () method scans through the changed bits and does a repaint request for each one that's set (and clears all the bits at the end).

Summary

Here is a quick summary:

In the mouse and keyboard event handlers:
Make the state change to the widget (if not time-consuming).
If there's anything time consuming in this handler, think about queueing the event and handling it in ::update ().
Call gnome_canvas_item_request_update ().
In the update method:
Process the queued input events, if any.
Recompute the bounding box if it's changed.
Calculate the minimal update area, then call gnome_canvas_item_request_repaint (), or the _uta version.
Take advantage of the precomputation phase if the rendering routines support it.
In the render method:
Use the second of the two render phases if the rendering routines support it.
levien.com Gnome home