Tuesday, November 30, 2004

Deleting code

Just encountered a nice post titled Deleting code by Ned Batchelder. I mostly share author's attitude toward the subject so there is no point to argue. Just have a small addition regarding the code you wrote as experimentation or in anticipation for something and then discover that YAGNI. In this situation I often found myself not simply delete this code but move it to some temporary file instead.

By convention, this file is often called scratch. or just x. Here are the various code snippets that were purged from the production code or even never managed to get there got accumulated over the time. I found it easier and simpler to look through this file than to rely on SCM.

Saturday, November 27, 2004

Ready? Set. Ship!

For several last months, I've been quitely working on my small sideline project which I'm now ready decided to get out of the door. This is my first take on the entrepreneurial-like venture and I have no idea how and where it would land and whether it would take off at all. But even if it won't succeed I glad I managed to drive the project to the point where it's shipped to the wild and not hung as a vaporware forever.

In spite of a lengthy feature sheet laying in front of me I decided to ship the most basic, bare-bones version that is (potentially) useful to my audience. There is a certain risk that such a primitive version would scare away prospective customers but the risk of developing for too long without being exposed to the real-life feedback is IMO much bigger. If there is any market at all for this kind of service it would shown itself even with such a limited implementation. Or at least, this is what I'm hoping for. ;-)

Oh, I just re-read the post and noticed I didn't even mentioned what the service is. ;-) It's located at the http://postman.com.ua and, specifically for google bots, allows you to send photos, letters and postcard to your relatives, friends and business partners in Ukraine. Actually, you could only send photos for now but the rest will follow (crossed my fingers).

Tuesday, November 23, 2004

Two step view pattern implementation with XSLT

For the project I'm currently working on (in Python/WebKit) I decided to use an XSL technology to handle a presentation issue, instead of a more stock options like Cheetah or PSP. The main reason for this was to escape from using niche solutions and try to employ more mainstreamo ones instead, leveraging benefits that comes from this.

For the presentation logic itself, I used a Two-Step View pattern, publicized by Martin Fowler's excellent book on enterprise patterns. The original implementation actually performed two xsl transformation. First one with a page-specific template, which were passed a page-specific data as XML input string and/or parameters dict. It returned an xml with a "logical markup" which were then passed to another xsl template, called site.xsl which is global per site and transformed this xml-based markup into proper (X)HTML.

It worked OK, except the fact that I had to call to XSL engine twice per page, in order to get a resulting (X)HTML which taken a considerable amount of time (about 0.2-0.4 seconds in total).
But recently, due to a nice tip from 4Suite co-author, I managed to fold these two steps into a single xsl transformation. The trick is a node-set() function which allows you apply the XSL engine to a fragment of output tree already produced.

Here is except from my implementation. The algorithm is as follows:
  1. Build a "logical markup" tree and save it to a variable.
  2. Apply an imported templates to transform this to required output format.

<xsl:import href="site.xsl" />

<xsl:output method="xml"
doctype-public="-//W3C//DTD XHTML 1.0 Strict//EN"

<xsl:template match="/">
<xsl:variable name="view">
<xsl:call-template name="logical" />

<xsl:apply-templates select="exslt:node-set($view)" mode="view"/>

<xsl:template name="logical">
<view:standard-page title="Test page">

<p>This is a test page</p>

And here is a greatly simplified version of the site.xsl page:

<?xml version="1.0" encoding="UTF-8" ?>

<xsl:stylesheet version="1.0"


<xsl:output method="xml" encoding="UTF-8"/>

<xsl:template match="view:standard-page" mode="view">
<title><xsl:value-of select="@title" /></title>

<xsl:apply-templates select="view:content" mode="view" />


<xsl:template match="view:content" mode="view">
<div style="min-height: 400px;">

<xsl:apply-templates mode="view"/>

<xsl:template match="*" mode="view"> <xsl:copy-of select="current()" /></xsl:template>


Being an XSLT newbie myself, the actual code is a bit awkward: I have to use a special namespace (view) and mode parameter of the xsl:template to escape possible infinite loops and to get namespaces right in the output document. Hopefully, an XSLT expert could provide a more streamlined implementation of the same idea.

The node-set() extension function support both by MSXML and 4Suite XSLT which I happened to use on this project. The only minor problem is that you have to declare the namespace a bit differently for these two engines.

My conclusions? I was able to simplify the presentation logic and get a considerable speed up (in a 2x-10x range) with the price of some boilerplate code to add to each xsl page and potential portability problem between other XSL engines. Looks truly like a win to me.

PS: Glad to have a NoPaste service at hand.

Friday, November 12, 2004

using other language to write unit tests

Just encountered an article on IBM's devWorks which gives an example on how to use Groovy to write unit tests for Java code. And this reminded me of experience I once had with mixing languages in unit testing.

In a rather big project I once took part, we used an in-house Tcl-based framework to perform system tests via a network. Some other libraries developed for that project were coded in C++ and Lua. I decided to test this libraries as well using the same Tcl framework I've already built. I used SWIG to make C++ code accessible from within Tcl. I used the same trick to access C API of the Lua interpreter and feed it with Lua code. Just imaging this mess -- evaluate Lua code from within Tcl virtual machine (written in C) via generated dispatching code through the Lua virtual machine and down to C, at last!

It was kind of nice to be able to load a C++ .so library into Tcl shell, experiment with it, call various methods and play with results. But as for production usage, it didn't worked out.
There were two main reasons:
  1. Generation step always was a bit brittle. Even though SWIG managed to handle all C++ constructs we used it required some manual setup and it sometimes was hard to figure out how to use the generated code.
  2. Using different languages (and I'd add, so distinctively different languages) were putting a significant burden on a developer. It just felt awkward.
I don't even mention that instead of developer who is proficient with one language you need a developer who is proficient in both. Plus could master SWIG tool.

Groovy was designed to be as close to Java as possible and it's interoperability with Java is excellent, so unit tests for Java code written in Groovy won't suffer from the same problems I had. Still, I think it's better to stick to a single language for both, no matter how close Groovy is to Java. If you buy arguments for Groovy's simplicity - then just write both production code and unit tests in Groovy. That's my opinion.

profiling unit tests

As were already noted elsewhere, the so-proud-of, "batteries included" Python standard library has a lot to be improved. And while Java tools often being critiqued as overly complex, many of python modules are arguably simplistic, not just "as simple as possible".

I run into this just yesterday, when tried to profile my unit tests.

Usually unit tests run in a fraction of a second. Sometimes though, most often because of some heavy-weight interface not being stubbed, they suddenly could take a several seconds. So I thought to myself: "It would be nice if I could a get report about which test methods ate most of the time".

After struggling for half an hour with a profiler (hotshot or the old one, whatever, they both use pstats module) I give up. Sure enough, I could get an overall program statstics, even filter it by name of the function with a regexp, but there were no way to trace from the actual function (like socket.connect) to a test method that called it (like testReportServerStatus). Or at least, I didn't find it. The latter is probably the case and the reason I was not able to find it is because pstats has a contrived API. I would say, almost a lack of it thereof. When called, it just prints to the screen what's been told but have no way to just return accumulated statistical data lump, so I could play with it by myself.

But then another idea came to my mind. Why do I have to struggle with a profiler in the first place? I already have a framework (unittest, that is) which handles the task of runing the test methods. It should be simple to extend it to allow time measurement and report results, shouldn't it? Well, it turned out more or less simple.

I had to extend both TestSuite class and TestRunner but it did worked. The only major problem I encountered is that TestSuite per se has no idea from which module a particular test has come. Luckily, I was already using my own function to gather test suites so it was simple to establish a module-to-suite mapping. Still, it prints only name of the modules not name of the actual test methods but at least it works. Any idea how to fix this is welcome, though.

Tuesday, November 09, 2004

A perspective on software methodologies

A interesting quote I read the other day by Humphrey, father of the CMM:
What people really want is a high-quality system that implements everything they want, at no cost, right now. Everything else is a trade-off.
What's striking in this metaphor is the idea of compromise. You always have to sacrifice something, one way or another. And some elaborate models, like the iron triangle by Alister Cockburn, were probably inspired by this idea.

I wonder, can we differentiate various software development methodologies by the approach they preach to find this compromise?

Wednesday, November 03, 2004

Programmer's HowTo

Several years ago, when trying to master Linux I had to read a lot of documentations, man pages and HOW-TOs in particular. The topics covered by these manuals/how-tos were rather routine (as it seems to me today) - things like modem setup, font setup, pppd configuration, etc. The distinguishing feature for most of the good ones is that they're tightly packed with useful information - often I had to re-read them in order to grok completely.

Recently I stumbled upon another HOW-TO: How to be a Programmer by Robert L Read. While he addresses a different topic - the topic of a professional growth as a programmer - he managed to preserve the spirit of these HOW-TOs: concise, clear and packed with insights.

That's one of the best texts I had read on a topic in the last few years and arguably the one with the best size/usefulness ratio. It's a pity I haven't encountered it earlier (the first version was released in 2000).