The Crazy Talk Series – An introduction to an alternative web architecture and construction method

If you have met me or worked with me in the last 15 years or so, you will know that I am pretty mad about the web as a platform, HTTP as a protocol and HTML as a state engine. Add to this the last 6 years or so practising agile techniques while building highly scalable websites and you tend to find that I have fairly unusual views in the field about how to build and architect web applications.

A lot of people say to me, Dan, you must write about X as the industry seems to be doing Y and while I find it very easy to talk in a face-to-face conversation or even a mailing list, writing an article or post has always been a challenge due to the one way nature of the conversation. Anyway after some gentle nudging by Sarah Taraporewalla, Christian Blunden and Martin Fowler I have decided to jump in head first and try and create a series of articles that will hopefully open a few peoples eye’s to some alternatives.

So enough with the why and onto the what I’m planning to cover in the series:

  • Caching on the wire not in the app
  • Addressable components
  • Personalisation, cachability and composition
  • Moving state off the web server
  • Web server decoupling, in-memory testing and lightning fast builds
  • Some alternative libraries
  • Common HTTP patterns

Google Chromium Team rocks!

So I noticed an issue with Google Chromium and it not updating the cached entry with new headers on a HTTP 304 response. So I logged the issue last weekend and bang! 6 days later the issue is resolved and I have a new version of the browser on my desktop.

Not only was the process super fast but I watched the fix being discussed then applied. I looked at the code change and the unit tests being updated all as it happened. Needless to same I am very impressed.

Interestingly I noticed the same issue with Opera and logged an issue with the exact same details on the same day, unfortunately Opera’s bug tracking system is private and so I have no idea if the issue is even being looked at. When they do eventually fix the issue I’ll still have to wait for sometime as they don’t have a public build server for me to download and try the fix before an official release.

Using a fullscreen editor to update my blog

So after rediscovering the distraction free editors to write reports, it seemed mad not to use the same environment to update my blog. I really like the fact that the editor only supports plain text and didn’t want to reduce the readability of the plain text by having lots of unneeded markup all over the place.

What I needed was a simple tool to convert my semi structured text into html and then I could just upload that to my blog. Ideally the tool would be a command line tool so I could easily automate the two steps.

I looked at a few different tools but in the end chose to use stx2any as it seemed to be a fairly close fit to the type of plain text I normally use, and also produced the cleanest html.

Then I started to look at tools to upload html into my WordPress blog, I was expecting there to be quite a few but can honestly say I didn’t even find one. I find this a bit wierd as blogs where originally just html and before that text (remember finger). So I started to look at Python scripts that interacted with the MetaWeblogAPI but as my Python is fairly limited I thought I could probably do it quicker in Scala (as thats what I’ve been mainly working in lately).

And so an hour or two later html2blog was born. And here is how I tied it all together:

stx2any --link-abbrevs --make-title off -T html $1 |
 tidy -asxhtml -qc -w 0 | java -jar html2blog.jar

Currently html2blog is very limited in that it always creates a new draft, so the next step will be to make it update an existing entry. I’d also like to get images working at some point.

Fullscreen Editors

So many moons ago I was reading about a fullscreen editor for Mac called WriteRoom. The basic premise is that it is a very limited editor that actively aims to block out all distractions from writing. It achieves this by:

  • Running fullscreen (so no desktop distractions)
  • Supporting plain text only (so no styling distractions)
  • No toolbar
  • No menubar
  • No statusbar
  • A few simple shortcuts to do what you need

Anyway I have been running Ubuntu for a number of years, and had often ran Nano fullscreen in a Gnome Terminal but was curious what might now be available. So a few moments googling and this was the list:

After trying each one, I decided that I actually really liked the q10 version and even though it runs perfectly on Linux via Wine, it just takes way to long to startup (2-3 seconds). Mean while the PyRoom version starts in less than 200ms, so after upgrading to 0.41 I was able to get PyRoom looking very similar to q10 with the following settings:

  • Green screen theme
  • 95% height
  • 62% width
  • Padding 5
  • No border
  • Autosave
  • BitStream Vera Sans font

The only thing I am missing is it remembering what file I was last editing, maybe it’s time I brushed up my Python skills.

Html Contracts – How semantic html can help your cross functional team

One of the pain points we see on web projects is the divide between client side and back end development. This pain might show itself in a number of ways:

  • Small changes in the HTML cause lots of tests to fail
  • Small changes to visual layout require large changes to the HTML which then causes the above
  • Developers say the work is done but it can’t be signed off as it looks terrible or doesn’t work in certain browsers
  • CSS or QAs want developer to add ID attributes to lots of elements so they can target them more easily

Now ideally all your developers should be poly skilled and understand javascript / CSS / HTML just as well as they understand java / C# / ruby but often the reality is not quite so rosy.
So if we are working in a world where we don’t have the ideal but still need to get the job done what can we do to reduce the pain?

Well the technique I have used on a number of teams is to come up with a “HTML contract”. An example might be as follows:

  • Everyone must make the HTML as semantic as possible
  • IDs are deprecated in favor of class attributes. Just use more specific CSS selectors to target elements
  • Developers will liberally add class attributes with semantic names even if they don’t need them immediately (QAs and CSS will use them even if you don’t)
  • CSS / Designers can add class attributes if needed but can not remove class attributes without pairing with a dev/QA.
  • Changing the HTML to support visual display (I’m talking about document order, float and clear) is severely frowned upon. If you have to do it consider doing it with javascript instead.
  • QAs are to use hand written XPath expressions in tests that match the domain and make extensive use of contains(@class, ‘someClassName’) and descendant:: rather than IDs or specifying HTML tags

Some more general tips that I find help:

  • Converting tables to divs is really no better apart from download size. Use divs to group things or as the root element for a control, use lists for lists of things, tables for tabular data etc.
  • Hacking HTML, CSS, Javascript just because you are fed up with IE6 is not acceptable
  • No Cut and Pasting from the web. Or mass import of Javascript / CSS fixes. Don’t put it in if you don’t understand what it does.
  • When you have a CSS issue keep deleting rules until you work out which rule is causing the problem then rebuild the rules up.

I post some examples in my next post

So how about truly private fields in C#?

UPDATE: Jim pointed out that you can access the field via reflecting over the delegate.  (See comment) Damn this is a bit like how java does anonymous access to private fields of the parent class. I wonder if you could use this for some nasty security violations as people tend to think local variables are safe from reflection.

After the crazy !@$%  with JavaScript yesterday I said to Christian, I bet we can do this with C# lambda. So the challenge was set….

class Purse
    public Func<int> get;
    public Action<int> set;

    public Purse(int money)
        get = () => { return money; };
        set = (newMoney) => { money = newMoney ; };

And here is the test …

var p = new Purse(2);
p.set(p.get() + 1);
Assert.AreEqual(3, p.get());

If you tried to use reflection, as expected there is no field to inspect.

Truly private fields in JavaScript

So I was chatting with Christian Blunden about JavaScript, and he asked if it was possible to have private fields in JavaScript.

Now the language doesn’t have a key word but I knew that you could use function scoping to achieve the same affect as I had seen the same thing done using the E programming language.

So after 5 minutes here is what we came up with:

function Purse(money) {
	this.getMoney = function() {
		return money;
	this.setMoney = function(newMoney) {
		money = newMoney;

This will create a truely private field that can only be accessed via the methods.

You can still mix your private getters and setters with prototype methods. eg:

Purse.prototype = {
	add : function( money ) {
		this.setMoney(this.getMoney() + money);

var p = new Purse(2);
p.add( 1 );
p.getMoney(); returns 3

If you tried to access the money field directly it would be undefined.

Pairing with Resharper (Switching Keyboard Layouts in Visual Studio)

So at my current client we have a bunch of Devs that are fairly recent converts to Resharper and a bunch of old hats who know IntelliJ or have used Resharper since Version 1. We are pairing but we know different keyboard layouts and don’t want to decrease productivity by making one lot relearn the other layout. So with some searching around I found out how to create new layouts and using a simple macro switch between them. 

Please see attached zip


An Example Visual Studio Keyboard settings file

Programatically changing the keyboard thread

MSDN Reference for binding keys in Visual Studio


What does it do?

Allow you to switch between “Resharper + Visual Studio” and IntelliJ shortcuts


Why would you do that?

You are pairing with someone who knows the other set of shortcuts


How to install?

  • Close Visual Studio
  • Run install.cmd from a drive (mapped if a network share) to copy the files to the default locations
  • Open Visual Studio
  • Tools – Macros – Load Macros Project… Select the ThoughtWorks folder Select the ThoughtWorks Project
  • Tools –  Options – Environment – Keyboard: Select the Resharper or IntelliJ


How to switch?

  • To switch to IntelliJ Layout press Ctrl-Shift-Alt-I
  • To switch to Resharper 4 Layout press Ctrl-Shift-Alt-R

It still does not work…

  • Tools – Options – Environment – Keyboard: Select the IntelliJ
  • Tools – Import and Export Settings… – Import selected environment settings – No, just import new settings – Browse – Select Switch.Shortcuts
  • Tools – Options – Environment – Keyboard: Select the Resharper
  • Tools – Import and Export Settings… – Import selected environment settings – No, just import new settings – Browse – Select Switch.Shortcuts


Can’t you create a decent installer?

I plan to but it looks like I might need to get my code signed by MS!

Lean and the fishing industry

This could be just verbal diarrhea, but I was watching Autum Watch last night and it struck me that the issues with over fishing and the fishing industry as a whole have a lot to do with all the waste inherent with manufacturing processes that use economies of scale to maximise profits. There is waste all over the industry:

  • The wrong fish get caught and killed then thrown overboard
  • Same again for fish the wrong size
  • Then fish go off or get damaged as they try and increase efficiency by doing bigger catches
  • Then fish don’t get sold because of changes / floods in the market (another trawler comes in at the same time)

I’m sure there are a lot more and I’d love to know the ratio of fish that make it to one’s plate with number caught.  Anyway people tend to think that the only alternative is some tree hugging world where you actually catch the fish yourself.  Toyota have shown that there are alternatives that are highly profitable, consumer driven and extremely effective. So the question is could Lean be applied to the fishing industry?  Could we have a pull system? And what size buffers or queues would we need for it to work?

Dan's technical ramblings