creative endeavors

It recently occurred to me that all of my life can be broken up in moments of consumption, moments of creation, and moments of doing just plain nothing. I think that usually people create and produce for work and consume and relax for fun.

What's special about me is that I need to create to really feel alive, that I have a small limit of how much I can consume. I shop only for necessities; I don't watch TV, rarely read novels and watch only hand-picked movies. Most of my information-consumption is studying specific subjects (favorite reading source: Wikipedia). This often inspires my own creations. Well, mostly it rather inspires ideas, most of which I'll never execute.

It's really hard to relax without consuming anything. Sometimes (especially at night) I eat although I'm not very hungry; just for relaxing. Sometimes I read stuff (especially links on Facebook) that I won't even remember a second later, that's more relaxation than consumption. Doing sports or taking a walk seem to be the least consumtive ways to relax. Cooking can be a good creative/productive way to relax, but it requires that I am not too exhausted in the first place. Talking to a close friend is also a great way to relax and to clear my mind of a lot of things going around in there.

Here are some ways in which I could be creative for my own pleasure:

• programming stuff
• making music
• drawing (architecture and inventions)
• inventing stuff
• building bicycles
• 3D modeling stuff (again, architecture and inventions)
• writing movie scripts
• writing religious books
• writing technical books 
• doing stuff with languages

Last year, I took a basic class in hand drumming (Djembe playing) and that was fun (I also met great people there), but I didn't find it creative enuf. I thought that after having a basic sense of rhythm, I could experiment with tunes. A very kind person gave me a guitar, but I soon found that the level of practice required before any creativity can take place is way more that I'm willing to invest. In any case I feel that I now have made up for the lack of "trying out music" in my childhood and can move on to other topics now.

Since the first anniversary of me being in Berlin which coincided with my first paid vacation since graduating from UofT I have been thinking what I want to do with the next year of my life. But since all my vacation was filled with busy trips, I didn't really have time to think about it. At least I took note of the past year's highlights and lowlights and then came this week: spontaneous vacation without a planned trip! In the first days I did some remaining home improvements following my move (also this summer after just a little more than a year in my first place in Berlin). But then I quickly started looking for long-term projects in which to put my precious vacation time.

The result of my deliberations is that I want to start both drawing by hand and 3D modeling with SketchUp (see previous post). I have also decided that the realm of Architecture shall stay a topic of consumption, while my first drawings and 3D models shall be related to the planetary gear adventures. I would also really like to finish a first (or rather zeroth) version of my planetary gear calculator program, but since I find programming so hard, I am shying away from any further work until I find myself with a really large block of uninterrupted-by-work time to concentrate on this.

In the language department, I have decided to learn Dutch, just because it is so easy. In the consumption and inspiration department, I am planning a trip to Asia next fall, but I am uncertain, if I will use the chance to study more of the Chinese language. Practice a bit, certainly yes, but learning new words, it's oh-so-hard...

The future of Wikipedia and the next year for myself

It's been a couple of years that Wikipedia remains one of the world's most-read websites, but stagnates in growth and doesn't offer any really new and exciting features. One of things that I had expected to boost Wikipedia's evolution was a more easy-to-use (read: WYSIWYG) editor and more support for writing articles on specialized subjects. Writing an article about a movie or about a chemical compound or a person's biography are all fundamentally different things and although Wikipedia has subcultures for many specialized domains (and portals and mentors) the software and interface for all those domains remains the same. Since all the knowledge is in the community, not the software, Wikipedia is at a disadvantage compared to other places on the web.

As an aside compare how Facebook encodes their domain knowledge in software: there's only a limited number of romantic relationship types that one can choose and one can only select one single to be in a relationship with. That's a really easy user interface! (Of course, it discriminates against polyamorous people, but Wikipedia would of course offer such settings in a biography interface. ;-) 


A good example for a knowledge-sharing website with goals that overlap Wikipedia's is stackexchange.com which combines Q&A with Wikis and uses many encoded-in-software practices to shepherd its users into creating an ever-better site and ever-better community. Another example would be the Google Places semi-wiki which offer's shop addresses, opening hours, and reviews directly integrated into Google Maps.

But the example that currently takes most of my attention is Google SketchUp with its 3D Warehouse of reusable models. When I started dreaming to redesign some of my favorite subway stations on the computer, I just has wished that such a thing existed. Well, in fact, it existed already, I just didn't know about it. Well, now all I need is to install Windows on my computer (so I can run SketchUp) and also buy a computer mouse.

I just hope that I will have enuf time and calm (meaning not be too exhausted from work) to take up this new hobby!

PS: What I want is to create 3D models as complete as this drawing from London, but for stations in Berlin in Paris.

For the interested: More from London. And even more.

Here's a nice picture from Paris, too:

(c) by Corbis

Feature-driven, design-guided, and tests-in.

I just typed an answer to a new comment on my last post. Apparently it became to long to be submitted as a comment itself, so here I am turning it into a new post. I starts out with some "loud thinking" but ends with some nice insights.

Hi James, thanks a lot for your comment. You're asking just the right questions and those questions help me see more clearly, what "my problem" with TDD is.

Now, ten days after I wrote that post and through your comment, I realize that there's actually a big gap in how TDD is summarized (especially Uncle Bob's version with the three laws) and in how TDD is actually successfully practiced. I find that when the three rules are taken literally (we tried that in some Dojos), then the development and actual design becomes cluttered with detail of each test case and it's less feature-driven as well as less pattern-guided than I would like. On the other hand, if I'm looking at successful agile development with lots of unit tests, then the three rules are just not visible in the process.
I think that maybe two social processes are at work here: on the one hand, good practices spread thru pair programming and people reading a lot of open-source code, but those practices often don't have catchy names. On the other hand, there's a very catchy concept called TDD and very simple "three rules" and people saying that just by following those rules and refactoring, everything else will follow. For example, some people say that good design automatically follows from testability because only loosely coupled systems are easily testable.

So, the reason I wrote this blog post is that the simple, catchy way, TDD is explained, just won't work. It's also just not true that TDD gives you an easy way to tell when you're done. I am currently working on a medium-complex system (roughly developed by a four-person team over two years) with high unit and integration test coverage and we repeatedly had incidents just because we forgot to add something here or there which didn't get caught by the tests. However, our code is simple enuf that those missing parts would become obvious if we just had a final code review after every iteration where we check all production and test code against (a longish) list of the specific level of done for the project. (Which includes error handling, logging, monitoring, etc.) That review is what we now regularly do. Sometimes we find missing things in tests, sometimes we find them in the source, in each case it's easily fixed before going live. So the seemingly obvious things like "TDD always gives you 100% coverage" or "with TDD you always know when you're done" are just not relevant in practice.

My conclusion after working in a "high unit-test coverage" project are that not tests should come first, but the design of very small parts (a method or a small class) should be first instead. The design is primarily guided by the user (caller) of that unit. Design is always finding a sweet spot between a desired feature on the one hand and technical considerations just as available technologies, efficiency, and -of course- testability, on the other hand. I don't think it matters whether you write the implementation (of a small unit) or its tests first as long as you get all tests to pass before you tackle the next unit. (Personally I prefer implementing it first, because the implementation often is a more holistic description of the problem. Only for complex algorithms (which I find to be rather rare), writing tests first seems to give a better start at properly understanding the problem.) By starting with the design (which most often is an interface specification), I find it much easier to think about the method or class in a holistic fashion and also figure out a set of test cases that's small yet covers everything I need. Would you say that this process is still TDD?

Fast tests with high coverage are very important to me, not least because refactoring is very important to me. But I don't like the term "test-driven" because the driver of development is always some external (non-technical) need, such as a feature or some resource-restriction ("make it faster"). Tests are just a technical tool (albeit an important one) and it's the design that creates interfaces which both fulfill customer needs and technical standards. I think of my development rather as "Feature-driven", "design-guided", and last not least "integrated-testing" (because tests are an integral part of the code). Maybe the term "tests-in" is more catchy? As long it isn't "driven...". After all, model-driven also didn't work that well... ;-)

How to write good software and why baby-step TDD is a scam

First off, I am obviously not going to tell you all about writing good software in a single blog post about TDD. Writing good software takes a lot of learning and a lot of practice. There have been countless books written on the subject and since this post isn't about a book list for software engineers either, I'll just mention one to give you an idea: Object-oriented software construction by Bertrand Meyer.
The company I work at has quite a large software development department and quite a good leadership for the latter. Our managers promote autonomy (developers choose the technologies and methods they think are best suited for the work) and learning on and off the job. For example, we have regular (voluntary) coding dojos (practice sessions) where a bunch of developers sits together to solve some simple problems with some new approaches. This is certainly an important part of writing good software.
Recently, we experimented with Test-Driven-Development (TDD), which some people also read as Test-Driven-Design. TDD as my colleagues introduced it to the rest of us is based on the following three rules:

1. You are not allowed to write any production code unless it is to make a failing unit test pass.
2. You are not allowed to write any more of a unit test than is sufficient to fail; and compilation failures are failures.
3. You are not allowed to write any more production code than is sufficient to pass the one failing unit test.

(Something most proponents of TDD would add is a fourth step to refactor the code while the tests are green, but when TDD is introduced and defined this step is usually not mentioned.)
Our company DOJOs and some reflection upon them have taught me that this is plain bullshit and here's why. In the last two decades, the profession of software development has embraced methods like automated (unit and integration) testing, iterative development, early testing (also called "tests first"), merciless refactoring, design patterns, automated builds and many more. All of those practices are great if done right. Now TDD comes along and claims to condense many of them together into an integrated framework based on the above rules. Going back and forth between tests and code is obviously iterative. Tests obviously have to be automated. You obviously need refactoring, because otherwise TDD will produce terrible code. So TDD dresses itself up as the natural evolution of agile development. But the truth is: TDD is a perversion of agile which over-applies agile principles in a way that doesn't make any sense any more.
Somebody who claims to do TDD either doesn't follow the three rules above or they're doing helplessly bad development. TDD is a scam because it contributes nothing new to the set of agile practices. If someone using “TDD” succeeds writing good code, it is due to the other agile practices, not due to the three rules above. TDD even obscures and ignores a lot of other important methods. SCRUM, for example, tells us to define minimal features and implement them including production code, automated tests, and all that's needed to deploy and run the feature live. SCRUM offers a lot of advice on what a minimal feature is, how to split stories and what's small enuf not to need any further splitting. TDD, on the other hand, splits iterations too much, ignoring SCRUM's advice. Design by Contract tells us how to write minimal interfaces by considering both the needs of the client and the provider and describing the interface succinctly in code. TDD, on the other hand, says that interface should emerge while they instead drown into a plethora of special cases. Finally, testing methods teach us how to design good (and minimal) test cases, get good coverage, and test most where it is needed most. TDD, on the other hand, says nothing about where you start, how to continue, or when you are done. Tests are always green, but when do you have enuf tests?
Think about that: there have been countless example demos of TDD on the internet, on conferences, in practice sessions, but have you ever even seen a small program development finished with TDD? To the contrary, the only thing I see are epic failures. (Thanks, Fred, for the great link!)
So, can we please forget about this exaggerated baby-step TDD, stick to established best practices, and move on writing good software?

Addendum, months later: I saw a good example of TDD in Freeman & Pryce's book "Growing Object-Oriented Software". Their interpretation is much better than the baby-step TDD seen in blogs. The book starts by summarizing established best practice OO design. Their example study is much more elaborate and the problem domain is actually related to the kind of software that professional Java developers are writing for money. If you want to know about the real thing, you have to take the time to read something longer than a couple blog posts.

Refactoring examples: little steps and big smells

My friendly coworker shared a video of Uncle Bob live refactoring some code. Since I love refactoring I was very excited to watch it, but a few minutes into the video my excitement turned into horror, disappointment and anger. Uncle Bob refactors a piece of smelly code, but instead of removing the cause of complexity (namely too many things being done at once), he just spreads the complexity out into many different methods which communicate with each other via member variables. The result looks cleaner and certainly has good naming and short methods, but it still has way too much complexity. And what's worse, with everything spread out in so many pieces, it's much harder to refactor to really simplify it to the core. And what's the worst of worst: even forty years after the invention of such useful principles as "command-query-separation", "separation-of-concerns", and functional programming, Uncle Bob happily violates all those great principles to clumsily cultivate complexity and call the result "Clean Code" and sell it for money. Skip the jump to see the code, good and bad.