lazy log

Logback hack to log different levels in the same package to different appenders

2011-07-25T07:38:00.001-07:00

Given the long title you could think that this is going to be a blog about a ridiculous edge case. It isn't, honestly! It is a blog about an edge case, but not ridiculous and not as far fetched as you think. Let's say that you have a class logging on DEBUG and TRACE. The debug messages contain (frequent) checks, but lack the full information needed to analyze the problem. This information (say a full xml message) is logged on TRACE level. Now your friendly system administrator decides that he needs this trace logging to go into a different file from the debug logging. He has good arguments and you don't want to pick a fight. Logback provides a lot of configuration options that you can do lots of neat things with. But it can't fill this requirement. Really? Well, there is a hack.

Setting up the experiment

-----------------------------------

Logback works with appenders and loggers. Appenders define where output goes and how it is formatted. Loggers define where logs come from and which appenders should be used to write to.

So if we want to log to a certain appender on a certain level we create a logger like this.

1 2 3 4

  <logger name="iwein.samples.foo">
    <appender-ref ref="STDOUT1" level="trace" />
    <appender-ref ref="STDOUT2" level="debug" />
  </logger>

The appender totally standard, but I'll include it anyway.

1 2 3 4 5

  <appender name="STDOUT1" class="ch.qos.logback.core.ConsoleAppender">
    <layout class="ch.qos.logback.classic.PatternLayout">
      <Pattern>STDOUT2 %logger{36} - %msg%n</Pattern>
    </layout>
  </appender>

The logging is done in a class in the package 'iwein.samples.foo', again just adding it for completeness.

1 2 3 4 5 6 7 8 9 10 11 12 13

package iwein.samples.foo;

//...

public class SomeLoggingThing {
    private static final Logger logger = LoggerFactory.getLogger(SomeLoggingThing.class);

    public static void saySomething(){
        logger.info("INFO");
        logger.debug("DEBUG");
        logger.trace("TRACE");
    }
}

I wrapped this in a testcase, but if you want all the code you'll just have to [grab it](https://github.com/iwein/logback-testdrive/commit/1b543645dc561ebb81237238d5698d95c34d4f4f). We now have the basic position from which the interesting stuff can start. Let's try to implement the requirement next.

First attempt: multiple appender-refs

-------------------------------------------------

We want our logger that is created in the java code up there to log to different appenders. You can do this by adding an appender ref, but we also want to log with different levels to different appenders. That's a little harder.

1 2 3 4

 <logger name="iwein.samples.foo" level="trace">
    <appender-ref ref="STDOUT1" />
    <appender-ref ref="STDOUT2" />
  </logger>

If I did the domain model for logback (which I didn't), the appender-ref would correspond to an object, on which the log level would be set. In reality, logback allows you to set a level on the logger, so one logger only has one level. Would be nice if you could do something like this:

1 2 3 4

  <logger name="iwein.samples.foo">
    <appender-ref ref="STDOUT1" level="trace" />
    <appender-ref ref="STDOUT2" level="debug" />
  </logger>

Which you can't... which is a bummer.

Second attempt: multiple loggers

--------------------------------------------

Now conceptually, if you add another logger, you could use a different level there and a different appender too: problem solved, TADA! So let's try.

1 2 3 4 5 6 7

 <logger name="iwein.samples.foo" level="trace">
    <appender-ref ref="STDOUT1" />
 </logger>

 <logger name="iwein.samples.foo" level="debug">
    <appender-ref ref="STDOUT2" />
 </logger>

When I run this in my test the output is:

STDOUT2 iwein.samples.foo.SomeLoggingThing - INFO

STDOUT1 iwein.samples.foo.SomeLoggingThing - INFO

STDOUT2 iwein.samples.foo.SomeLoggingThing - DEBUG

STDOUT1 iwein.samples.foo.SomeLoggingThing - DEBUG

What happens is that there is only one logger in the end with two appenders and its level set to the last logger loaded. Kraznoks!

After some headscratching we came up with another solution. It isn't pretty, but it's the first one to actually do the job.

Third attempt: filter in the appender

-----------------------------------------------

Before you run away screaming, let me assure you that I know this isn't how it _should_ work, but it's the only way I could come up with so far. I'll log an improvement request in the logback issue tracker of course.

So how about you log to both appenders, but filter out the trace messages you don't want in one appender? That would work...

The downside is though that I had to make my appender look like this:

1 2 3 4 5 6 7 8 9 10 11 12

  <appender name="STDOUT2" class="ch.qos.logback.core.ConsoleAppender">
    <filter class="ch.qos.logback.core.filter.EvaluatorFilter">
      <evaluator> <!-- defaults to type ch.qos.logback.classic.boolex.JaninoEventEvaluator -->
        <expression>return Level.TRACE.equals(Level.toLevel(level)) &amp;&amp; logger.startsWith("iwein.samples.foo");</expression>
      </evaluator>
      <OnMismatch>NEUTRAL</OnMismatch>
      <OnMatch>DENY</OnMatch>
    </filter>
    <layout class="ch.qos.logback.classic.PatternLayout">
      <Pattern>STDOUT1 %logger{36} - %msg%n</Pattern>
    </layout>
  </appender>

Which is nothing short of an abomination. See http://jira.qos.ch/browse/LBCLASSIC-284 for progress on this issue.

Posted via email from Iwein's braindumps

Found in the wild: uncertainty

2011-05-09T02:26:00.001-07:00

Let met just keep this short and let the code speak for itself:

1 2 3 4 5 6 7 8

  // The next fields counts the number of time the decide method is called
  //
  // IMPORTANT: This field can be updated by multiple threads. It follows that
  // its values may *not* be incremented sequentially. However, we don't care
  // about the actual value of the field except that from time to time the
  // expression (invocationCounter++ & 0xF) == 0xF) should be true.
  private long invocationCounter = 0;

Three letters apply here.

Posted via email from Iwein's braindumps

Confession of a developer junkie

2011-03-22T02:19:00.001-07:00

I like splitting wood. It’s a simple physical activity and I’m good at it. I can tell you exactly why I like splitting wood; it is very similar to why I like doing some other things, like building software for example. There has been a lot of discussion in software development about this thing some people call craftsmanship

When you split a log (assuming you have a decent tool) things are quite simple. There are a lot of physics involved, like the direction of the grains, the cohesion between them. The moisture is important too, the momentum of the axe, the angle… But when I swing I don’t think of those. I just look at the log, know where I want to hit it and then it happens the way I saw it in my head. It doesn’t always go right, but it does a lot of times. When it goes right I know it before I register it. Right at the moment the axe has its maximum momentum and the direction is set I know there is nothing to correct and the scenario can play out the way it should. And then there is the moment when the axe hits the wood and sinks in effortlessly. There is the well articulated “chk” immediately followed by a “pang” as the tension makes the log snap apart. If you hit as hard as you can you will send two parts flying away, but that doesn’t give me most satisfaction. I like it when I hit just hard enough and two parts end up standing next to each other. I can immediately go on and take another swing without stooping over and straightening the logs. That feeling is the most addictive thing I’ve found in life. I can continue splitting wood far beyond comfort, just to get another fix of that feeling. I am not alone in this I believe.

A long time friend of mine is a baseball player. He told me a similar story about hitting a ball just right. The pitcher throws, not too slow, but you catch the ball with your eye. You swing at it and as the bat cuts through the air, you know it’s going to connect right. Then the ball hits the bat so that you hardly feel it. You just year the “ping” and you know it’s a home run. I’m not a baseball player, but apparently there is nothing like that feeling. I believe it is the same feeling that I get from splitting logs.

A related thing has been described before by Csíkszentmihályi Mihály. It is referred to as “Flow” or “The Zone”. If you haven’t seen Csíkszentmihályi’s work you definitely should have a look (feel free to copy paste now to avoid spelling errors). I believe this immediate feeling of joy as you do something right lies at the basis of flow.

With some effort I can get this feeling when developing software too. It is even possible to share this feeling with a whole team of developers. When that happens I love my job so much I’d do it for free until I starve. With some more effort it is possible to do this on a project that actually delivers business value to a customer. This means I can do this great thing without starving if I play my cards right. I need a way to get the people that stop me from doing this to get a clue and let me do my thing. You probably know the types I’m talking about: it’s the manager up high that makes me work on a virtual desktop that runs vista; it’s the architect that forces me to use a five year old version of a bad framework and makes me deploy to Weblogic 5; it’s the customer that waves three years old specs at me when I ask him a question about how he wants you to implement a feature. It is also the colleague that refuses to talk to me about my real problem if it is not related to cool new obscure technology X that somebody blogged about this morning.

If I can get these people to grow up by talking to them about Agile, XP, or Craftsmanship, or Lean, or Oompa Loompa’s; I will do it. I’m a junky remember? I want to get my daily flow fix! When Robert Martin said we were tired of writing crap, it didn’t ring true with me. I’m not tired of it at all. I’m prepared to write a lot of crap to get it right just once. If we do have a nice bit of marketing on the word craftsmanship I’m fine to surf that wave for now. Next year we’ll call it Real Engineering, the year after Real Development. Why the hell not. As long as some of us get what it is about and they are in my team I’m totally cool with it.

Posted via email from Iwein's braindumps

Ja tegen proteïnen, nee tegen vieze shakes

2011-03-22T02:04:00.001-07:00

Ongeveer een half jaar geleden kreeg ik last van mijn rug. Het advies van de dokter en fysiotherapeut was om zo veel mogelijk te bewegen en zo weinig mogelijk achter de computer te zitten. Omdat ik vastbesloten was, mijn rug (en buik) zo snel mogelijk in vorm te krijgen, heb ik het advies opgevolgd. Deze blog heeft daar enigszins onder geleden, maar omdat ik nu een slanke en fitte dertiger ben, mag ik weer lekker lang achter mijn computer zitten ;-).

Gezond eten en regelmatig bewegen zijn onmisbaar om een gezond lijf te krijgen en te houden. Als je eenmaal lekker aan het bewegen bent geslagen, wordt het een sport om steeds iets meer te tillen en langer door te gaan. Op het feestje van Wouter, kwam The four hour body' ter sprake. Om een goddelijk lijf te krijgen hoef je volgens Timothy Ferriss slechts 4 keer in een maand te trainen en moet je 100 gram extra eiwitten per dag eten. Als je bedenkt dat er in een eitje hooguit 7 gram proteïne zit, moet je dus al gauw 14 eieren per dag eten... Ze zijn niet gemakkelijk weg te werken, want eieren vullen goed. Je gaat van eieren enorm stinken. Bovendien is een eenzijdig dieet niet gezond. Dus op welke andere manier krijg je dan 100 gram proteïnen per dag extra binnen?

Een eenvoudige, maar niet bepaald prettige manier, is het drinken van weipoeder shakes. Je haalt zo gemakkelijk je quotum, maar er kleven nogal wat nadelen aan:

door de proteïnen verdwijnt je hongergevoel, daardoor is het lastig om voldoende gezond voedsel te blijven eten,
je kunt er vreselijk verstopt van raken dus je moet extra vezels eten, maar je had al minder honger dus dat is een hele opgave,
het is hartstikke vies. De paar slokken die ik geproefd heb, waren meteen de allerlaatste. Op het kokhalsen na werd ik niet ziek, maar drie keer per dag een halve liter drinken is me echt te gortig.

Dus weipoeder wordt het liever niet.

Hoe krijg je op een gezonde en prettige manier 100 gram proteïnen per dag extra binnen?

Eerder zijn de 14 eieren al voorbij gekomen. Er zijn verschillende gerechten te vinden om eieren makkelijker verteerbaar en lekkerder te maken. Voeg steeds verschillende groenten en verse kruiden toe en je hebt een heel andere smaaksensatie. Kijk bijvoorbeeld even naar mijn Thaise omelet. Het vullende effect van de eieren is hiermee nog niet opgelost. Dus om je quotum te halen moet je ook:

de hoeveelheid koolhydraten in je dieet limiteren (minder pasta, rijst en brood),
langzame koolhydraten eten en vooral geen snelle (geen snoep, maar brood),
vaak sporten,
proteïnen en vetten eten om je honger na het sporten te stillen,
granen en fruit voor en tijdens het sporten eten,
je eten over meer maaltijden verdelen.

Andere overheerlijke gerechten met veel proteïnen:

Gerookte zalm met mierikswortelmayonaise

Tegenwoordig vind je in bijna ieder restaurant wel een visgerecht met wasabi. Het zou dan ook saai zijn om op deze blog een recept met wasabi te geven. In plaats van wasabi gebruik ik mierikswortel (wasabi en mierikswortel zijn broertjes). Mierikswortel heeft een scherpe smaak die op mosterd lijkt. Mierikswortel is in de gewone supermarkt in een potje te verkrijgen, maar je kunt het ook mierikswortel zelf kweken. Waarschuwing: in tegenstelling tot wasabi groeit mierikswortel in Nederland fantastisch, je krijgt het nooit meer uit je tuin.

100g gerookte zalm (laten we niet kinderachtig doen)
wat rauwkost naar smaak (veldsla doet het goed, maar ook ijsbergsla of komkommer is er lekker bij)
3 eetl. mayonaise (mag je zelf maken, of niet natuurlijk)
1 eetl. gemalen mierikswortel
beetje verse dille

Meng de mierikswortel, de mayonaise en de dille in een kommetje, zodat het een lekker sausje wordt. Leg de zalm op een bord en serveer de saus en rauwkost erbij.

Als je dan zo'n lekker stukje zalm op hebt dan kun je er vast nog wel een biefstuk bijhebben:

Biefstuk op spinazie met lijnolie tobasco dressing
Neem een goede biefstuk. De Marqt is prima, maar er zijn ook redelijke biefstukken in de schappen van de supermarkt te vinden, neem vooral niet de lichtroze geverfde kiloknallers.

1 biefstuk
200g verse spinazie
scheutje lijnzaadolie
tobasco naar smaak

Plaats een bord in de oven en verwarm de oven voor op 75 graden Celsius. Braad de biefstuk in weinig boter of een beetje rijstolie rondom snel aan. Haal hem uit de pan en leg hem ingepakt in aluminiumfolie op je bord in de oven. Gooi de spinazie om in de nog hete pan, totdat alle blaadjes slap geworden zijn (duurt ongeveer 2 minuten). Haal je bord en de biefstuk uit de oven en leg de spinazie op het voorverwarmde bord. Leg de biefstuk op het spinazienestje, schenk er de lijnzaadolie en tabasco over.

Met deze twee gerechten heb je dus het equivalent van 8 eieren te pakken. In vis zit namelijk ongeveer 25 gram proteïnen per 100 gram, daarbij zitten er Omega 3 en 6 vetzuren in die erg goed schijnen te zijn tegen hart en vaatziekten. In biefstuk zit ongeveer 28 gram proteïnen per 100 gram. Het mooie is dat je van dit soort gerechten lang niet zo'n vol gevoel krijgt als van 8 eieren. Ik kan het in ieder geval prima vóór mijn avondmaaltijd wegwerken. Als je bij de lunch nog een omelet eet, heb je de extra proteïnen al bijna te pakken.

Als je geld genoeg hebt en een goeie visboer in de buurt, kun je de rest mooi met oesters aanvullen :).

Eet smakelijk!

Posted via email from Koken en Eten

Curing distributed Agile Illnesses: Diagnosis

2011-02-26T09:12:00.001-08:00

In my session on 'When Agile Offshoring fails' at AgileNCR this Saturday I decided to harvest ideas from the attendees on different illnesses. The results were beyond my expectations; in this blog I will share a few of them and give a few preliminary comments on the result. The general plan is described better with the following diagram.

First I would ask the attendees to come up and write down all the symptoms related to failing distributed agile teams on a sticky. These we put on one part of the walls of the room. Then I would ask them to write causes for symptoms on stickies and put them on another part of the wall. Finally we would all try to come up with a possible mitigation for each cause. It worked beautifully, let me show you.

Interestingly I intended to do in this session exactly what Olav would be urging for the first day of the conference: analyze agile failures so we can inspect and adapt and become more effective. But let me get to some details before I give up and fall asleep.

The following is a (very poor) picture that I took of my hotel wall after rearranging the stickies I had stuffed in my bag.

Now that I have this, the real work can begin. First I will look at the causes and cures to see what fits with what, sift out the duplicates and see if I can formulate a couple of likely experiments. Then I intend to formulate a challenge and post that at the Xebia blog.

One attendee commented (and rightfully so) that it was a bit ungratifying that we didn't come to a full conclusion after the session. There's no closure here for sure. While the intent of the session was to sparkle the interest as opposed to satisfying the attendees, there is no reason to be mean. Here are a few highlights that I can see through the fog already:

Frequent co-location is mentioned the most as a solution by far, it mitigates whole bunch of problems
By far the most problems evolve around exchanging information, but a remarkable second is trust

One other interesting thought that keeps nagging at me is that many of the listed symptoms and causes are reenforced by larger team sizes. This is the case with both co-located teams and distributed teams, but because the mitigation here is usually sought in colocation I am inclined to consider that in co-location larger team sizes are possible than in distributed team. Given the other factor that co-location is expensive, this poses an interesting optimization problem that only in certain particular configurations distributed teams are more cost effective than co-located teams. It would be great to get some data on this...

I'm now torn between digitizing the cards on linoit and my bed. I'm affraid the bed is winning at the moment. I hope you enjoyed this, stay tuned for more analysis.

Posted via email from Iwein's braindumps

Easy peasy websockets with node.js and socket.io

2011-01-20T08:23:00.001-08:00

With our Web Sockets incubator we had a rough start on Atmosphere grinding through the samples. There is no intention to bash Atmosphere, it looks like an interesting approach. However, it should be a lesson to all aspiring framework builders out there: getting your samples in top shape matters a lot. This week we tried out socket.io and I wasn't too hopeful as I had planned all kinds of things throughout the evening. The result of a 40 minute pairing session with Sonny Gill was pretty good and I'll share the basic highlights here in a very short post.

First we installed node.js and used npm to install socket.io. Next we moved on to create a simple app that exposes a socket on the server. Finally we created a client that opens a socket and sends some test string over the wire.

The code you can find on github as usual: https://github.com/xebia/app-incubator-web-sockets/blob/master/socketio-sample. All in all it is less than 50 lines of javascript and html and it simply works as expected. Not bad for less than an hour of fiddling! In our next session we'll look into the possibilities of streaming video over Web Sockets, or maybe we will look at another framework to see if it performs any more interesting magic tricks. That's all for now, stay tuned!

Posted via email from Xebia App Incubator

Force breakthrough by forcing mistakes

2011-01-11T01:36:00.001-08:00

A week or so ago I started to play Math Workout to rekindle my degenerated calculation skills. After a while I got stuck at a certain speed, making no mistakes anymore. I tried to push myself to go faster, but without making mistakes but that just didn't work. Then I tried a different approach: I forced myself to go fast, mistakes or no. After a couple of sessions, I went back to making no mistakes and to my surprise I was around 8% faster. I have a theory about why this is and I'm applying it to life without hesitation from now on, so read on if you want to share the fun.

What's Math Workout

I'll give you some background on Math Workout first, so you can understand the example. Math Workout is a game that let's you solve simple calculations and times you while you do so. The main goal is to be as fast as you can. For example, on easy level the hardest addition is 8+7 and the hardest multiplication is 4*6 (I'm considering 5*6 easier). Dead easy as you can see. The trick though is that you need to solve 20 of these problems in less than 23 seconds, and I can tell you that's tricky enough because you need to read the problem, come up with the answer and type it correctly in about a second.

What goes wrong if you don't make mistakes

Somehow I got into a rhythm. See the numbers, see the operator, answer pops up, type it. There are probably a few more steps that I was doing unconsciously, but the point is that I was taking a fixed amount of time for each step. Sometimes I found myself breaking the rhythm just before I punched the wrong number and then taking some fixed amount of time again to correct myself. Whenever that didn't happen my time would be between 26 and 27 seconds consistently. Basically I had put a few safety checks in (which I probably didn't need most of the time) that were slowing me down. Whenever I pushed myself, I would occasionally parse the wrong operator, type the wrong number, pop up the wrong answer and quickly revert to the strategy that worked. Once I noticed this, I knew I needed to break the pattern.

Forcing errors

I decided to change the pattern, and ignore the mistakes at first. I decided to ignore the operator check and assume that my brain would be able to pick up the numbers and the operator in one go. I made 5 errors out of 20 questions and decided that I was bad at picking the operator, but it must be working somehow, otherwise I would have seen more in the range of 10 mistakes. I gave myself positive feedback by thinking I was a smart cookie for figuring that out. I repeated it a few times and saw the errors go down and the times with it, more positive feedback. After a mere 5 runs I focussed on making no mistakes again, and I was below 25 seconds. Mind you, before that, trying for over 50 runs there was no way in hell that I was going to go under 26, my top times were averaging around 27.5 seconds. By forcing the errors I had broken the pattern and found a more optimum way of behaving. Much faster than waiting for some luck would have done.

What's going on here

When you make a mistake you tend to give yourself negative feedback "aww shoot!" and this tells you to avoid the behavior that led to this the next time. When you don't make mistakes and you have a better time than before you give your self positive feedback and reinforce the behavior that led to the result. When you try something new and it doesn't pay off immediately, you will give yourself negative feedback. We crave positive feedback, so we keep on aiming to get that result improving, even if our method is flawed and we're in the wrong rhythm. It's hard to break a pattern that reasonable results but is suboptimal. The main reason here is that when left to it's own self learning devices, the variation in methods goes down rapidly. It takes a conscious effort or a lucky coincidence to break out of the suboptimum. The lucky coincidence becomes less and less likely the more you reinforce the suboptimal behavior. This is caused by a thing called Extinction Burst which is intended to pulls you back into the behavior that worked in the past.

Download now or preview on posterous

PastedGraphic-3.pdf (21 KB)

What's to learn?

It pays off to try things, even if the first results are not impressive follow through for a bit and see if you might just be crossing a chasm and things are better on the other side. I've seen this pattern not only in my own behavior, but also in the behavior of groups of students and teams of software developers. As long as we didn't plot the graph of all possibilities the only way to find out that behavior is suboptimal is to explore and find a better type of behavior. With Math Workout this is easy, because if I can't make the required time my behavior must be the problem. If there is no required time and nobody knows which goals to set (like in Software development, or any kind of creative work really) you can never be sure of what works best until you've literally tried everything. The trick is to find a way to reward yourself for trying the new thing that doesn't relate too much to the old reward of squeezing every last drop out of a poor method.

I'm not saying you should be trying everything right now, but how about trying one thing each week? That would already be a huge improvement for most of us I'm sure.

Posted via email from Iwein's braindumps

Olijfolie

2010-09-24T05:58:00.001-07:00

Het is in Nederland niet makkelijk om goede olijfolie te vinden. Als je de spullen van de supermarkt gewend bent, weet dan dat er de gelegenheid is om een ster te winnen simpelweg door het gebruik van betere olie. In deze blog ben ik op zoek naar een goede, betaalbare olie en waarom de beste olie die ik kon vinden niet in de winkel te koop is.

Bij het produceren van olijfolie worden verschillende extractie methoden gebruikt. De belangrijkste factoren zijn de lage temperatuur (eerste koude persing) en de manier van malen waarbij zo min mogelijk de pit en schil van de olijf meegeperst moeten worden.

Deze beide criteria zijn in directe tegenspraak met de kwantitatieve grenzen die het winstgevend produceren van olie in hun greep houden. Immers: hoe meer olie per olijf, hoe goedkoper de productie. Hierom worden latere persingen uitgevoerd waarbij de gemalen olijven fijner worden gemalen en worden gekookt.

Op zoek naar goede olie zijn veel speciaalzaken te vinden die wel prima olie hebben, maar deze olie is duur. 20 euro per liter is de prijs die de olijfoliespecialist in Culemborg bedingt. Ik betaal graag voor goed kwaliteit, maar als het wat minder kan (qua prijs dan) ben ik ook te vinden.

Vriend Tiziano nam mij een halve liter olie van zijn ouderlijk bedrijf in Italie mee. Vriend Guido bracht mij een kruikje olie van het vat dat hij in Kroatie had gekocht. De olie uit Kroatie kan zich gemakkelijk meten aan de producten van Kastelli, en de olie van Tiziano is beduidend beter (smaken verschillen, maar het zegt toch iets).

Even rekenen hoor… Er is olie die in het land van herkomst voor 5 euro per liter (of minder) verkocht kan worden, en beter is dan de olie die je voor drie keer dat bedrag in Nederland koopt. Deze olie komt niet op de markt (in het Italiaanse geval), en al helemaal niet op de internationale markt (in beide gevallen). Voor olie van de kwaliteit waar we het over hebben betaal je in de VS rustig 50 dollar per liter, en in het noorden van Europa is 40 euro per liter ook niet vreemd.

Het aanbod in het land van herkomst (Italie) is groot, maar de prijzen zijn daar zo laag dat de boeren de moeite niet nemen hun olie ter verkoop aan te bieden. De vraag in het noord-westen is groter dan het aanbod aldaar (anders zouden de prijzen niet zo hoog liggen). Het grote gat tussen de prijzen die de boeren in Italië krijgen en de prijs die in de winkels in Nederland betaald word is moeilijk te verklaren.

In elk geval, ik laat de conspiracy theories aan de lezer over, maar als je op vakantie gaat naar een warm land, vergeet dan niet een beetje olie bij een boer te kopen. Wel eerst even proeven hè?

Posted via email from Koken en Eten

Reaction to "Why ORCL was right to sue ..."

2010-08-23T01:49:00.001-07:00

Posted via email from iweinfuld's posterous

Forging intelligent Teams instead of staffing Projects with intelligent people

2010-08-20T04:54:00.001-07:00

In my experience with consulting and development, project teams (both Agile and traditional) invariably face issues with availability of team members. In one (rather long and messy) talk on craftsmanship Robert Martin came up with an interesting idea that I'd like to explore a bit more. It might actually solve this problem if used in the right way in a large organization. The idea is simple: organize into long term teams of upto 12 people and accept work from different projects as a team.

This is actually the way small development companies have worked for ages, and they have done quite well on it. The problem is that when projects and clients get bigger they tend to turn things upside down and instead of pushing work to teams that can handle it, they start pulling resources (eww I hate that word when applied to human beings) into the project. This process is called staffing.

There is a problem with staffing. People are not resources. The difference between people and resources (think coal, plated steel, money) is that they change based on changes in their environment. This ability is defined by Stephen Hawking as intelligence, and I like that definition. The big upside of intelligence is that it can actually solve problems for you and that is something that typical resources cannot do. If intelligence is the stuff that allows you to come up with solutions, resources are the stuff that you consume to implement them. A profound difference here is that coming up with a solution actually increases the amount of intelligence, whereas implementing a solution always decreases the amount of resources.

If managers pretend that people are resources and projects are black boxes that turn resources into implemented solutions they will get two things: shortage of intelligence and shortcomings in implemented solutions. The shortage of intelligence is caused by the fact that a project starts being about finding a solution and invariably more and more becomes about implementing it. The former makes you smarter (increasing intelligence) the latter dumbs you down (decreasing intelligence). If you only implement solutions you lose your ability to adapt to change, but if all you ever do is adapt to change you might go crazy. There is an optimum between the amount of change a person has to handle and the amount of time he needs to regain his footing, and this is different per person.

This is all well and fine, but how will we get those essential team members available then? I'm getting to it, don't worry.

First we put a team together thinking about a certain context, not a project. Then we let them establish a healthy environment for them to adapt to and increase their intelligence. This causes team members to be available to each other by default. The we let the project managers sell the work to a team, possibly multiple teams. Each team takes on what they are comfortable with but they stick together. Now this chunk of work might be too little or they might get bored with it. Then they take on some more work, possibly from another project, but they keep together as a team.

If a team member wants to leave this is a career move, not a ephemeral thing you can do for a couple of hours on friday if you feel like it. You can't be part time on one team and part time on another. If you want to take on some work you have to convince your team, or make your career move and be gone totally. Now the question of when person X is available to person Y is extremely simple. When they're in the same team they can count on each other. If they are not on the same team they might have a meeting some time.

This should up a lot of things, I'm looking forward to try it. If anyone already has experience with this approach on a large scale or moving a large development organization into this mode I would love to hear some war stories.

Posted via email from iweinfuld's posterous

The Copy, then Rename pattern for file based integration

2010-07-27T11:41:00.001-07:00

This is a blog for those unlucky many of you that, like me, had to solve the file integration problem. Two processes are integrating over a directory, one writes (W), one reads (R). Now the reader picks up a file while the writer is still writing to it. BOOM, R caughs and dies. Sounds familiar? Then read on.

I know I promised a blog on a deadline free utopia, but while I'm chewing over that one I need to jot down this thought. That's what blogs are for right to capture the excess of an overflowing mind?

So back to R and W that can't seem to get their processes synced. How do you prevent R reading that file that W is still writing? Locking will work, but it's clunky and tricky. Looking at the timestamp might work in most of the cases, but it is not guaranteed to work under high load. Can you say heisenbug? So, what can we do.

Well in fact the solution is too simple to spend more than a few sentences on. It is old, battered and scarred, and it will be around for a while in the future. Just Copy the file to a temporary name and then Rename it to its final destination. Maybe I'll include an image later, but you should get the point, right?

Posted via email from iweinfuld's posterous

BBQ experiment: Grilled pineapple

2010-07-19T08:12:00.001-07:00

I'm having a barbecue with some Xebia colleagues next Tuesday. When I have a couple of guests over for the first time, I always like to play it safe a little bit, but the following two experiments just seem too easy, so I'll take the risk:

Grilled Pineapple (adapted from Ananas al Forno):

For the original recipe look at this old blog. I don't see a reason why it wouldn't work on the barbecue, and potentially improve the flavor given the right type of smoke. I'm thinking sage should work.

Marinaded sweet potato

I have no clue yet how I'm going to pull it off, but I've made marinaded carrot shaslic's before and that worked out pretty well, so I'm guessing that it's "just" a matter of getting them done just right and finding the proper marinade for them.

I'll update this post with the story of success or as it may happen disaster after Thursday, if you have any tips, warnings or other first hand accounts please help me out before I make a fool of myself.

Posted via email from Koken en Eten

Please stop moving deadlines

2010-06-20T00:55:00.001-07:00

I've been struggling with deadlines for a long time, and I've finally discovered the pattern that makes my struggle most painful. They move all the time. A moving target is harder to hit than a stationary target, so my theory is that the more deadlines move the more futile planning becomes. The good thing about time is that it (usually) only moves in one direction, but man has invented budgets to compensate for this glaring lack of complexity.

So what can we do? Well just never, ever move a deadline. I will go into detail some later post on how to avoid deadlines altogether, but for now let's just consider what happens if you make the deadline stationary.

When you refuse to move the deadline the thing will not be DONE.

This is not a problem. Nothing is ever done. Done is an invention by people who like tickboxes. When the deadline comes you just do what you planned to do at that deadline, or you conclude that you have failed. Failure is good. Failure is the only true done. If you have failed you are free to go do something else, probably something that you have found the inspiration to do when you were dragging around Doomed To Fail project X. A deadline that can't be moved helps you to fail early and this will save time and money.

When you refuse to move the deadline the thing will not be as good as it can be.

This is a misleading statement. If you're building a cruise control driver, "as good as it can be" is a well defined concept, but most of us aren't building those kind of applications. What we're building is usually the implementation of some idea somebody dreamed up with very limited input. It might be a very good idea, but it's not well defined. In fact it's our job to define it based on small hints the product owner is giving us. This makes "as good as it can be" a moving target. The longer we wait to deliver, the further ahead the dream of the business will get ahead of our actual implementation. And the bigger the disappointment when we finally do deliver what we have.

But just look at what we have now! We can't deliver like this, right?

Maybe you're right - you're probably not though. Delivering is easy and delivering something that sorta works is much better than delivering nothing at all. You don't know if the business can break even with the thing you consider crap before someone tries. It is your obligation to try this as soon as possible. Cheating your customer into thinking he's going to get rich is not professional.

Much more important though: the longer you wait, the more you're going to think you're not ready. Dreams go faster than reality and by definition you will never catch up. By delivering whatever you have on the deadline and not moving it you're delivering reality to the dreamers. The earlier you do this the more they will keep liking you.

Ok, no more moving deadlines. But what happens after the deadline?

Well, its simple. After the deadline people will invariably think it can be done better. If you deliver your ideas of how to do things better with the unfinished product, you might get a chance to fix the things you wanted to fix before delivering. The good news is that you will get that chance based on realistic expectations. And if you don't get that chance you're free to do something else, which might not be a bad thing at all.

Posted via email from iweinfuld's posterous

Stir fried spicy spinach

2010-06-18T11:02:00.001-07:00

There's nothing original about stir fried spinach with a dash of other flavors, but this combination was pleasant enough to mention anyway.

Per person:

200g spinach
2 cloves garlic
2 thai chilies (a normal chili pepper should work fine too)
juice of half a lemon
1 tbsp grenadine concentrate (Ayurveda)
arachide oil

Heat the oil over medium heat, add the ginger in fine slices so it can get acquainted with the oil. When the ginger is getting soft add the garlic (bashed and chopped) and chilly, also finely sliced. After a minute or so, add the spinach and increase the heat. Stir until the spinach is done and add the grenadine and stir again. Finally turn of the heat and add the lemon. The taste is quite funky, but definitely worth the 10m that it takes to make.

Posted via email from Koken en Eten

Untitled

2010-06-14T10:34:00.001-07:00

I made this for lunch yesterday and it was even better than I expected:

3 eggs
handful of coriander leaves
two hands full of mushrooms
juice of half a lime
2 tbsp of skimmed milk
1 tbsp sesame oil
1 tbsp sweet soya sauce
1 tsp thai fish sauce
rice oil (to fry in)

Chop the coriander and whisk all ingredients together except for the rice oil and the mushrooms. Slice mushrooms. Heat up a frying pan and toss in the oil and mushrooms. Fry until the mushrooms have lost most of their liquids and give off their aroma.Reduce the heat (or take the pan off the heat for the moment). Whisk the mushrooms into the omelet mixture. Make sure the pan is clean (if you have a good pan you can just stare at it for 0.5 seconds), add some more oil if needed and gently poor the omelet mixture into the pan so that the mushrooms are spread well. Gently fry the omelet on medium heat, turn it if you feel like it. When it's golden brown and completely solid you can eat.

(all the amounts in this recipe are wild guesses as usual, I don't believe in careful measurements when cooking)

Posted via web from Koken en Eten

Note to self: how to sparsely check out a git repository

2010-06-14T02:15:00.001-07:00

Today I discussed with Dave Syer about some improvements in Spring Integration, which I had sandboxed in my messy github repository. A sparse checkout would be convenient, but we both didn't know how to do it. I even thought git didn't support it, but boy was I wrong!

iwein:si-sandbox$ git init

Initialized empty Git repository in /tmp/si-sandbox/.git/

(master) iwein:si-sandbox$ git config core.sparsecheckout true

(master) iwein:si-sandbox$ echo message-store/ >> .git/info/sparse-checkout

(master) iwein:si-sandbox$ git remote add origin git://github.com/iwein/Spring-Integration-Sandbox.git

(master) iwein:si-sandbox$ git pull origin master

The important thing to remember is that this also works with git-clone. Your index will still have everything, just your checkout (working copy generated from the index) will not. This is a big difference with svn sparse directories, and as usual it's an improvement.

Posted via email from iweinfuld's posterous

Using negative feedback to prevent disaster in Software Development

2010-06-13T05:03:00.001-07:00

Invariably we are taught that when working with people we should give them plenty of positive feedback whenever they do something right and be very careful with negative feedback. I've never completely bought into this hugging hippie attitude, but it seems there is a good point there: if we're all nice to each other and keep a positive spirit, we get more done right? The problem with this approach is that it doesn't take into account that it might be important to not get something done. Some ideas are just terrible and they should be killed off as soon as possible.

When I had the idea for this blog I thought it would fit on twitter. "Positive feedback induces brilliance, negative feedback prevents disaster. The latter is sadly undervalued." i twote. It seems this could use a few backing arguments and today I've made time to provide them. I will explain why negative feedback is important and how you can apply it to your advantage.

What's the problem with positive feedback?
Positive feedback induces brilliance. Like the brilliant glow of a nuclear meltdown is caused by positive feedback. If you're doing something absolutely great, positive feedback will cause you to do more of it and it will cause others to try and emulate your behavior. Brilliant right? But what if you're doing something that is ever so slightly off target? Will positive feedback make you augment your course? Will it make others slightly vary your behavior before emulating it? I don't think so. You need negative feedback to change.

I have a background in physics and some electronics and signal analysis is part of that background. In electronics positive feedback doesn't have the credits that it has in social studies. It is viewed as a destructive force that will ruin your signal, make your appliance useless, or even destroy things. Let's look at the example of a feedback loop in it's most generic form:

Download now or preview on posterous

PastedGraphic-1.pdf (18 KB)

In the diagram you see input being processed by a system G the output of G is connected to its input via a feedback loop. If the feedback is negative or zero, the output will be strictly related to the input. If the input drops, the output goes to zero. If the feedback is positive two things can happen: first, if the feedback is less than the input, the output will eventually go to zero, but it takes more than one loop. In audio this is called an echo. It's not dangerous, but it can pollute the output if the echo is of a sound that you didn't want to hear. If the feedback signal is stronger than the input, when the input drops the output will continue to increase. In audio this is called a feedback loop, the typical symptom is a high frequency tone that gets louder and louder until either the speaker gives or the amplifier can't go any louder. This is not considered a good thing generally.

If we apply this model to human interaction, it still works. Let's say the input is an idea by developer Joe, G is the team. Let's say the idea is to improve quality by adding more test cases. Let's say the idea pays off the first loop and everybody is very happy. This causes some serious positive feedback. The feedback will say: "More testing is good". And more resources will be invested in testing. Depending on the positive attitude of the team this will lead to more and more resources being spent on creating tests, until there are no more tests to write or the team is out of resources. As much as I like tests, that's crazy. So developer Bill will come with the great idea to spend less time on testing. The first loop this will get positive feedback because the team gets stuff done again and the tests are in place to keep them safe. Two loops later we're in the hole again, quality is crap, coverage is abominable, we're stuck. Rinse and repeat. Does this really happen? You bet!

There is a dangerous psychological phenomenon that is entirely caused by positive feedback in groups, dubbed Group Think

The reason that people fall into this trap in my opinion is that they have the instincts to give positive feedback and act on it. Also our education system is based on it because it works well on small children. But in fact getting only positive feedback makes you lazy and stupid. What can we do? Simple: introduce negative feedback.

I thought negative feedback was bad?

If negative feedback means bashing then yes, it's bad. If negative feedback means using force, it's bad. If negative feedback is used to block others, it's bad. But it doesn't have to be that way. In the case were Joe suggested testing the first iteration good things happened, but then it spun out of control.

Download now or preview on posterous

PastedGraphic-5.pdf (18 KB)

The team lost track of the goal: working software and started focussing on some symptoms like test coverage.

In the next image the green arrow are the actions the team is taking, the red arrows are the negative feedback that is applied to reach their goal instead of overshooting it.

Download now or preview on posterous

PastedGraphic-4.pdf (19 KB)

The important thing is not to stop the team from doing what it's doing, just to stop it from doing it too much. Make the team do it better instead of more. In some cases a really terrible idea comes up and no time should be wasted on it. In those cases people usually have no problem killing off the idea. The problem arises when a good idea comes up but the implementation is not completely right. It's only human to want to move forward when you're going strong and that makes it very easy to overshoot the goal entirely. The trick is to gradually apply more corrective negative feedback once the team gets closer to a goal.

Thinking about decision processes using this metaphor works for me. I hope you find it useful too.

Posted via email from iweinfuld's posterous

How to build Spring Integration from the sources

2010-05-27T00:02:00.001-07:00

This is a guide on setting up your machine for Spring Integration development. I've got questions about this many times so I thought it was time I put the answer where Google could see it. Let's say you want to create a patch for a bug in Spring Integration, or you want to run a test case to see how some component works. First you need to take that annoying hurdle of getting the project to build locally. You're going in the right direction, but you've missed some turns. If that sounds familiar, read on!

Before you go into the IDE part, make sure you have got the basics right. I can't stress this enough, if you can't get it to work from the command line you're doomed, no tool can help you. So the 5 first steps don't involve an IDE.

Prepare your system for Spring Integration development
0. Forget about your IDE and open up a terminal (on Windows there are some extra steps).

1. make sure you have a command line svn client (there are many installation procedures) the end result should be:
(master) iwein:si$ svn --version
svn, version 1.6.11 (r934486)
compiled Apr 19 2010, 23:04:06
Any version later than 1.6 will do I think.

2. make sure you have the right version of Java:
(master) iwein:si$ java -version
java version "1.6.0_15"
Java(TM) SE Runtime Environment (build 1.6.0_15-b03-219)
Java HotSpot(TM) 64-Bit Server VM (build 14.1-b02-90, mixed mode)

This might be a bit confusing and I want to avoid the details, but you need to remember that Spring and also Spring Integration are compatible with Java 1.5, but the build is not necessarily. 1.6 is fine.

3. make sure you have the right version of Maven:
(master) iwein:elmar-trunk$ mvn -version
Maven version: 2.0.9
Java version: 1.6.0_15
OS name: "mac os x" version: "10.6" arch: "x86_64" Family: "mac"

Any 2.0.x version will do (except the ones with bugs :) ).

4. check out the trunk:
iwein:si-temp$ svn co https://src.springsource.org/svn/spring-integration/trunk/
This will give you a lot of output, but no errors.

5. Build the project:
iwein:si-temp$ cd trunk/
iwein:trunk$ mvn clean install
Again a lot of output, but no errors

6. Import the project in your IDE
Using the power of google you can find guides for:

It's a matter of taste how you do it, and if these fancy IDE's are not for you there is always vim. Point is that as long as you make sure everything works from the command line, the IDE support is just a convenience that you could live without (theoretically).

Other options
If you don't like Subversion, you can also use git. Make sure you have git and git-svn installed and then run:
git svn clone https://src.springsource.org/svn/spring-integration/trunk/ -r HEAD
If you don't know Subversion yet, I'd recommend this option.

If you are bored with Maven you could also check out the Spring Integration gradle build by Jeroen van Erp. Using a fork makes you dependent on Jeroen for porting commits on the Subversion HEAD, or your own merge skills, but the coolness of gradle might very well be worth it.

Let me know if you run into trouble or if you think I missed a step. Happy hacking!

Posted via email from iweinfuld's posterous

Simplifying Spring Integration testing

2010-05-16T05:23:00.003-07:00

It seems like last week that I would happily write things like "public abstract class BaseMyCompanyTest extends AbstractTransactionalDatasourceSpringContextTests {" and think I was the man. It seems like yesterday that Spring 2.5 came along and made things a lot easier with @ContextConfiguration. But now I find myself grinding against the boilerplate again.

Before I start tearing it down let me first tell you that I highly respect the work by Sam Brannen and that I have gotten very good mileage out it for over two years now. There is always room for improvement and that is the aim of the following experiment.

The TestContext framework that comes with Spring might be very useful, it is also quite complex. You can only use it with a custom testrunner and and you need to pass the location of the configurations to be loaded in a class level annotation. This means it is non trivial and asymmetric to load multiple contexts in a single test. Also the incantations you have to put on the class are not exactly concise (although they are much better than the AbstractTooFreakinLongClassNames options of old).

Since JUnit 4.7 there is @Rule. And you can do some pretty cool stuff with it. When combining it with Spring though I wasn't impressed. I felt that Spring Test was making it hard to use the power of Rules. This is ironic, since it is just this JUnit feature that would make Spring Test a lot better.

How cool would it be if you could write:

1 2 3 4 5

  @Rule
  public TemporarySpringContext context = new TemporarySpringContext("context.xml");

  @Autowired
  ApplicationContext thisShouldBeWired;

And it would just work?

Gues what, I've got a green test that says it does! It only took me 50 lines of code, which you can find in my spring sandbox on github.

The important class is this the TemporarySpringContext (which I might give a better name soon):

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33

public class TemporarySpringContext extends TestWatchman {
  /**
   * Cache of Spring application contexts. This needs to be static, as tests
   * are typically destroyed and recreated between running individual test methods.
   */
  static final ContextCache contextCache = new ContextCache();

  private ConfigurableApplicationContext context;
  private final String[] contextLocations;

  public TemporarySpringContext(String... contextLocations) {
    this.contextLocations = contextLocations;
    try {
      context = contextCache.contextForLocations(contextLocations);
    } catch (Exception e) {
      throw new RuntimeException(e);
    }
  }

  @Override
  public Statement apply(Statement base, FrameworkMethod method, Object target) {
    context.getAutowireCapableBeanFactory().autowireBean(target);
    return super.apply(base, method, target);
  }

  public ConfigurableApplicationContext getContext() {
    return context;
  }
  
  public void dirtyContext(){
    contextCache.markDirty(contextLocations);
  }
}

Because it extends TestWatchman you can hook into al the phases of your test by simply overriding a method. Because it is just a MethodRule field you can add different contexts to your test class. There is plenty of room to polish this, but it has more potential than the SpringJUnit4ClassRunner I think.

Since most of the heavy lifting is still being done by Spring Test behind the scenes by the way. The ContextCache is the only complex part (yet) but I have some tricks to pull to make @Transactional and such work. If you fix it before me I owe you a beer :)

Posted via email from iweinfuld's posterous

Excuse me, your fly is open

2010-05-16T05:23:00.001-07:00

After I showed a colleague a trick in my IDE he told me to write a blog post about it. I told him to write it himself. This all happened almost a year ago. The problem with this post is that it's about something that is a bit embarrassing. I'm losing my appetite for talking about it, but yet, I come across more setups that are wrong than setups that are right. This post is mostly about some settings in Eclipse (and also in IntelliJ IDEA) that are pretty moronic and that you should change asap to prevent bugs in your code. Otherwise you might be seen with an open fly and you wouldn't want that would you?

First a word on keyboard shortcuts

First of all: learn to use your tool! Here's a list of things that I see people wasting effort on:

- click through the menu (unless you're trying to find what the keyboard shortcut is you're looking for)

- declare a variable or field (use extract variable instead)

- type the skeleton of a method (your IDE can do that much faster using a quick fix)

- create a new type other than a test case yourself (generate everything from a test case, it's faster, trust me on this one)

There are many more variations on the same theme, the last point is arguable, some people are really fast if they start to hash out the model before beginning a test. My point is that it pays off to find the shortest path to a certain solution and figure out what shortcuts get you along that path fastest. The best way to train yourself to use the keyboard is to stick to the first point. There is even an eclipse plugin that can enforce keyboard usage (but that's a little too harsh for my taste). There are many blogs describing the keyboard shortcuts in great detail as well.

Templates

When you stop using your advanced typing skills and let the IDE do the typing for you, you will have moved on to using templates. In eclipse, if you look for templates under preferences (just type templates in the filter).

You will find quite a few templates that are useful, some that you'll never use and the best news is that you can create your own too. I can recommend renaming the JUnit4 template to tst (instead of the default Test in eclipse) and tweaking it a bit to your coding standards, but that's not why we're here now.

Not all these templates are sensible, and that is what I want to to fix.

New method body

A new method you can create by just trying to use the method and then asking Eclipse to create the missing method for you.

You'll end up with a method like this:

 1 2 3 4 5 6 7 8

public class Reverser {

	public void reverse(String string) {
		// TODO Auto-generated method stub

	}

}

Which is pretty bad because it will allow your code to run without complaining and leave a crappy piece of comment in there. It's not the end of the world though, and you'll notice later right? You probably won't, but you don't have to, more on that in a minute. First let's make the method return a String.

Stop, don't start modifying the method, just delete it. Eclipse will generate it correctly for you later.

Modify the test a bit so you need a String return value and try the quick fix again.

Now Eclipse gives you this:

 1 2 3 4 5 6 7 8

public class Reverser {

	public String reverse(String string) {
		// TODO Auto-generated method stub
		return null;
	}

}

This is not only pretty bad, its retarded. Just run the test case and see for yourself. The only plausible result of this will be a NPE, and the NPE will happen in another class so you have to retrace your steps to find out where you need to fix it. I want my test case to throw me a stack that starts exactly where I need to continue developing. Even worse, if you're generating methods that are not invoked directly from a test case, this might lead to NPE's flying around in production. This will not do.

So what can we do? Well, it's simple: just modify the template responsible. This template is called 'Method body' and should edit it to look something like this.

Now if you delete the generated method again and regenerate it, it should look better.

 1 2 3 4 5 6 7 8

public class Reverser {

	public String reverse(String string) {
		throw new UnsupportedOperationException(
				"Iwein left this implementation as an exercise to the reader");
	}

}

If we run the test case now it will fail. But the big improvement is that it will fail correctly. What I mean by failing correctly is that the failure actually points out what is wrong: you have failed to implement a generated method. This is infinitely more informative than an NPE. As a bonus you can click the stack trace in the test and actually go to the method that you need to implement.

Just double click the line that is selected in the above screenshot. Pretty cool huh?

Catch blocks

There is one more template I want you to modify. Check out the 'Catch block body' template. This is the template that gets used when you use the 'Surround with try/catch' quick fix.

The resulting code looks like this:

 1 2 3 4

		} catch (Exception e) {
			// TODO Auto-generated catch block
			e.printStackTrace();
		}

I can tell you that there is plenty of code like that running in production systems. Granted, checked exceptions are hard to deal with, but surely we can do better than this.

This isn't going to make your program run any better, but it sure is going to make it fail better. And that, dear reader is what this post is all about. If things fail early with an informative message, you (and those that come after you) are going to spend less time in a debugger and more time fixing actual problems.

I've used Eclipse for the screenshots, but I can assure you that on my IDEA setup I have done the same fixes, and it wasn't very hard to figure out either. So now that you've read this there is no excuse to carelessly return null or print a stack trace anymore, even if you're not using Eclipse.

Don't be caught with an open fly again!

Posted via email from iweinfuld's posterous

Combining TemporaryFolder @Rule with Spring 3

2009-10-11T00:58:00.000-07:00

I like TDD, I like Springs @ContextConfiguration framework and I like JUnit 4.7's new Temporary folder. Getting the latter two to play nice isn't as easy as I hoped, so here's an attempt at a way to make them work.

Let's say you need to configure an incoming directory on a component using external configuration. For example in Spring Integration:

<file:inbound-channel-adapter directory=#{config.directories.store} ... />.

I'm using expression language to load a property from a config object, which I wire in turn as a Spring bean. In my main application this config object can just be a for example, and in my test I can override that bean with an inner class from my test.

public static class Config {
public static final Map directories = new HashMap();
}

If you then use the containing context in a JUnit test it will be loaded up once the context is created, so you need to make sure to put the store value in the map @BeforeClass. Now I'd like to use a TemporaryFolder, to store files in the input directory of the channel adapter, but you can't refresh the property of a singleton bean each test. The @Rule will be applied before each test, resulting in the second test using a different directory from the context. A shame.

A working solution is to just remove the @Rule annotation from the TemporaryFolder and manually invoke create() and delete(). I'd like to know if there is a better solution.

Maven sample project can be found on GitHub.

If you give it a go, let me know if you find a neater option!

Spring Milestones and Snapshots with maven

2009-09-04T04:28:00.000-07:00

It took me way longer than I wanted to set up a simple maven project using the Spring M4 and the latest snapshot of Spring Integration. For some reason this is not documented prominently on springframework.org. I commit on Spring Integration, so maybe I should do something about this, on the other hand, blogging is easier... I'll hand you the fix that still worked for me in 2009.

I'm a bit of a maven rookie. I've been using it for years now, but I still feel uncomfortable using it and I have given up on ever having a complete grasp of how it works. The general idea and functionality I understand and sympathize with, but other than that it's too painful to recommend. On the other hand, I just want my build to work and learning Gradle isn't on the top of my list if I need to get started now. Maven itself is tolerable but the plugin and repository landscape isn't.

I'm just going to save you 45 (if not more) minutes of searching and cursing before you find hints on the project pages.

Don't use the search tools for maven central (I like mvnrepository) for Spring dependencies.
Don't use SpringSources enterprise repository for milestones and snapshots. They might be there, but they end up there later than on S3.

There is a milestone and snapshot repository in S3, as described by Ben Hale in a blog. Use the repository snippets from that blog (also included here). Look carefully at the url of the s3browser.

Do use http://s3browse.com/explore/maven.springframework.org/

The repositories are in s3, and while that is very cool, it's also a pain for you because you can't explore them with the url copied from the repository element. Keep a comment on that in your pom if you have colleagues.

<repository>
 <id>spring-milestone</id>
 <name>Spring Portfolio Milestone Repository</name>
 <url>http://s3.amazonaws.com/maven.springframework.org/milestone</url>
 <!-- browse via http://s3browse.com/explore/maven.springframework.org/milestone -->
</repository>

<repository>
 <id>spring-snapshot</id>
 <name>Spring Portfolio Snapshot Repository</name>
 <url>http://s3.amazonaws.com/maven.springframework.org/snapshot</url>
 <!-- browse via http://s3browse.com/explore/maven.springframework.org/snapshot -->
</repository>

If you need artifacts from S2's enterprise repository too you can end up with a whopping 6 repository snippets just for your spring artifacts. Keeping track of which artifact comes from which can be daunting. If you're in the position to do so, using Nexus (or similar) can be used to mitigate that.

There are a lot more details to all this (related to OSGi and maven versioning schemes), but if you're just interested in making it work, you should be grand now.

Hope this helps.

Push vs. Pull or Why company mailing lists are evil

2009-01-10T03:20:00.000-08:00

Everybody that works for a large organization knows the semi Observer anti-pattern. In some form or another at least. It works like this: The Big Boss thinks that you need to know, so he pushes information to you through email. He is too lazy to push this corporate spam to all personnel individually so you get added to a list automatically. At least some of your bosses don't know that there is such a thing as BCC so soon all personnel knows all the lists they are on.

The next step is that people start holding discussions using the email addresses for those lists. I've seen these types of discussions spiral out of control in different firms I've worked for. This causes unreasonable load and distraction for the employees and can even cause the whole email system to collapse under the strain. I can tell you this is not the first time something like this has happened.

What makes this behavior unforgivable, is that there is such a simple solution.

Use RSS or newsgroups to let people subscribe to replacements for these lists and clearly separate directed communications (to a few people) from news and open discussions. By using this simple solution (and working for a company that actually does this right), I've managed to reduce my incoming email by 80%. It is still some work to go through all the mails after a few days of inactivity, but the thing is it is not absolutely impossible anymore. The other immensely satisfying improvement is that most mail I get I actually want to read.

I do have thousands of unread items piling up in my RSS reader and nntp client, but those I know I can safely discard when they get stale. They don't eat up disk space and I can always search them.

There are reasons to keep lists in the company anyway. For example some employees are too lazy or stupid to subscribe to RSS lists and newsgroups, if you are legally required to send out a communique an email push might come in handy. In those cases: use BCC. If something should not be done, make it impossible. Guidelines for Reply-All and the pleas and threats to follow them are just more unneccesary noise.

So bosses all over the world, if you read this: be smart and stop spamming your employees, they hate you for it and they're right.

I saw another complaint about many emails in general. I bet that this guy is subscribed to his own blog using email. He ends his post with:

If you are the person with the idea to save us all, send me an email and tell me all about it. Actually, strike that. Drop by my house and tell me all about it. I don’t want your message to get lost in my inbox

I'm not a venture capitalist, but still the difference is striking: my mailbox is completely read, and I spend less than an hour a day to keep it that way.

Syncing calendars... nothing is ever easy

2008-10-12T01:07:00.000-07:00

I have a problem of managing multiple callendars. My wife has access to my Google calendar, my company has access to my Zimbra calendar. I use a Mac, and I like to use iCal if possible. I use Plaxo, TripIt and Dopplr, which provide ical interfaces/synchronisation, just like Google and Zimbra.

All this shouldn't be too original, so I expect that other people have this problem. Granted it is not an easy problem to solve, because synchronisation isn't easy. Let me first explain how I'm managing things now (and how I screw up). Then go over the options and then over what I really want.

Now I manage my calendars through CalDav. This works really well for a single calendar, but it does nothing for you in terms of synchronisation. If my office manager add's an appointment I see it show up in the right calendar, if I get a trip confirmation and send it to TripIt same story if my wife makes an appointment I can see it show up too. Nice overview, clear separation of calendars. Perfect, as long as I do the managing. That is the problem, my wife manages the calender of our private life, my manager manages the calender for the jobs. They have no idea what the other is doing.

So I screw up like this: if my manager plans for me to be abroad he checks with me and puts it in the calendar, then I have to make sure it shows in my private calendar that I'm abroad, otherwise my wife will plan something conflicting in the home calendar.

The underlying problem is that I am just one person, so everybody planning for me to be somewhere needs visibility on where I am. Main options:

Be more disciplined and copy things over
Use a single calendar and synchronize

The first option is what I'm doing now. Fine tuning this would mean I create a single calendar that lists my availability and make sure that all planners have read access to that from a single interface that they use for planning. The advantage is that this way the planners do not have to worry about fine grained appointments, just about available time (in a certain location).

The second option would allow me to be very lazy, but it would also result in a privacy issue where my manager can see that I went to a party on thursday evening or something. I don't care that much about privacy. I do care about my manager moving around my dentist appointment from the private calendar, so the solution is not ideal.

An ideal situation would allow me to share availability across calendars, and restrict write access and access to detailed information based on different calendars/sets of appointments.

The annoying thing is that there are no tools that allow this. Requirements are:

Have to use Zimbra for work
Have to use Gcal for private
Want to use iCal as interface
Want to integrate CalDav calendars
No manual syncing

The only way to get close would be to install different sync utilities for my mac and sync with a local calendar. The proper way to do it is to allow CalDAV calendars to be synced in ical, but there is no support for this in iCal natively. The only workable solution (other than discipline) is to run sync utilities like Spanning Sync and the Zimbra iCal plugin. This is bound to screw up my calendars at some point I'm sure...

Update svn on Leopard to play nice with subclipse

2008-08-23T01:03:00.000-07:00

Not so long ago I moved forward to STS 1.1 which is based on Eclipse 3.4 Ganymede. Of course I needed a svn plugin and then I ran into this issue.

Mac OSx packages an old version of collabnet subversion (1.4.x). And that doesn't play nice with subclipse. The fix suggested there is to use an old version because:

For Windows users, it is a simple matter of installing the new Subversion 1.5 client and making sure JavaHL is on the path. For Unix users, this workaround obviously defeats the purpose of having a distro with package management and requires manual installation of the libraries in the correct locations.

If you truly understand that and it is a blocking concern for you (notice the hint of sarcasm), don't read on. There is however a very simple solution to run the right version and just make it work.

Download and install http://www.collab.net/downloads/apple/download.html
sudo rm /usr/bin/svn* (you might want to check if you haven't got any other programs named svn* before you do this)

The second step is needed because the collabnet package installs simlinks into /usr/local/bin and the programs in /user/bin take precedence.

Once this is done you're good to go, and with the latest versions.

Update: I've moved on to do this with MacPorts, lately their svn versions are fine, and the installation is much simpler.

Correct Simple Singletons

2008-08-20T00:43:00.000-07:00

This is going to be a very short blog. It's not about concurrency. It's about avoiding to think about concurrency. As was the original intent of this blog it is about being lazy, but still not being wrong.

Yesterday I read a very old blog while digging through the web for reasons to be scared of the volatile keyword. I found plenty, but this one is a gem: http://blogs.msdn.com/brada/archive/2004/05/12/volatile-and-memorybarrier.aspx. It's about .Net code, but the basic problems also apply in Java. The link to http://www.cs.umd.edu/~pugh/java/memoryModel/DoubleCheckedLocking.html is worth a read too.

If you can think about concurrency but you don't want to do it every day like me adhere to these rules when writing singletons:

When you plan on using the singleton write:

public class Singleton {
  private static final instance = new Singleton();
  private Singleton() {}
  public static Singleton get() {
    return instance;
  }
}

When you don't plan on using the singleton, but you still feel the urge to implement it write:

public class Singleton {
   private static instance;
   private Singleton() {}
  public static synchronized Singleton get() {
    return instance;
  }
 }

When you're not going to use the singleton a lot, it is probably more memory efficient to just use a disposable instance, hence removing the need for the singleton at all. So the recipe goes like this:

Don't use a singleton if you don't need to
Use the simple (static final) variant if you do need a singleton
Use the synchronized variant if you are intent on going on the downward slope
Of course you are special, so go on and do it the hard way.

Don't repeat yourself, just quote yourself.

2008-07-09T10:18:00.000-07:00

I just read an article by Dan North about how the DRY principle doesn't work with Test code. Obviously I agree with him, because it is almost trivial once you've figured it out on your own. I already met many people that did and were quite happy with themselves for it, actually that could have been me after I had about 1 year of experience with TDD.

When I read the linked blog I mused over the main counter argument against dropping DRY, it is actually the argument to adopt it: what about the maintenance hell of all that repeated code. Arguably if you use proper mocking or stubbing (especially mocking) you shouldn't have too much trouble with this, but wouldn't it be nice if there was a solution for this?

So what are the requirements:

We want to get readable code (not requiring you to constantly switch source files or scroll)
We want little writing effort (not repeating the same code snippet over and over)

Requirement 2. is covered by OO, AOP and friends, but they completely ruin 1. in certain scenarios (e.g. test code). To get 1. you will have to drop 2... NOT good enough!

I am a fan of wikis and as you might know many have a feature called templates that specifically targets these two requirements. How about template in code. Just add a template construct and let the IDE optionally show the actual code inline.

I like it.

My first mobile internet

2008-03-25T04:48:00.000-07:00

I've just replaced my well over 4y old battle scarred Nokia 6100. It's not that it was really broken, but just that my definition of 'the basic necessities' has changed. Let me give an overview of the definition over the years:

In 1998 when I bought my first mobile phone:

Be able to make a call

In 2004 when I got my third phone:

Be able to make a call
Be able to use sms
Battery life at least 1 week/4h calling

In 2008:

Be able to make a call
Be able to use sms
Be able to connect my laptop to internet
Battery life 1 week

Now in 2008 I have a job that requires me to travel and I "need" to have internet at all times. I decided to do what operators have been blabbing about for 6 years now and combine my phone needs with my internet needs. We do have UMTS now so it should be OK, right? Right...

To put things into perspective I will just give some hints to how my 'basic' internet needs are pretty far advanced compared to my phone needs. I for example:

expect ANY website to render in under 0.5 seconds
routinely download files over 300M and don't expect to wait longer than it takes for me to get coffee.
store my complete life (contacts, documents, ...) online and don't expect to notice the difference.

So I got myself a unlimited data subscription and a UMTS capable phone (SE W660i if anyone cares). And I started connecting it to the internet.

Many geeks will look pitiful at my incompetence, but it took me 4 hours to get things working. Kudo's to http://www.taniwha.org.uk/ and the guys from SE for doing my providers and Apple's work for them. Funny that the guys that get the most money from me (Apple) are the least helpful in getting things to work properly. Ah well, I should have gotten an iphone, but that doesn't have umts... anyway, that's a whole different rant.

Now that everything works I am just curious at the speed I can get out of my new toys (http://text.dslreports.com):

40 kbit/sec
1.978s latency

Erm.. Those guys from telfort are really missing the point here I think. WTF!

When I made webpages last millennium the general rule was to keep images under 100k so that with this speed it would take me, say 20 seconds PER IMAGE to load the page.

I looked at http://www.umtsworld.com/technology/dataspeed.htm and the theoretical maximum is still pretty disappointing...

It's 2008 dammit, where is my flying car!

Comparing beans or why equality is subjective

2007-05-22T04:28:00.000-07:00

In my project I use Hibernate and DWR so between my database and application I have to deal with Hibernate fiddling with my objects and between my client (browser) and application I have to deal with the DWR.

This all seems well and fine and generally it is. In fact kudos are in order.

However, there is no need to list all the key features again, you can just check the websites of the products for that. What I would like to describe is the fundamental problem of comparing objects that these frameworks do not solve.

The essence of the problem is that in a pure object oriented environment all objects have identity (== in Java) and when you go beyond the boundaries of your environment that law doesn't apply anymore. In other words: once
your application let's go of an object, that object is gone forever. To illustrate an example is appropriate.

Say in my application I have some simple user management. A User object can be persisted to the database and retrieved when a user logs in. The user can edit his profile (user properties) in the client and since I'm using Ajax i don't want the user to have to reload and redo his changes if I restart my application.

In real life each user is unique, so I expect the same from my application, database and client. Sounds simple, but it isn't. First let's have a look at the hibernate persistence layer. In the relational database there is no identity by default (it is impossible to distinguish two records with the exact same columns). This is easily fixed by adding a primary key to the table. Hibernate can do this for you. You just have to put some sort of Id property on your Pojo and tell Hibernate to use a sequence to fill in this property once the object is saved (eww).
Now there are three ways to obtain an object:

create it yourself
get it from hibernate
get it from somewhere else (the client)

In the first case the Id will not be initialized (obviously, because Hibernate is in charge of that). In the second case the Id will definitely be initialized, otherwise the pk constraint would be violated. The third case is arbitrary. If the client got the object from the application in the first place there is a good chance that it has been saved before. In that case the client better keep track of the id. If the client is trying to create a new user there will not be an id.

Actually it is even a bit more complicated, as I might want to allow the application to pass objects to the client before they are saved. In this scenario the client and the application need to be able to negotiate identity between objects without polluting the id property that Hibernate needs to fill once the object gets saved. (If you get lost here it is probably a good idea to read up on Hibernate and the equals() method.)

Now take a note that I have been talking about identity and equality. Equality is the functional identity (i.e. two distinct objects in the jvm could be representing the same object from the users perspective).

If a user loads his User object in the client and after some changes he hits the save button, the application needs to be able to see that the existing User with a certain identity should be saved (resulting in a database update). You could establish this behavior with a smart implementation of the equals() method.

It is important to make a clear distinction between functional equality and technical identity. To illustrate suppose the application is a friends network. One user is logged in and updating his details. Another user is logged in and adding the first user to his network to do this he fills in the user form, the client sends the new user object to the application. The application needs to check if the user is equal to a known user to prevent duplication. Also there need to be two different instances of the first user. The identity of these user objects needs to be managed over the DWR pipes.

Now we have two objects in our application that are functionally equal, but technically distinct. This exposes the problem I'm trying to get at. Since the two objects are equal, but not identical either updates done in the first session do not show up in the second or you would have to constantly manage conflicting changes when either user saves the object.

Elegant solutions are most welcome.

What happened to LOP?

2007-04-25T01:48:00.000-07:00

Wouter attended me to this interesting article by Sergey Dmitriev. Blunt axe summary: Language oriented programming will allow the programmer to naturally create DSL's as the need arises. Using the OO paradigm where applicable, using other paradigms in other domains. This will keep the code look more like the design and less like the unreadable crap that we normally write. Be honest: your beautiful creations are not exactly poetry to the untrained eye, right?

There are some more articles about it primarily the one by Martin Fowler, but nothing new really since 2005.

My theory is that getting an application (damn thing) to work it is just too much fuss to use something like MSP. The potential maintenance hell will lead managers to the conclusion that sticking to skills that they have sent their programmers to trainings for will be better. Who hasn't been to that design meeting where somebody started by drawing a very familiar picture on the whiteboard and said: "This is the service layer, this is the business layer, this is the data layer ...."? After that meeting everybody takes their template design documents, SLA's, status updates... and starts filling in the name of the project.

Creativity should be done by humans, filling in templates should be done by computers. But in the end humans tend to favor familiar territory. To be perfectly honest, most of my hours are spent doing stuff that should be automated.

Anyways, it would be nice if this LOP made it's way into mainstream software development. Time to polish up those true programming skills and forget about the compiler in the back of your head that you shouldn't rely on anyway.

Subclipse annoyance

2007-04-13T06:07:00.000-07:00

To help the unwary that will follow in my footsteps:

How to change the stored credentials for a subversion repository in Eclipse (using Subclipse)
Subclipse is a wonderfull tool that helps a bunch when working with eclipse on a svn repo. However there are annoyances. This one i lost an hour on today.

Imagine that you have a friend that starts working on your machine and checks out a project. While he's working he doesn't want to provide credentials on every commit so he checks the save-pass checkbox. Now you have a project on which all commits get done as his user. Now you want to continue working on the project after he's gone. But you don't want to assume his identity. You need to remove/change the stored credentials. And if possible you don't want to do it with a nice button or option in the menu... Don't get your hopes up :/

So you try:

fiddling with the properties of the project -> no luck
fiddling with the properties of the location in the repository explorer -> no luck
deleting the repository and recreating it
deleting the project and recreating it
combinations of 3 and 4 laced with power cycles -> no luck

The problem is hinted at here http://svn.haxx.se/subusers/archive-2006-08/0030.shtml.

Another problem you're experiencing is that subclipse uses JavaHL or SVNKit for the connection. These adapters are conveniently storing credentials for you. Subclipse doesn't care and doesn't provide an interface to influence this.

If you don't want to look for the config files and "fix" it from the comfort of eclipse. You can around this by just switching interfaces from Window->Preferences->Team->SVN
(Otherwise just go on and fiddle with the authentication files as described in the link)

Well, how's that for a bit of useless information?