How to build Spring Integration from the sources

This is a guide on setting up your machine for Spring Integration development. I've got questions about this many times so I thought it was time I put the answer where Google could see it. Let's say you want to create a patch for a bug in Spring Integration, or you want to run a test case to see how some component works. First you need to take that annoying hurdle of getting the project to build locally. You're going in the right direction, but you've missed some turns. If that sounds familiar, read on!

Before you go into the IDE part, make sure you have got the basics right. I can't stress this enough, if you can't get it to work from the command line you're doomed, no tool can help you. So the 5 first steps don't involve an IDE.

Prepare your system for Spring Integration development
0. Forget about your IDE and open up a terminal (on Windows there are some extra steps).

1. make sure you have a command line svn client (there are many installation procedures) the end result should be:
(master) iwein:si$ svn --version
svn, version 1.6.11 (r934486)
   compiled Apr 19 2010, 23:04:06
Any version later than 1.6 will do I think.

2. make sure you have the right version of Java:
(master) iwein:si$ java -version
java version "1.6.0_15"
Java(TM) SE Runtime Environment (build 1.6.0_15-b03-219)
Java HotSpot(TM) 64-Bit Server VM (build 14.1-b02-90, mixed mode)

This might be a bit confusing and I want to avoid the details, but you need to remember that Spring and also Spring Integration are compatible with Java 1.5, but the build is not necessarily. 1.6 is fine.

3. make sure you have the right version of Maven:
(master) iwein:elmar-trunk$ mvn -version
Maven version: 2.0.9
Java version: 1.6.0_15
OS name: "mac os x" version: "10.6" arch: "x86_64" Family: "mac"

Any 2.0.x version will do (except the ones with bugs :) ).

4. check out the trunk:
iwein:si-temp$ svn co https://src.springsource.org/svn/spring-integration/trunk/
This will give you a lot of output, but no errors.

5. Build the project:
iwein:si-temp$ cd trunk/
iwein:trunk$ mvn clean install
Again a lot of output, but no errors

6. Import the project in your IDE
Using the power of google you can find guides for:

• importing a maven project in IDEA
• using m2eclipse
• working with maven projects in NetBeans

It's a matter of taste how you do it, and if these fancy IDE's are not for you there is always vim. Point is that as long as you make sure everything works from the command line, the IDE support is just a convenience that you could live without (theoretically).

Other options
If you don't like Subversion, you can also use git. Make sure you have git and git-svn installed and then run:
git svn clone https://src.springsource.org/svn/spring-integration/trunk/ -r HEAD
If you don't know Subversion yet, I'd recommend this option.

If you are bored with Maven you could also check out the Spring Integration gradle build by Jeroen van Erp. Using a fork makes you dependent on Jeroen for porting commits on the Subversion HEAD, or your own merge skills, but the coolness of gradle might very well be worth it.

Let me know if you run into trouble or if you think I missed a step. Happy hacking!

Posted via email from iweinfuld's posterous

Simplifying Spring Integration testing

It seems like last week that I would happily write things like "public abstract class BaseMyCompanyTest extends AbstractTransactionalDatasourceSpringContextTests {" and think I was the man. It seems like yesterday that Spring 2.5 came along and made things a lot easier with @ContextConfiguration. But now I find myself grinding against the boilerplate again.

Before I start tearing it down let me first tell you that I highly respect the work by Sam Brannen and that I have gotten very good mileage out it for over two years now. There is always room for improvement and that is the aim of the following experiment.

The TestContext framework that comes with Spring might be very useful, it is also quite complex. You can only use it with a custom testrunner and and you need to pass the location of the configurations to be loaded in a class level annotation. This means it is non trivial and asymmetric to load multiple contexts in a single test. Also the incantations you have to put on the class are not exactly concise (although they are much better than the AbstractTooFreakinLongClassNames options of old).

Since JUnit 4.7 there is @Rule. And you can do some pretty cool stuff with it. When combining it with Spring though I wasn't impressed. I felt that Spring Test was making it hard to use the power of Rules. This is ironic, since it is just this JUnit feature that would make Spring Test a lot better.

How cool would it be if you could write:

1 2 3 4 5

@Rule   public TemporarySpringContext context = new TemporarySpringContext("context.xml");   @Autowired   ApplicationContext thisShouldBeWired;

And it would just work?

Gues what, I've got a green test that says it does! It only took me 50 lines of code, which you can find in my spring sandbox on github.

The important class is this the TemporarySpringContext (which I might give a better name soon):

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33

public class TemporarySpringContext extends TestWatchman {   /** * Cache of Spring application contexts. This needs to be static, as tests * are typically destroyed and recreated between running individual test methods. */   static final ContextCache contextCache = new ContextCache();   private ConfigurableApplicationContext context;   private final String[] contextLocations;   public TemporarySpringContext(String... contextLocations) {     this.contextLocations = contextLocations;     try {       context = contextCache.contextForLocations(contextLocations);     } catch (Exception e) {       throw new RuntimeException(e);     }   }   @Override   public Statement apply(Statement base, FrameworkMethod method, Object target) {     context.getAutowireCapableBeanFactory().autowireBean(target);     return super.apply(base, method, target);   }   public ConfigurableApplicationContext getContext() {     return context;   }      public void dirtyContext(){     contextCache.markDirty(contextLocations);   } }

Because it extends TestWatchman you can hook into al the phases of your test by simply overriding a method. Because it is just a MethodRule field you can add different contexts to your test class. There is plenty of room to polish this, but it has more potential than the SpringJUnit4ClassRunner I think.

Since most of the heavy lifting is still being done by Spring Test behind the scenes by the way. The ContextCache is the only complex part (yet) but I have some tricks to pull to make @Transactional and such work. If you fix it before me I owe you a beer :)

Posted via email from iweinfuld's posterous

Excuse me, your fly is open

After I showed a colleague a trick in my IDE he told me to write a blog post about it. I told him to write it himself. This all happened almost a year ago. The problem with this post is that it's about something that is a bit embarrassing. I'm losing my appetite for talking about it, but yet, I come across more setups that are wrong than setups that are right. This post is mostly about some settings in Eclipse (and also in IntelliJ IDEA) that are pretty moronic and that you should change asap to prevent bugs in your code. Otherwise you might be seen with an open fly and you wouldn't want that would you?

First a word on keyboard shortcuts

First of all: learn to use your tool!  Here's a list of things that I see people wasting effort on:

- click through the menu (unless you're trying to find what the keyboard shortcut is you're looking for)

- declare a variable or field (use extract variable instead)

- type the skeleton of a method (your IDE can do that much faster using a quick fix)

- create a new type other than a test case yourself (generate everything from a test case, it's faster, trust me on this one)

There are many more variations on the same theme, the last point is arguable, some people are really fast if they start to hash out the model before beginning a test. My point is that it pays off to find the shortest path to a certain solution and figure out what shortcuts get you along that path fastest. The best way to train yourself to use the keyboard is to stick to the first point. There is even an eclipse plugin that can enforce keyboard usage (but that's a little too harsh for my taste). There are many blogs describing the keyboard shortcuts in great detail as well.

Templates

When you stop using your advanced typing skills and let the IDE do the typing for you, you will have moved on to using templates. In eclipse, if you look for templates under preferences (just type templates in the filter).

You will find quite a few templates that are useful, some that you'll never use and the best news is that you can create your own too. I can recommend renaming the JUnit4 template to tst (instead of the default Test in eclipse) and tweaking it a bit to your coding standards, but that's not why we're here now.

Not all these templates are sensible, and that is what I want to to fix.

New method body

A new method you can create by just trying to use the method and then asking Eclipse to create the missing method for you.

You'll end up with a method like this:

1 2 3 4 5 6 7 8

public class Reverser { public void reverse(String string) { // TODO Auto-generated method stub } }

Which is pretty bad because it will allow your code to run without complaining and leave a crappy piece of comment in there. It's not the end of the world though, and you'll notice later right? You probably won't, but you don't have to, more on that in a minute. First let's make the method return a String.

Stop, don't start modifying the method, just delete it. Eclipse will generate it correctly for you later.

Modify the test a bit so you need a String return value and try the quick fix again.

Now Eclipse gives you this:

1 2 3 4 5 6 7 8

public class Reverser { public String reverse(String string) { // TODO Auto-generated method stub return null; } }

This is not only pretty bad, its retarded. Just run the test case and see for yourself. The only plausible result of this will be a NPE, and the NPE will happen in another class so you have to retrace your steps to find out where you need to fix it. I want my test case to throw me a stack that starts exactly where I need to continue developing. Even worse, if you're generating methods that are not invoked directly from a test case, this might lead to NPE's flying around in production. This will not do.

So what can we do? Well, it's simple: just modify the template responsible. This template is called 'Method body' and should edit it to look something like this.

Now if you delete the generated method again and regenerate it, it should look better. 

1 2 3 4 5 6 7 8

public class Reverser { public String reverse(String string) { throw new UnsupportedOperationException( "Iwein left this implementation as an exercise to the reader"); } }

If we run the test case now it will fail. But the big improvement is that it will fail correctly. What I mean by failing correctly is that the failure actually points out what is wrong: you have failed to implement a generated method. This is infinitely more informative than an NPE. As a bonus you can click the stack trace in the test and actually go to the method that you need to implement.

Just double click the line that is selected in the above screenshot. Pretty cool huh?

Catch blocks

There is one more template I want you to modify. Check out the 'Catch block body' template. This is the template that gets used when you use the 'Surround with try/catch' quick fix. 

The resulting code looks like this:

1 2 3 4

} catch (Exception e) { // TODO Auto-generated catch block e.printStackTrace(); }

I can tell you that there is plenty of code like that running in production systems. Granted, checked exceptions are hard to deal with, but surely we can do better than this.

This isn't going to make your program run any better, but it sure is going to make it fail better. And that, dear reader is what this post is all about. If things fail early with an informative message, you (and those that come after you) are going to spend less time in a debugger and more time fixing actual problems. 

I've used Eclipse for the screenshots, but I can assure you that on my IDEA setup I have done the same fixes, and it wasn't very hard to figure out either. So now that you've read this there is no excuse to carelessly return null or print a stack trace anymore, even if you're not using Eclipse.

Don't be caught with an open fly again!

Posted via email from iweinfuld's posterous

Combining TemporaryFolder @Rule with Spring 3

I like TDD, I like Springs @ContextConfiguration framework and I like JUnit 4.7's new Temporary folder. Getting the latter two to play nice isn't as easy as I hoped, so here's an attempt at a way to make them work.

Let's say you need to configure an incoming directory on a component using external configuration. For example in Spring Integration: <file:inbound-channel-adapter directory=#{config.directories.store} ... />.

I'm using expression language to load a property from a config object, which I wire in turn as a Spring bean. In my main application this config object can just be a for example, and in my test I can override that bean with an inner class from my test.

public static class Config {
public static final Map directories = new HashMap();
}

If you then use the containing context in a JUnit test it will be loaded up once the context is created, so you need to make sure to put the store value in the map @BeforeClass. Now I'd like to use a TemporaryFolder, to store files in the input directory of the channel adapter, but you can't refresh the property of a singleton bean each test. The @Rule will be applied before each test, resulting in the second test using a different directory from the context. A shame.

A working solution is to just remove the @Rule annotation from the TemporaryFolder and manually invoke create() and delete(). I'd like to know if there is a better solution.

Maven sample project can be found on GitHub.

If you give it a go, let me know if you find a neater option!

Spring Milestones and Snapshots with maven

It took me way longer than I wanted to set up a simple maven project using the Spring M4 and the latest snapshot of Spring Integration. For some reason this is not documented prominently on springframework.org. I commit on Spring Integration, so maybe I should do something about this, on the other hand, blogging is easier... I'll hand you the fix that still worked for me in 2009.

I'm a bit of a maven rookie. I've been using it for years now, but I still feel uncomfortable using it and I have given up on ever having a complete grasp of how it works. The general idea and functionality I understand and sympathize with, but other than that it's too painful to recommend. On the other hand, I just want my build to work and learning Gradle isn't on the top of my list if I need to get started now. Maven itself is tolerable but the plugin and repository landscape isn't.

I'm just going to save you 45 (if not more) minutes of searching and cursing before you find hints on the project pages.

Don't use the search tools for maven central (I like mvnrepository) for Spring dependencies.
Don't use SpringSources enterprise repository for milestones and snapshots. They might be there, but they end up there later than on S3.

There is a milestone and snapshot repository in S3, as described by Ben Hale in a blog. Use the repository snippets from that blog (also included here). Look carefully at the url of the s3browser.

Do use http://s3browse.com/explore/maven.springframework.org/

The repositories are in s3, and while that is very cool, it's also a pain for you because you can't explore them with the url copied from the repository element. Keep a comment on that in your pom if you have colleagues.

<repository>
 <id>spring-milestone</id>
 <name>Spring Portfolio Milestone Repository</name>
 <url>http://s3.amazonaws.com/maven.springframework.org/milestone</url>
 <!-- browse via http://s3browse.com/explore/maven.springframework.org/milestone -->
</repository>

<repository>
 <id>spring-snapshot</id>
 <name>Spring Portfolio Snapshot Repository</name>
 <url>http://s3.amazonaws.com/maven.springframework.org/snapshot</url>
 <!-- browse via http://s3browse.com/explore/maven.springframework.org/snapshot -->
</repository>

If you need artifacts from S2's enterprise repository too you can end up with a whopping 6 repository snippets just for your spring artifacts. Keeping track of which artifact comes from which can be daunting. If you're in the position to do so, using Nexus (or similar) can be used to mitigate that.

There are a lot more details to all this (related to OSGi and maven versioning schemes), but if you're just interested in making it work, you should be grand now.

Hope this helps.

Push vs. Pull or Why company mailing lists are evil

Everybody that works for a large organization knows the semi Observer anti-pattern. In some form or another at least. It works like this: The Big Boss thinks that you need to know, so he pushes information to you through email. He is too lazy to push this corporate spam to all personnel individually so you get added to a list automatically. At least some of your bosses don't know that there is such a thing as BCC so soon all personnel knows all the lists they are on.

The next step is that people start holding discussions using the email addresses for those lists. I've seen these types of discussions spiral out of control in different firms I've worked for. This causes unreasonable load and distraction for the employees and can even cause the whole email system to collapse under the strain. I can tell you this is not the first time something like this has happened.

What makes this behavior unforgivable, is that there is such a simple solution.

Use RSS or newsgroups to let people subscribe to replacements for these lists and clearly separate directed communications (to a few people) from news and open discussions. By using this simple solution (and working for a company that actually does this right), I've managed to reduce my incoming email by 80%. It is still some work to go through all the mails after a few days of inactivity, but the thing is it is not absolutely impossible anymore. The other immensely satisfying improvement is that most mail I get I actually want to read.

I do have thousands of unread items piling up in my RSS reader and nntp client, but those I know I can safely discard when they get stale. They don't eat up disk space and I can always search them.

There are reasons to keep lists in the company anyway. For example some employees are too lazy or stupid to subscribe to RSS lists and newsgroups, if you are legally required to send out a communique an email push might come in handy. In those cases: use BCC. If something should not be done, make it impossible. Guidelines for Reply-All and the pleas and threats to follow them are just more unneccesary noise.

So bosses all over the world, if you read this: be smart and stop spamming your employees, they hate you for it and they're right.

I saw another complaint about many emails in general. I bet that this guy is subscribed to his own blog using email. He ends his post with:

If you are the person with the idea to save us all, send me an email and tell me all about it. Actually, strike that. Drop by my house and tell me all about it. I don’t want your message to get lost in my inbox

I'm not a venture capitalist, but still the difference is striking: my mailbox is completely read, and I spend less than an hour a day to keep it that way.

Syncing calendars... nothing is ever easy

I have a problem of managing multiple callendars. My wife has access to my Google calendar, my company has access to my Zimbra calendar. I use a Mac, and I like to use iCal if possible. I use Plaxo, TripIt and Dopplr, which provide ical interfaces/synchronisation, just like Google and Zimbra.

All this shouldn't be too original, so I expect that other people have this problem. Granted it is not an easy problem to solve, because synchronisation isn't easy. Let me first explain how I'm managing things now (and how I screw up). Then go over the options and then over what I really want.

Now I manage my calendars through CalDav. This works really well for a single calendar, but it does nothing for you in terms of synchronisation. If my office manager add's an appointment I see it show up in the right calendar, if I get a trip confirmation and send it to TripIt same story if my wife makes an appointment I can see it show up too. Nice overview, clear separation of calendars. Perfect, as long as I do the managing. That is the problem, my wife manages the calender of our private life, my manager manages the calender for the jobs. They have no idea what the other is doing.

So I screw up like this: if my manager plans for me to be abroad he checks with me and puts it in the calendar, then I have to make sure it shows in my private calendar that I'm abroad, otherwise my wife will plan something conflicting in the home calendar.

The underlying problem is that I am just one person, so everybody planning for me to be somewhere needs visibility on where I am. Main options:

• Be more disciplined and copy things over
• Use a single calendar and synchronize

The first option is what I'm doing now. Fine tuning this would mean I create a single calendar that lists my availability and make sure that all planners have read access to that from a single interface that they use for planning. The advantage is that this way the planners do not have to worry about fine grained appointments, just about available time (in a certain location).

The second option would allow me to be very lazy, but it would also result in a privacy issue where my manager can see that I went to a party on thursday evening or something. I don't care that much about privacy. I do care about my manager moving around my dentist appointment from the private calendar, so the solution is not ideal.

An ideal situation would allow me to share availability across calendars, and restrict write access and access to detailed information based on different calendars/sets of appointments.

The annoying thing is that there are no tools that allow this. Requirements are:

• Have to use Zimbra for work
• Have to use Gcal for private
• Want to use iCal as interface
• Want to integrate CalDav calendars
  
• No manual syncing

The only way to get close would be to install different sync utilities for my mac and sync with a local calendar. The proper way to do it is to allow CalDAV calendars to be synced in ical, but there is no support for this in iCal natively. The only workable solution (other than discipline) is to run sync utilities like Spanning Sync and the Zimbra iCal plugin. This is bound to screw up my calendars at some point I'm sure...