Frenzy Fixtures in Scala

Following on from my post about Friday Feedback Frenzies, Mark, Ben and I looked at how to work out frenzy pairs. The idea with the frenzies is that you exchange feedback with two people each week, and that you rotate the people that you do this with. When I have organised these previously I have generally just picked people at random. Mark, Ben and I decided to generate a frenzy fixture (if you will).

Mark came up with the idea of using something like a football/tennis game scheduling algorithm. We found a Round Robin cyclic algorithm to get us started. Our first step was to get this algorithm working in Scala:

      val numberOfPlayers = 12
      for (round <- 1 to 11) {
        for (court <- 1 to 6) {
          print((if(court == 1) 1 else (round + court - 2) % (numberOfPlayers - 1) + 2),
                (numberOfPlayers - 1 + round - court) % (numberOfPlayers - 1) + 2)
        }
        println()
      }

The output from this is below. Each row represents a round and each bracketed pair in a row represents a match-up of two players on a court.

(1,2)(3,12)(4,11)(5,10)(6,9)(7,8)
(1,3)(4,2)(5,12)(6,11)(7,10)(8,9)
(1,4)(5,3)(6,2)(7,12)(8,11)(9,10)
(1,5)(6,4)(7,3)(8,2)(9,12)(10,11)
(1,6)(7,5)(8,4)(9,3)(10,2)(11,12)
(1,7)(8,6)(9,5)(10,4)(11,3)(12,2)
(1,8)(9,7)(10,6)(11,5)(12,4)(2,3)
(1,9)(10,8)(11,7)(12,6)(2,5)(3,4)
(1,10)(11,9)(12,8)(2,7)(3,6)(4,5)
(1,11)(12,10)(2,9)(3,8)(4,7)(5,6)
(1,12)(2,11)(3,10)(4,9)(5,8)(6,7)

Next we moved from everything being 1 indexed to being 0 indexed. We also changed the fixed player, who always plays on court 0, to be the last one in the player list. This was done only because it made the loops more readable:

      val numberOfPlayers = 12
      for (round <- 0 to 10) {
        for (court <- 0 to numberOfPlayers/2 - 1) {
          print(((if(court == 0) numberOfPlayers - 1 else (court + round) % (numberOfPlayers - 1))),
                (((numberOfPlayers - 1) - court + round) % (numberOfPlayers - 1)))
        }
        println()
      }

Then the output was:

(11,0)(1,10)(2,9)(3,8)(4,7)(5,6)
(11,1)(2,0)(3,10)(4,9)(5,8)(6,7)
(11,2)(3,1)(4,0)(5,10)(6,9)(7,8)
(11,3)(4,2)(5,1)(6,0)(7,10)(8,9)
(11,4)(5,3)(6,2)(7,1)(8,0)(9,10)
(11,5)(6,4)(7,3)(8,2)(9,1)(10,0)
(11,6)(7,5)(8,4)(9,3)(10,2)(0,1)
(11,7)(8,6)(9,5)(10,4)(0,3)(1,2)
(11,8)(9,7)(10,6)(0,5)(1,4)(2,3)
(11,9)(10,8)(0,7)(1,6)(2,5)(3,4)
(11,10)(0,9)(1,8)(2,7)(3,6)(4,5)

Our next step was to switch domains from sports to feedback frenzies:

val participants = 12
val weeks = 10
for (frenzy <- 0 until weeks) {
  for (pair <- 0 until participants/2) {
    print(((if(pair == 0) participants - 1 else (pair + frenzy) % (participants - 1))),
          (((participants - 1) - pair + frenzy) % (participants - 1)))
  }
  println()
}

Then we changed to creating a collection of all the pairs:

      val participants = 12
      val weeks = 10
      val frenzyPairs = for {
        frenzy <- 0 until weeks
        pair <- 0 until participants/2
      } yield (((if(pair == 0) participants - 1 else (pair + frenzy) % (participants - 1))),
                (((participants - 1) - pair + frenzy) % (participants - 1)))
      frenzyPairs.foreach(print)
    }

And the output…

RangeM((11,0), (1,10), (2,9), (3,8), (4,7), (5,6))
RangeM((11,1), (2,0), (3,10), (4,9), (5,8), (6,7))
RangeM((11,2), (3,1), (4,0), (5,10), (6,9), (7,8))
RangeM((11,3), (4,2), (5,1), (6,0), (7,10), (8,9))
RangeM((11,4), (5,3), (6,2), (7,1), (8,0), (9,10))
RangeM((11,5), (6,4), (7,3), (8,2), (9,1), (10,0))
RangeM((11,6), (7,5), (8,4), (9,3), (10,2), (0,1))
RangeM((11,7), (8,6), (9,5), (10,4), (0,3), (1,2))
RangeM((11,8), (9,7), (10,6), (0,5), (1,4), (2,3))
RangeM((11,9), (10,8), (0,7), (1,6), (2,5), (3,4))

As Ben quite rightly pointed out if we take two rows a time, then each person will exchange feedback with two people in the same frenzy. Our final step was to group each pair of successive rows. To do this we first moved to having two nested for loops. We thought that this would make it easier to do the grouping, but it ended up being redundant. Here our final version of the frenzy fixture generator:

      val participants = 12
      val rounds = 10
      val frenzyPairs = for (frenzy <- 0 until rounds)
                      yield for(pair <- 0 until participants/2)
                            yield (((if(pair == 0) participants - 1 else (pair + frenzy) % (participants - 1))),
                                  (((participants - 1) - pair + frenzy) % (participants - 1)))

      val (evenOnes,oddOnes) = frenzyPairs.toList.zipWithIndex.partition(_._2%2 == 0)
      evenOnes.zip(oddOnes).foreach(println)

Our final output was:

((RangeM((11,0), (1,10), (2,9), (3,8), (4,7), (5,6)),0),(RangeM((11,1), (2,0), (3,10), (4,9), (5,8), (6,7)),1))
((RangeM((11,2), (3,1), (4,0), (5,10), (6,9), (7,8)),2),(RangeM((11,3), (4,2), (5,1), (6,0), (7,10), (8,9)),3))
((RangeM((11,4), (5,3), (6,2), (7,1), (8,0), (9,10)),4),(RangeM((11,5), (6,4), (7,3), (8,2), (9,1), (10,0)),5))
((RangeM((11,6), (7,5), (8,4), (9,3), (10,2), (0,1)),6),(RangeM((11,7), (8,6), (9,5), (10,4), (0,3), (1,2)),7))
((RangeM((11,8), (9,7), (10,6), (0,5), (1,4), (2,3)),8),(RangeM((11,9), (10,8), (0,7), (1,6), (2,5), (3,4)),9))

The end result could be tidied up a little but it does provide us with a frenzy fixture for a given number of participants. Given more time we would have used names for participants rather than just numbers.

Friday Feedback Frenzies

The Friday Feedback Frenzies (cheesy name I know) is something I introduced on our project team about 6 weeks ago. The idea is that on a Friday afternoon every week (or every other week) you give and receive feedback from a couple of people on the project team. Each week you rotate the people that you do this with.

Why frenzy?
In his first post about feedback Pat Kua discussed the idea that we should invite feedback. I agree with him on this. I also would like to add that we should invite feedback from everyone on the project team. The trouble is – and perhaps this is not true across all cultures (project or otherwise) – I’ve not seen this happen.

Not giving and receiving feedback costs teams and relationships in all sorts of ways. As a developer, the pair programming dynamic is one of the most obvious places where the impact of not giving feedback is evident. In my experience people that are finding pairing with a team member difficult opt for silence, passive aggressiveness or just plain ol’ aggressiveness. I’m the first to admit that I’ve done all of these things on occasion. I’ve found that regularly giving and receiving feedback is a much healthier alternative.

So how is it going (in my opinion)?
Overall the frenzies have been going well. One person in our team commented at our last retro that he thought team communication had improved since they started.

The mechanics…
We are still maturing the process of running the frenzies. Initially we began by exchanging feedback with the people that we had worked most closely with that week. This was ok but there was some hesitation in the group because people had to approach one another. We’ve since tried more of a speed dating style – everyone is given the names of two people that they will exchange feedback with the day before.
The effort and ceremony around participating in the frenzy is not supposed to be onerous – grab a meeting room or a bench in the kitchen area and have a quick chat one-on-one. I have tended to pre-prepare some dot points for each person that I am giving feedback to – some for strengthening confidence and some for improving effectiveness.

What’s going well?

• One thing I really like about receiving feedback regularly is that it can be incorporated very quickly. It’s not as useful to get feedback on how you could improve after the project as finished at annual review time.
• The feedback is more specific. How many times have you scratched your head trying to think of a good example to support your feedback? Feedback that covers a short window of a few days/weeks means that you can have conversations that include a lot of detail about specific events, like: ‘remember when we were pairing on story ABC and then….’.

What’s not going so well?
I think that there are two areas that we still need to work on in the near term:

• First up, the feedback process seems to take a long time. I think this is partly related to the fact that some people have worked on the project for a few months and have not previously exchanged feedback.  I’ve not made attempts to curb the time because obviously there is a lot to cover. I think this will come down as the feedback cycle time decreases.
• We have tended to skip the feedback sessions if there are other meetings on. I don’t think this is a good pattern and is related to the fact that our frenzies seem to be at a less-than-frenzied pace.

So why frenzy?
The idea of having a regular event is to make giving and receiving feedback something that is part of normal team life. I understand that having such a ritual may be seen as contrived and as adding process overhead. This is not the intention and my hope is that the ‘scheduled’ aspect disappears entirely over time as the team just does it naturally. This idea is to promote the feedback culture.

Log4j SMTPAppender

Our current project is due for another release to production very soon. Last week Tom added another appender into our log4j configuration for our UAT enviroment. This appender is an SMTPAppender and it sends emails to a group list for our project team. This new appender is set to the ERROR severity. The configuration for this appender looks like this:

log4j.appender.email=org.apache.log4j.net.SMTPAppender
log4j.appender.email.From=me@email
log4j.appender.email.To=us@email
log4j.appender.email.Subject=[AppName] UAT Exception
log4j.appender.email.SMTPHost=host
log4j.appender.email.SMTPUsername=user
log4j.appender.email.SMTPPassword=pass
log4j.appender.email.layout=org.apache.log4j.PatternLayout
log4j.appender.email.layout.ConversionPattern=%d{ISO8601} %-5p [%t %x] [%c{1}] – %m%n
log4j.appender.email.Threshold=ERROR

As of this morning we had recevied 17 emails from UAT. There were 8 types of errors and  the most occurrences of any one type of error was 6. We always had access to this information in the log files of course, but getting the emails has had several advantages:

• The emails draw your attention – the visibility of problems in UAT has been increase.
• We’re more inclined to investigate the cause of the error/exception earlier. Previously the devs would generally only become aware of exceptions in UAT when one of our QAs asked one of us to take a look. Having the email going to all the devs also means that more people look at the exception and we are able to troubleshoot it more easily.
• It’s easier to group exception types – we can collect data about the exceptions more easily.

The plan is to include an SMTPAppender in the log4j configuration for our production environment. I think this is a very good idea and would recommend doing it.

iBATIS

We are using iBATIS on my current project for XML-based object relational mapping. I downloaded the developer guide recently because I was interested to learn about the tool and how this part of our code works. This is what I found out:

1. Repositories
We have ten or so repositories in our code base that all look something like this:

import my.common.util.Maps;
import my.domain.Account;
import my.domain.OrderInfo;
import my.domain.OrderInfoState;
import org.springframework.orm.ibatis.SqlMapClientOperations;

import java.util.List;
import java.util.Map;

public class OrderRepository {

    private final SqlMapClientOperations template;

    public OrderRepository(SqlMapClientOperations template) {
        this.template = template;
    }

    public void insert(OrderInfo orderInfo) {
        template.insert("insertOrder", orderInfo);
    }

    @SuppressWarnings({"unchecked"})
    public List findOtherPendingOrdersForAccount(Account account, String orderIdToExclude) {
        Map params = Maps.create();
        params.put("accountNumber", account.getNumber());
        params.put("orderIdToExclude", orderIdToExclude);
        params.put("orderRequestPendingStates", OrderInfoState.PENDINGSTATES);

        return template.queryForList("findPendingOrdersForAccount", params);
    }

    public void setCompletedNotificationSent(String orderId) {
        template.update("setOrderAccountType", orderId);
    }
}

Each repository has a dependency on SqlMapClientTemplate which according to the Spring API is a “helper class that simplifies data access via the iBATIS SqlMapClient API, converting checked SQLExceptions into unchecked DataAccessExceptions”. We use the template to execute SQL statements that are defined in XML SQL Map files – but let’s get back to that in a moment.

2. Next let’s have a look at how this is all wired together in Spring:

<bean id="OrderRepository">
 <constructor-arg ref="sqlMapClientTemplateForApp"/>
</bean>

<bean id="sqlMapClientTemplateForApp">
 <property name="sqlMapClient" ref="sqlMapClientForApp"/>
</bean>

<bean id="sqlMapClientForApp">
 <property name="configLocation" value="classpath:/sqlmap.xml"/>
 <property name="dataSource" ref="appDataSource"/>
</bean>

<bean id="appDataSource" destroy-method="close">
 <property name="driverClassName" value="${my.datasource.driver}"/>
 <property name="url" value="${my.datasource.url}"/>
 <property name="username" value="${my.datasource.username}"/>
 <property name="password" value="${my.datasource.password}"/>
</bean>

Bean sqlMapClientForApp has a property configLocation which has been set to "classpath:/sqlmap.xml". Inside sqlmap.xml we have a few custom handlers (more on that later too) as well as a list of XML SQL Map files (one for each repository):

<?xml version="1.0" encoding="UTF-8" ?>
<!DOCTYPE sqlMapConfig PUBLIC "-//batis.apache.org/DTD SQL Map Config 2.0//EN"
 "http://ibatis.apache.org/dtd/sql-map-config-2.dtd">

<sqlMapConfig>
 <typeHandler javaType="my.domain.Money"
 callback="my.domain.MoneyTypeHandlerCallback"/>

 <sqlMap resource="sqlmaps/orderinfo.xml"/>

</sqlMapConfig>

4. The juicy bit – the SQL Map files

These are XML descriptor files that are used by the iBATIS Data Mapper to map JavaBeans to SQL prepared statements. As explained in the developer guide, iBATIS accepts parameters as input into prepared statements and builds result objects from the ResultSet returned. Below is a cut down version of a typical iBATIS SQL Map configuration file from our code. It has three statements defined – each corresponding to the methods in the repository class above. I have annotated the file with comments explaining the iBATIS syntax:

<sqlMap namespace="orderInfo">

 <!-- Type aliases for fully qualified class names. -->
 <typeAlias alias="orderInfo" type="my.domain.OrderInfo"/>
 <typeAlias alias="account" type="my.domain.Account"/>

 <resultMap id="orderInfoResult" class="orderInfo"> <!-- This class will be instantiated and populated from the data in the ResultSet -->
 <!-- This controls that data that is taken from the ResultSet and how it maps to columns -->
 <result property="orderId" column="order_id"/>
 <result property="orderDate" column="order_date"/>
 <result property="userId" column="user_id"/>
 <result property="account.number" column="account_id"/>
 <result property="account.type" column="account_type"/>
 <result property="account.name" column="account_name"/>
 </resultMap>

 <sql id="order_info_attributes"> <!-- This fragment can be included in the SQL statements. See below -->
 order_info.id,
 order_info.order_id,
 order_info.order_date,
 order_info.user_id,
 order_info.account_id,
 order_info.account_type,
 order_info.account_name
 </sql>

 <!-- This select query uses the resultMap declared above-->
 <!-- This is a mapped statement. These can have param maps as input and result maps as output -->
 <select id="findPendingOrdersForAccount"
 parameterClass="map"
 resultMap="orderInfoResult"> <!--The parameterClass "map" is an alias for a java.util.Map -->
 <!--The SQL can be anything that is valid for the JDBC driver -->
 SELECT DISTINCT
 <include refid="order_info_attributes"/> <!-- This includes the order_info_attributes fragment above -->
 FROM order_info INNER JOIN order_request ON order_info.order_id = order_request.order_id
 WHERE order_info.account_id = #accountNumber#  <!-- This is an inline parameter map reference-->
 AND order_info.order_id != #orderIdToExclude#
 AND (
 order_request.state IN (
 <iterate property="orderRequestPendingStates" conjunction=",">
 #orderRequestPendingStates[]#
 </iterate> <!-- This is an iterate dynamic element. The property attribute defines the array or java.util.Collection or java.util.Iterator type to be iterated over -->
 )
 </select>

 <insert id="insertOrder"
 parameterClass="orderInfo">  <!-- The parameterClass is used to restrict the objects that can be passed as an input parameter -->
 INSERT INTO order_info(order_id, order_date, user_id, account_id, account_type, account_name)
 VALUES(#orderId#, #orderDate#, #userId#, #account.number#, #account.type#, #account.name#)
 <selectKey resultClass="int" keyProperty="id">  <!-- iBATIS supports auto-generated keys -->
 SELECT LAST_INSERT_ID() AS id
 </selectKey>
 </insert>

 <update id="setOrderAccountType" parameterClass="string">
 UPDATE order_info SET account_type = blah WHERE order_id = #value#
 </update>

</sqlMap>

5. Custom type handlers

Some of our repositories use the iBATIS custom type handlers declared in the sqlmap.xml file (above) to map columns to specific types in our domain. For example the sellPrice field on Product is Money type:

<resultMap id="productMapping">
 <result property="sellPrice" column="sell_price"/>
 <result property="status" column="status"/>
</resultMap>

iBATIS will use the MoneyTypeHandler below to create a Money instance from the sell_price value. Each one of these handlers implements the TypeHandlerCallback interface.

import my.domain.Money;
import com.ibatis.sqlmap.client.extensions.ParameterSetter;
import com.ibatis.sqlmap.client.extensions.ResultGetter;
import com.ibatis.sqlmap.client.extensions.TypeHandlerCallback;

import java.math.BigDecimal;
import java.sql.SQLException;
import java.sql.Types;

public class MoneyTypeHandler implements TypeHandlerCallback {

    public void setParameter(ParameterSetter parameterSetter, Object obj) throws SQLException {
        if (obj == null) {
            parameterSetter.setNull(Types.DECIMAL);
        } else {
            Money money = (Money) obj;
            parameterSetter.setBigDecimal(money.toBigDecimal());
        }
    }

    public Object getResult(ResultGetter resultGetter) throws SQLException {
        BigDecimal value = resultGetter.getBigDecimal();
        return (value != null) ? new Money(value) : Money.ZERO;
    }

    public Object valueOf(String value) {
        return value;
    }
}

Lists, maps etc

On our project we are using some utility/toolbox Java classes that Tom, Alex and others on our project have made.  You can find versions of them similar to the ones we are using on our project in Tom’s example J2EE web application. There is also a slightly different flavour of them available here. Most of the classes help you to manage and mutate collections.  We have found them to be very useful on our project. In fact, we currently have 152 uses of the create method from the Lists class (see below) in our code base (which works out to be about one in every three classes).

We use the Lists the most overall. Tom and I used it a couple of weeks ago to get some data from a repository. This was based on what we’d grabbed in another repository and then narrowed the result further using a select statement. All this in only three quickly assembled lines of Java:

Listfoos=fooRepository.findBySomeCriteria();
Listbars=Lists.map(foos,new BarMapper(barRepository));
bar=Lists.select(bars,new CompletedBarsMatcher());

Here is the code in the Lists class:

import org.hamcrest.Matcher;
import org.hamcrestcollections.Function;
import org.hamcrestcollections.FunctionMapper;
import org.hamcrestcollections.RejectMatcher;
import org.hamcrestcollections.Selector;

import java.util.ArrayList;
import java.util.Collection;
import java.util.Collections;
import java.util.List;

public class Lists {

    public static  List create() {
        return new ArrayList();
    }

    public static  List create(Collection instances) {
        return new ArrayList(instances);
    }

    public static  List create(T... instances) {
        List list = create();
        Collections.addAll(list, instances);
        return list;
    }

    public static  T first(List list) {
        return first(list, null);
    }

    public static  T first(List list, T defaultValue) {
        return isEmpty(list) ? defaultValue : list.get(0);
    }

    public static  T last(List list) {
        return last(list, null);
    }

    public static  T last(List list, T defaultValue) {
        return isEmpty(list) ? defaultValue : list.get(list.size() - 1);
    }

    public static  boolean isEmpty(List list) {
        return (list == null) || list.isEmpty();
    }

    public static  boolean isNotEmpty(List list) {
        return !isEmpty(list);
    }

    public static  boolean contains(List list, Matcher matcher) {
        for (T item : list) {
            if (matcher.matches(item)) {
                return true;
            }
        }
        return false;
    }

    public static  T findFirst(List list, Matcher matcher) {
        for (T item : list) {
            if (matcher.matches(item)) {
                return item;
            }
        }
        return null;
    }

    public static  int count(List list, Matcher matcher) {
        int count = 0;
        for (T item : list) {
            if (matcher.matches(item)) {
                count++;
            }
        }
        return count;
    }

    public static  List select(List list, Matcher matcher) {
        return (List) Selector.select(list, matcher);
    }

    public static  List reject(List list, Matcher matcher) {
        return (List) RejectMatcher.reject(list, matcher);
    }

    public static  List map(List list, Function function) {
        return (List) FunctionMapper.map(list, function);
    }

    public static  U reduce(List list, U initialValue, Reducer reducer) {
        U result = initialValue;
        for (T item : list) {
            result = reducer.reduce(item, result);
        }
        return result;
    }
}

You can see that several of the methods call functions in the hamcrest-collections library in this version of Lists. Tom was saying that he doesn’t like that they return an Iterable that then needs to be cast to a List (in this case). Note also the implementation of the reduce function that Tom and Alex wrote. It is actually mapping and reducing – but I’ll let Alex give all the details about that one in his promised blog post

Seeing the real thing

About three weeks ago I moved onto a new project that has already been going for several months. My arrival on the team coincided with the team also physically moving into a bigger space in a different building about 10 minutes walk from where they were previously. However the users of one of the systems that we integrate with remained in the original location.

Last week Nick and I were pairing on two defects that were raised in QA in our application. Both of these were indirectly caused by some data coming from our fake test implementation of one of our integration points. Our aim was not to make our fake any more capable or complicated than we possibly needed it to be. However, we did need to modify our fake, both to avoid raising bugs, or fix them where they occurred.

A question arose about how the real implementation of the system that we are faking works. Nick called up the users of the real system and within 15 minutes we were back at the old building chatting with them.

The ability to quickly get in touch with real users and see exactly how they do their jobs is extremely beneficial for a project team. There is no description or diagram that can replace the context and ‘ah ha’ kind of understanding that you get when you see the real thing. I remember having the same kinds of experiences in past jobs that I’ve had when I was an aerospace engineer. Going down and seeing the relevant parts (or even similar parts) on the production line gives you tangible knowledge and recognition that no detailed “fancy” CAD simulation, or description, can.

Not being able to access real systems and users is one of the biggest challenges  facing any deivery team. Where you are not working in the same immediate space as the people involved (either directly or indirectly) with the team, you are immediately at a disadvantage. Lucky for us, our recent move was only a small one, and direct contact can still be maintained.

This need for ongoing contact is something that ThoughtWorks is very aware of when doing distributed agile projects – now I know why.

On mocking and Spring

Two weeks ago I was pairing with Alex on some Java integration tests for an application that uses Spring for dependency injection. Whilst writing one of the tests we realised that we needed to mock some behaviour on one of our Spring configured classes. Instinctively the simplest way to do this would be to replace the bean in our application context with another one, but alas after some brief investigation we discovered this is not an option.We ended up using the same pattern that was already in use in our code base, and I’m sure it’s one that some people will be familiar with:

First up we extracted an interface from the class that whose behaviour we wanted to mock:

public interface FooRepository {
    Foo loadFor(XYZ xyz);
}

Next we created a new Application Context XML configuration file to use for the test and included the ‘real’ one from our application. We also defined a stub class that implemented our new interface. It also had the same id as the class that we wanted to mock behaviour for meaning that the pre-existing definition would be overridden:

<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
 xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
 xsi:schemaLocation="http://www.springframework.org/schema/beans
 http://www.springframework.org/schema/beans/spring-beans.xsd">
 <bean id="FooRepository" class="FooRepositoryStub"/>
 <import resource="applicationContext.xml"/>
</beans>

Now for the behaviour that we want in the test…. Singleton scoped Spring beans are loaded at start-up (unless you have lazy loading which we do not). Hence if we want to be able to define specific behaviour inside a test we can’t use on our stub implementation alone – rather we need to use the stub as a wrapper for a delegate:

public class FooRepositoryStub implements FooRepository {

    private FooRepository delegate;

    public Foo loadFor(XYZ xyz) {
        return delegate.loadFor(xyz);
    }

    public void setDelegate(FooRepository delegate) {
        this.delegate = delegate;
    }
}

The delegate can then be used to provide the mocking that we wanted all along:

@RunWith(SpringJUnit4ClassRunner.class)
@ContextConfiguration(locations = "classpath:applicationContext-test.xml")

public class ValidationIntegrationTest {

    @Autowired
    private FooRepository FooRepository;
    private FooRepository FooRepositoryDelegate = mock(FooRepository.class);

    @Before
    public void initializeRepositoryDelegates() {
        ((FooRepositoryStub) FooRepository).setDelegate(FooRepositoryDelegate);
    }

    @Test
    public void shouldInvalidateXYZWhenTheABCRuleIsViolated() {
        XYZ xyz = givenSomeContext();
        andTheUserHoldsFoo(xyz);
        whenSomethingHappends(xyz);
        thenItShouldFailWithReasons(xyz, "XYZ failed because of the ABC rule was violated");
    }

    private void andTheUserHoldsFoo(XYZ xyz) {
        Foo foo = mock(Foo.class);
        when(FooRepositoryDelegate.loadFor(xyz)).thenReturn(foo);
    }
    ....
}

So, with only an interface, another ApplicationContext XML file, a stub and a mock, we can define the behaviour in the test that we wanted (phew)! I’ve used PicoContainer before and only just now realise how easy with had it in setting up tests. Of course there may be an easier way to do all this – and if you have one then I’d love to hear from you.

Melbourne Scala Users Group

On Monday night I braved a blustery wet Melbourne night to go to the first Scala Melbourne Users Group meeting at IBS.

The first speaker was Ben Hutchison. Ben presented Combining Object-Oriented and Functional programming, focusing on the features of the language that stand out for him. Ben said that he has really enjoyed what he’s been doing in Scala although he admits it has been a bit of a steep learning curve.
Here’s some things that Ben mentioned that I learnt from the presentation:

• Functions are first class objects in Scala. From the example Ben showed it appears that you do have to know a little about the syntax in order to understand what the arguments to a method are and what its return type is.
• You can override methods with fields and all operators are just methods.
• You can declare a variable as mutable or immutable using the keyword var (mutable) or val (immutable).
• Ben spent quite a bit of time on traits, which I basically understand from him to be not unlike modules in Ruby. It is possible for a class to inherit multiple traits, and traits can even inherit from other traits (and even classes in unusual circumstances!).The traits concept seems very powerful – for example Ben mentioned one trait called Ordered that when applied to a class effectively gives your class a group of methods like < (less than) and > (greater than) and all you have to do is implement the compare method.
• Ben talked about mixing OO and functional programming – something that Nick taught me on my last project. In particular Ben spoke about how functions should be side effect free – an idea that Eric Evans talks about in Chapter 10 in Domain Driven Design.

Mark Ryall was the next presenter and he spoke about Scala’s XML support as well as a bit about using Lift on the Google App Engine. From Mark’s presentation I learnt:

• It is possible to assign some XML to a variable in Scala
• You can do either DOM or SAX style parsing of XML, and if your not happy with those options there are some pretty cool extraction operators, that look a bit like XPath when they’re used as Mark pointed out.
• There’s also XML pattern matchers that can match on XML nodes, but will not work on element attributes at the moment at least.
• You can generate an XML document by calling savefull. It’s also possible to insert Scala code into an XML document and then write that out.
• Mark has got a Lift application working on the Google App Engine although apparently it was tricky to get going. One of the really impressive things about what he showed was the really concise user authentication that used the Google Accounts.

Jeremy Mawson was the third speaker for the evening and he focused on testing and more specifically on using specs, which I’d not heard of before. In a lot of ways the tests that he showed us looked like RSpec tests but the syntax does not read quite as nicely as RSpec does.

• Jeremy showed us how you can nest expectations in specs.
• Apparently there are over 100 matchers already in specs – and if that’s not enough you can write your own
• spec makes use of implicit function definition to write the tests
• specs is one of a handful of Scala testing frameworks 
• You can use JMock and Mockito in Scala

Overall I’d say it was a really successful meeting. I certainly learnt a lot from the three speakers. I would recommend getting along to the next meeting if you’re in Melbourne. Ben did mention that he was thinking of making the next meeting a practical one. Perhaps he can check out Mark’s blog entries about the TW Sydney Wednesday Night Dojo for some ideas.

Testing database naming conventions in Java

I was recently reminded about some tests that we ran as part of the continuous integration build on my last project – tests that made sure that the database table, column and constraint names adhered to the client’s standards. The idea is quite simple and did on a number of occasions during development prove to be very useful. These tests certainly avoided any last-minute pre-release manual testing.

The tests were all contained in a single class and ran from a Buildr project (below). The tests use JUnit and Hamcrest along with Java SQL:

public class DatabaseNamingStandardsTest {

    private static Set tableNames;
    private static Connection connection;

    @BeforeClass
    public static void connectToOracle() throws SQLException,
                                                ClassNotFoundException,
                                                IOException {

        String configFileName = System.getProperty("config_file");

        Properties properties = new Properties();
        properties.load(new FileInputStream(configFileName));

        String url = properties.getProperty("database.url");
        String driver = properties.getProperty("database.driver");
        String username = properties.getProperty("database.username");
        String password = properties.getProperty("database.password");

        Class.forName(driver);

        connection = null;
        connection = DriverManager.getConnection(url, username, password);

        tableNames = listOfTableNames();
    }

    @AfterClass
    public static void tearDownConnections() throws SQLException {
        connection.close();
    }

    private static Set listOfTableNames() throws SQLException {
        Statement statement = connection.createStatement();
        ResultSet resultSet = statement.executeQuery(
                "SELECT table_name\n" +
                "FROM user_tables");
        Set result = new HashSet();
        while (resultSet.next()) {
            String tableName = resultSet.getString("TABLE_NAME");
            result.add(tableName);
        }
        return result;
    }

    @Test
    public void tableNamesShouldBeOfMaximum25CharactersInLength() {
        for (String tableName : tableNames) {
            assertThat(tableName.length(), lessThan(26));
        }
    }

    @Test
    public void ColumnNamesShouldBeOfMaximum30CharactersInLength() throws SQLException {
        Statement statement = connection.createStatement();
        for (String tableName : tableNames) {
            ResultSet resultSet = statement.executeQuery(
                    "SELECT * \n"
                    +"FROM user_tab_columns \n"
                    +"WHERE table_name='" + tableName + "'");
            while (resultSet.next()) {
                String columnName = resultSet.getString("Column_Name");
                assertThat(columnName.length(), lessThan(31));
            }
        }
    }
....
}

And here is the Buildr project definition…

define 'database-naming-standards' do
  test.with LIQUIBASE_JAR, JUNIT_JAR, HAMCREST_JAR, ORACLE_CLIENT_JAR
  test.using :properties => { :config_file => ("../../#{$oracle_properties_file}")}
  test.setup(
    ORACLE_DB_SCHEMA.rollback_all,
    ORACLE_DB_SCHEMA.update
  )
  package(:jar)
end

LiquiBase: Inserting data from a CSV file

BA couple of weeks ago Nick pointed out to me that it was possible to insert data into a LiquiBase change set from a CSV file. This Load Data feature was added in version 1.7.0 of LiquiBase and is particularly useful for two reasons:

1. It keeps the length of the changelog file down
2. It’s easier to edit a CSV file than it is an XML file, especially when you’re adding a large amount of data to a table

While this is a great option there are a couple of things worth keeping an eye out for if you go down this path…

First, dates:
As I’ve written about before, we are using HSQLDB for development but need to support Oracle. We’ve had relatively few problems in working with both, except for the date compatibility issue.

When inserting XML date data into table entries we were using SYSDATE as the value. This works fine in XML, but does not work when loading data from a CSV file. In the CSV case LiquiBase attempts to create a formatted timestamp from the ‘SYSDATE’ string. We tried a number of ways to enter dates that were compatible in both HSQLDB and Oracle, including adding extra columns in the CSV to cater for the different databases (see below). We also tried updating the table date columns after inserting the entries using the CSV file, but this was messy because those columns have not-null constraints.

The solution to the date compatibility problem turns out is very simple (drumroll) – just define the date in one of the date formats in the Liquibase.util.ISODateFormat class (below) and LiquiBase will generate a database specific date for you!

yyyy-MM-dd'T'HH:mm:ss

yyyy-MM-dd HH:mm:ss

yyyy-MM-dd'T'HH:mm:ss.S

Second, columns:

The documentation for the CSV load data feature gives the impression that you need to map each column in the CSV to one in the table. In fact LiquiBase will map the CSV file column headings to table column headings automatically. You only need to map the columns that have a different heading in the CSV to that in the table. Note however that we found it was necessary to map the columns containing date data, even if the headings were the same:

    <loadData tableName="My_Table" file="My_File.csv">
        <column name="CREATED_DATE" header="CREATED_DATE" type="DATE"/>
    </loadData>

Unfortunately the automatic mapping also means that you cannot have extra columns in the CSV.