Social Startup 48

Last weekend I was a participant in Social Startup 48. I was a really enjoyable weekend full of great conversation, new people and big ideas and I was really pleased that ThoughtWorks was one of the sponsors.

The weekend kicked off on Friday night with a terrific talk from Andrew Mahar about his projects including Infoxchange. After that we divided into groups and spent an hour at talking through our initial suggestions for social enterprises. My team was the sustainability team and we converged on the idea for creating ‘a platform for recycling’ for our initial pitch.

On the Saturday morning all the teams started early developing their ideas. My team grabbed a wheely whiteboard and started to flesh out our platform idea using the business model canvas and some freehand drawing. We realized within an hour that a whole recycling platform was too big to chew so we pivoted and tried to come up with some fresh ideas that were more tangible. By lunchtime we thought we had a plan – pop-up hard rubbish collection events.

One of the highlights of the weekend for me was the talk that we had on Saturday at lunchtime from Marcus Westbury who founded Renew Newcastle:

“Renew Newcastle aims to find artists, cultural projects and community groups to use and maintain these buildings until they become commercially viable or are redeveloped. Renew Newcastle is not set up to manage long term uses, own properties or permanently develop sites but to generate activity in buildings until that future long term activity happens.”

After lunch on Saturday we had the opportunity to meet with some of our mentors and go through our progress so far. I was really impressed with all the terrific ideas and people that I met over the weekend. We had some great mentors including Susan Jones from ReadySetStartup and Tia Saunders from Zookal. When we tried to explain it to our mentors we realized we didn’t really have a great business model at all so it was back to the drawing board for us. By the end of the day we’d decided on a bulky item recycling service for people who live in high rise apartments called ReMoveMe.

On Sunday there was more crunching of ideas. Our group focused on whether our enterprise would really be viable and how we could grow it. Other teams got out into the street to shoot videos, pick up matching tee-shirts for their pitches from Dimmys and conduct surveys. The energy in the room for the whole weekend was amazing and many of our presenters mentioned it when they walked in.

Sadly the weekend had to close at some point. All the team pitches on Sunday evening were amazingly creative and collaborative. I think all the participants spoke and it was great to see so many guests in the room at Inspire9 for the final session. ReMoveMe was voted the best idea by the judges :-).

I had a real ball at SS48 and I’d encourage everyone to participate in a weekend event if they get the chance.

The Lens Retrospective

Last week I was asked to facilitate a retrospective for a team and I decided to try out a new format. I’d facilitated their first retro a few weeks earlier and the scope had been for a whole release: 10 weeks. The scope of the second one covered only two weeks and during this shorter period time the team had been preparing for their next release by doing release planning activities and clearing technical debt. Given the reduced timeframe and the fact that the team was preparing for a fresh start, I wanted the format of the retrospective to include some forward thinking (or ‘futurespective’) elements.  

Last year I purchased a copy of HBR’s 10 Must Reads On Managing Yourself. One of the articles (‘Manage Your Energy, Not Your Time’ by Tony Schwartz and Catherine McCarthy), discusses techniques that are used to help people balance their perception of events. This is done by asking people to “view (a situation) through any of three new lenses”. In the article they give an example of three lenses that can be used for a conflict situation. 

Borrowing from the lens idea, I asked the participants in the retrospective I ran to consider the last two weeks through three lenses: 

1. The Reverse Lens: Given what we know now, what learnings and appreciations can be take from the last 2 weeks? 

2. The Wide Lens: Regardless of the outcomes of the project, what do we hope to try, learn and grow? 

3. The Long Lens: Imaging that it’s 6 months from now, what’s the situation (practices, achievements etc)? 

The group asked for about 10 minutes for individual brainstorming. After that I did some grouping and clarifying before people voted for what they wanted to discuss. The rest of the 1 hour session was spent working our way through the discussion points. 

The team came out with 4 practical actions. I asked for feedback on the format at the end of the session and everyone was very positive. The only point that came up that I would consider before running this retro style again was that they felt was more difficult to generate ideas for the Wide Lens category than it was for the other two. That said, there were just as many sticky notes on that sheet as on the other two. 

If you decide to try out this format then please let me know how it goes. 

Being a trainer at the JAMS 2012 Workshop

At the end of March I had the pleasure of being one of 5 trainers at the JAMS 2012 Workshop at the Makerere University in Kampala, Uganda. The workshop was run by ThoughtWorks in partnership with the Grameen Foundation, with further sponsorship from Yo and SMS Media. 

Most of the 50 workshop participants were second or third year students from the university. For many of them their only prior experience with Java was from a single semester subject. Before we arrived we thought that the ratio of 50 participants to only 5 trainers was going to be quite challenging. We needn’t have worried though because all the participants were very patient and very switched on. 

The curriculum of the workshop was a combination of agile business analysis, project management and, for the majority of it, hands on developer skill training. As part of the course we ran quite a few of the object boot camp exercises. These exercises, put together by ThoughtWorkers several years back, are often used for internal training. They aim to reinforce the principles of object orientation and introduce some design techniques and patterns. The bootcamp was run in Java, as is often the case. 

Despite never having heard of JUnit and not being clear on language essentials like constructors and equals methods, the participants really impressed us with how they applied themselves. The bootcamp exercises intentionally give participants bandwidth to try things out for themselves and discover what works and what doesn’t. Group discussion is used to drive out the learning points and as instructors we look for code smells like YAGNI. Interestingly with the JAMs group we rarely found that we needed raise the same point twice – even where we expected to. Writing unit tests first and implementing value objects are just two examples of concepts that the students picked up immediately and just started using.

In the last 2 days of the workshop we worked on a web application with the group. The app uses the Java Play framework. It’s called MakVibes and it’s intended help Makerere students track what’s happening on campus. The idea for the app came from one of the workshop participants – Alex. It’s already deployed on Heroku and it’ll be extended in the monthly code jams that are currently being planned. The jams will also involve the attendees working on some code kata problems as well as tech talks. There is a lot of fantastic enthusiasm in Kampala for this kind of ongoing learning as well as a desire to increase the use of technology in business. Uganda is an exciting place to be!

Things that made me go hmmmm…

Over the last couple of months I’ve been working on a Scala project. There is nothing quite like immersing yourself in a language to increase your knowledge. Along the way I figure that I must have made every possible syntax mistake. This post is about three gotchas – detail of the language has tripped me up.

Method invocation with parentheses

As a general rule, it’s not necessary to use parentheses when invoking Scala methods that have either none or one argument. There are two cases where this does not apply however:

1. Default arguments:
Soon after making the switch over from Scala 2.7.7 to 2.8.0 we made our first attempt at using a default parameter. It went something like this:

object MyObject {
  def something(someArg: String = "Fred"){ println(someArg) }
}

class Foo() {
  MyObject something
}

This results in this error:

error: missing arguments for method something in object MyObject;
follow this method with `_' if you want to treat it as a partially applied function
         MyObject something

As pointed out here “If a method definition includes parentheses, when you call it you must also use parentheses. Presumably this still holds if all the arguments to the method have default values.”

2. Using println:

scala> println 10
<console>:1: error: ';' expected but integer literal found.
       println 10
               ^
scala> println(10)
10

According to Programming In Scala: “Note that (the no parentheses) syntax only works if you explicitly specify the receiver of the method call. You cannot write ‘println 10’, but you can write ‘Console println 10’.”

Case?

In my opinion one of the best things about Scala is the first-order functions that are defined on the TraversibleOnce trait. I’d use a couple of them on any given day. I’ve often needed to perform an operation on each key-value pair in a Scala Map. On more than one occasion I’ve started off writing something like this:

val myMap = Map("hay" -> "stack", "killer" -> "whale")
myMap.filter((key, value) => key == "killer").map((key, value) => value)

Which results in this error:

error: wrong number of parameters; expected = 1

Doing this another way using a for expression gives a better result however:

scala> for((key,value) <- myMap if (key == "killer")) yield(value)
res19: scala.collection.immutable.Iterable[java.lang.String] = List(whale)

Programming In Scala explains that the Scala compiler translates all for expressions into combinations of higher order methods:
“A for expression of the form:
for (x <- expr1 if expr2) yield expr3
is translated to:
for (x expr2)) yield expr3”

and: “A for expression of the form:
for((x1, ..., xn) <- expr1) yield expr2
translates to:
expr.map{case (x1,...,xn) => expr2}"

So if I want to use higher order methods as I attempted to do first, I need:

scala> myMap.filter({case(key, value) => key == "killer"}).map({case(key, value) => value})
res29: scala.collection.immutable.Iterable[java.lang.String] = List(whale)

Or the equivalent shorter version:

scala> myMap.filter(_._1 == "killer").map(_._2)
res30: scala.collection.immutable.Iterable[java.lang.String] = List(whale)

Booleans

A java.lang.Boolean does not implicitly convert to a Scala Boolean. You need to convert the java.lang.Boolean into a java boolean type using the booleanValue method in order for the conversion to be possible:

scala> val javaBoolean: java.lang.Boolean = true
javaBoolean: java.lang.Boolean = true

scala> val scalaBoolean: Boolean = javaBoolean
<console>:13: error: type mismatch;
 found   : java.lang.Boolean
 required: scala.Boolean
       val scalaBoolean: Boolean = javaBoolean
                                   ^

scala> val scalaBoolean: Boolean = javaBoolean.booleanValue
scalaBoolean: Boolean = true

Scala Multiple Constructors

A couple of weeks ago I created a new type of exception for use in our application. If I’d been using Java I would have written something simple like this:

public class MyException extends RuntimeException {

    public MyException(String message) {
        super(message);
    }

    public MyException(String message, Throwable throwable) {
        super(message, throwable);
    }
}

The Scala multiple constructor model is different to the Java one, as described in this article. In Scala, only the primary class constructor can call a superclass constructor, meaning that it’s only possible to invoke only one superclass constructor. The code above makes calls to two constructors on the RuntimeException class. So how can we write a Scala equivalent? Fortunately one of the answers posted to this question explains a pattern for how to do this:

object MyException {
  private def create(message: String): RuntimeException = new RuntimeException(message)

  private def create(message: String, throwable: Throwable): RuntimeException = new RuntimeException(message, throwable)
}

class MyException private (exception: RuntimeException) {
  def this(message: String) = this (MyException.create(message))

  def this(message: String, throwable: Throwable) = this (MyException.create(message, throwable))
}

I think the pattern is really neat but certainly a lot more complicated that the Java equivalent. I guess this is one of the cases where using Java classes in Scala can be tricky.

Scala Singleton Objects

There are two types of singleton objects in scala – companion objects and standalone objects. According to Programming in Scala:

“A singleton object definition looks like a class definition, except instead of the keyword class you use the keyword object… When a singleton object shares the same name with a class, it is called that class’s companion object. You must define both the class and its companion object in the same source file. The class is called the companion class of the singleton object. A class and its companion object can access each other’s private members… A singleton object that does not share the same name with a companion class is called a standalone object. You can use standalone objects for many purposes, including collecting related utility methods together, or defining an entry point to a Scala application”

Whilst I like the pattern of separating what would otherwise be static methods in Java into companion objects, I do find at times that they are a bit unintuitive, specifically in these two cases:

1. As shown in the brief example below it’s necessary to include the name of the companion object when invoking one of its methods, even from within the companion class. Coming from a Java background and therefore not being used to having static methods in a separate object, I’ve found that I’ve on occasion forgotten to do this.

class Foo {

  def bar {
    println(Foo.createName)
  }

}

object Foo {

  def createName: String = Math.random.toString

}

Programming in Scala has an example with a slightly neater way of handling this situation – including an import statement at the top of the companion class:

class Foo {
  import Foo._

  def bar {
    println(createName)
  }

}

object Foo {

  def createName: String = Math.random.toString

}

2. Along a similar line, I have also been caught out trying to invoke the methods of a companion object via an instance of the associated companion class. The code below shows how its still necessary to call the createName method from the Foo companion object even though we have the someFoo Foo instance:

class Foo {

  def bar(name: String){
    println(name)
  }

}

object Foo {

  def createName: String = Math.random.toString

}

class Consumer {

  val someFoo = new Foo
  someFoo.bar(Foo.createName)

}

Overloaded methods in Scala

I recently discovered that it’s not possible to have default parameters on more than one overloaded method alternative in Scala. The methods that I was working on looked a bit like this:

class Bar() {
  def foo(data: Map[String, String], list: String, field: String = "username"): Boolean = { 
      //do some processing
      true
  }

  def foo(data: String, list: String, field: String = "username"): Boolean = {
      //do some processing
      true
  }
}

Given that it’s possible to work out which alternative needs to be called from the type of the first argument, I incorrectly assumed that these declarations would be ok . As a work-around I created another pair of overloaded methods (see below). If you know of a better alternative then please let me know.

class Bar() {
  def foo(data: Map[String, String], list: String, field: String): Boolean = {
      //do some processing
      foo(data, list, field)
  }

  def foo(data: String, list: String, field: String): Boolean = {
      //do some processing
      true
  }

  def foo(data: Map[String, String], list: String): Boolean =
      foo(data, list, "username")

  def foo(data: String, list: String): Boolean =
      foo(data, list, "username")
}

Map Reduce in Mongo

Recently I was interested in finding out which of our users has received the most coupons. We store information about each coupon in a MongoDB document that looks something like this:

db.coupons.findOne()
{
    "_id" : ObjectId("4cb314e7c42af81a30ec3938"),
    "recipient" : {
        "$ref" : "users",
        "$id" : ObjectId("4cb314c7c42af81a238c3838")
    },
    "text" : "Foo coupon",
    "dateTime" : "Wed May 12 2010 00:00:00 GMT+0100 (GMT)",
}

I started off looking at the Aggregation page of the MongoDB help documents. This page lists a number of querying options:

• Count – useful if I wanted to know the number of coupons received by just one member, hence not quite what I was after
• Distinct – for a whole other category of problems
• Group – this looked exactly like what I needed, at least until I read the fine print: “Note: the result is returned as a single BSON object and for this reason must be fairly small – less than 10,000 keys, else you will get an exception. For larger grouping operations without limits, please use map/reduce.” We have 631,000 coupon documents. So map/reduce it is…

Following the example given here, I ended up with these functions:

m = function(){ emit(this.recipient.$id, {coupons : 1})};

r = function (key, values) {
    var sum = 0;
    for (var i = 0; i < values.length; i++) {
        sum += 1;
    };
    return { coupons: sum };
}

Running this:

res = db.coupons.mapReduce(m,r);

And then querying the results:

db[res.result].find().sort({"value.coupons" : -1})

Gave the answer I needed:

{ "_id" : ObjectId("4cb314c5c42af81aa14f3838"), "value" : { "coupons" : 2236 } }
{ "_id" : ObjectId("4cb314c8c42af81adbb13838"), "value" : { "coupons" : 1133 } }
{ "_id" : ObjectId("4cb314c7c42af81ab2923838"), "value" : { "coupons" : 782 } }
...

This is a fairly straight forward map/reduce and was quite a nice introduction to them.

Scalatra, Scalate and Scaml

A while ago I set about creating a webapp for monitoring the status of the various Quartz jobs that we use to keep our main application ticking. It was put together quickly using Simple Build Tool and Scalatra. I needed a templating engine and noticed that Scalatra has support for Scalate (although at the time the support was only experimental). I decided to give it a try. The first step was to add the dependencies for Scalate and Scalatra-Scalate into our build file:

val scalatraScalate = "org.scalatra" % "scalatra-scalate_2.8.0" % "2.0.0.M1"
val scalateCore = "org.fusesource.scalate" % "scalate-core" % "1.2"

I soon ran into the Scalatra issue with using Scalate and Logback and instead decided to use Scalate on its own. Referring to the Scalate Embedding Guide, I created an instance of the Scalate TemplateEngine class in a trait that is extended by my Scalatra servlet:

trait ScalateTemplateEngine {
   def render(templatePath: String, context: Map[String, Any], response: HttpServletResponse) = {
       val templateEngine = new TemplateEngine
       val template = templateEngine.load(templatePath)

       val buffer = new StringWriter()
       val renderContext = new DefaultRenderContext(templateEngine, new PrintWriter(buffer))

      context.foreach({case (key, value) => renderContext.attributes(key) = value})
      template.render(renderContext)

      response.getWriter.write(buffer.toString)
   }
}

Then, referring to this excellent blog post, I created an index.scaml file. The scaml file (below) is concise and reasonably readable. The only hiccups I encountered in making it were:

1. Figuring out the syntax and correct indenting of the for loop
2. Realising that all the values the map are automatically converted to Options (hence the many get calls).

-@ val title: String = "Scheduler"
-@ val header: String = "Scheduler"
-@ val quartzStatus: String = "No quartz status information available"
-@ val scheduledTasks: List[Map[String, String]] = Nil
-@ val quartzInformation: String = "No quartz information available"
!!! 5
%html
  %head
    %title= title
  %body
    %h1
      != header
    %h5
      != quartzStatus

    %table{:border => 1}
      %thead
        %td Task
        %td Status
        %td Quantity
        %td Last Attempt Start Time
        %td Last Attempt Finish Time
        %td Last Successful Run Start Time
        %td Last Successful Run Finish Time
        %td Last Error
        %td Action
      = for(scheduledTask <- scheduledTasks)
        %tbody
          %td= scheduledTask.get("name").get
          %td= scheduledTask.get("status").get
          %td= scheduledTask.get("quantity").get
          %td= scheduledTask.get("lastRunStart").get
          %td= scheduledTask.get("lastRunFinish").get
          %td= scheduledTask.get("lastSuccessStart").get
          %td= scheduledTask.get("lastSuccessFinish").get
          %td= scheduledTask.get("lastException").get
          %td
            %form(method="post")
              %input(type="submit" name={scheduledTask.get("action").get} value="Run now")

Using Mockito in a Scala unit test

Our project has been using ScalaTest for unit and integration testing. For some of unit tests we have been using the Mockito mocking library. Recently I was caught out on two occasions when I was writing a Scala unit test that had some mocked classes.

The first head scratching moment was caused by a verification statement like this one:

verify(myClass).createSomething(org.mockito.Matchers.eq(name), anyString)

I had added an import statement for the Matchers class and expected to be cooking with gas….. but then this error appeared:

[error] MyClass.scala:47: type mismatch;
[error]  found   : Boolean
[error]  required: String
[error]     verify(myClass).createSomething(eq(name), anyString)
[error]                                      ^
[error] one error found

Why was the eq method returning a Boolean and not the type of the name variable (ie String)? I had written similar things in Java many times before. I realised that the eq method being used is the one that is defined in the Scala AnyRef class:

def   eq  (arg0: AnyRef)  : Boolean

This was simply fixed by explicitly calling the eq method from the Matchers class:

verify(outboundEmailService).createSomething(org.mockito.Matchers.eq(listName), anyString)

A little later on I ran into another problem with a verification. Once again, I thought there was not much to it:

verify(myClass).sendToGroup(org.mockito.Matchers.eq("foo"), startsWith(name), org.mockito.Matchers.eq(something))

But then….

org.mockito.exceptions.misusing.InvalidUseOfMatchersException:
Invalid use of argument matchers!
0 matchers expected, 3 recorded.
This exception may occur if matchers are combined with raw values:
//incorrect:
someMethod(anyObject(), "raw String");
When using matchers, all arguments have to be provided by matchers.
For example:
//correct:
someMethod(anyObject(), eq("String by matcher"));

I eventually realised that this meant that I had not provided matchers for all the parameters. In fact the method does have a fourth parameter, but it has a default value. I hadn’t expected to need to provide a matcher for the fourth parameter but as the docs state “If you are using argument matchers, all arguments have to be provided by matchers.”