Liz Douglass

Posts Tagged ‘Scala

Things that made me go hmmmm…

with 2 comments

Over the last couple of months I’ve been working on a Scala project. There is nothing quite like immersing yourself in a language to increase your knowledge. Along the way I figure that I must have made every possible syntax mistake. This post is about three gotchas – detail of the language has tripped me up.

Method invocation with parentheses

As a general rule, it’s not necessary to use parentheses when invoking Scala methods that have either none or one argument. There are two cases where this does not apply however:

1. Default arguments:
Soon after making the switch over from Scala 2.7.7 to 2.8.0 we made our first attempt at using a default parameter. It went something like this:

object MyObject {
  def something(someArg: String = "Fred"){ println(someArg) }
}

class Foo() {
  MyObject something
}

This results in this error:

error: missing arguments for method something in object MyObject;
follow this method with `_' if you want to treat it as a partially applied function
         MyObject something

As pointed out here “If a method definition includes parentheses, when you call it you must also use parentheses. Presumably this still holds if all the arguments to the method have default values.”

2. Using println:

scala> println 10
<console>:1: error: ';' expected but integer literal found.
       println 10
               ^
scala> println(10)
10

According to Programming In Scala: “Note that (the no parentheses) syntax only works if you explicitly specify the receiver of the method call. You cannot write ‘println 10’, but you can write ‘Console println 10’.”

Case?

In my opinion one of the best things about Scala is the first-order functions that are defined on the TraversibleOnce trait. I’d use a couple of them on any given day. I’ve often needed to perform an operation on each key-value pair in a Scala Map. On more than one occasion I’ve started off writing something like this:

val myMap = Map("hay" -> "stack", "killer" -> "whale")
myMap.filter((key, value) => key == "killer").map((key, value) => value)

Which results in this error:

error: wrong number of parameters; expected = 1

Doing this another way using a for expression gives a better result however:

scala> for((key,value) <- myMap if (key == "killer")) yield(value)
res19: scala.collection.immutable.Iterable[java.lang.String] = List(whale)

Programming In Scala explains that the Scala compiler translates all for expressions into combinations of higher order methods:
“A for expression of the form:
for (x <- expr1 if expr2) yield expr3
is translated to:
for (x expr2)) yield expr3

and: “A for expression of the form:
for((x1, ..., xn) <- expr1) yield expr2
translates to:
expr.map{case (x1,...,xn) => expr2}"

So if I want to use higher order methods as I attempted to do first, I need:

scala> myMap.filter({case(key, value) => key == "killer"}).map({case(key, value) => value})
res29: scala.collection.immutable.Iterable[java.lang.String] = List(whale)

Or the equivalent shorter version:

scala> myMap.filter(_._1 == "killer").map(_._2)
res30: scala.collection.immutable.Iterable[java.lang.String] = List(whale)

Booleans

A java.lang.Boolean does not implicitly convert to a Scala Boolean. You need to convert the java.lang.Boolean into a java boolean type using the booleanValue method in order for the conversion to be possible:

scala> val javaBoolean: java.lang.Boolean = true
javaBoolean: java.lang.Boolean = true

scala> val scalaBoolean: Boolean = javaBoolean
<console>:13: error: type mismatch;
 found   : java.lang.Boolean
 required: scala.Boolean
       val scalaBoolean: Boolean = javaBoolean
                                   ^

scala> val scalaBoolean: Boolean = javaBoolean.booleanValue
scalaBoolean: Boolean = true

Written by lizdouglass

December 15, 2010 at 9:25 am

Posted in Uncategorized

Tagged with

Scala Multiple Constructors

with 3 comments

A couple of weeks ago I created a new type of exception for use in our application. If I’d been using Java I would have written something simple like this:

public class MyException extends RuntimeException {

    public MyException(String message) {
        super(message);
    }

    public MyException(String message, Throwable throwable) {
        super(message, throwable);
    }
}

The Scala multiple constructor model is different to the Java one, as described in this article. In Scala, only the primary class constructor can call a superclass constructor, meaning that it’s only possible to invoke only one superclass constructor. The code above makes calls to two constructors on the RuntimeException class. So how can we write a Scala equivalent? Fortunately one of the answers posted to this question explains a pattern for how to do this:

object MyException {
  private def create(message: String): RuntimeException = new RuntimeException(message)

  private def create(message: String, throwable: Throwable): RuntimeException = new RuntimeException(message, throwable)
}

class MyException private (exception: RuntimeException) {
  def this(message: String) = this (MyException.create(message))

  def this(message: String, throwable: Throwable) = this (MyException.create(message, throwable))
}

I think the pattern is really neat but certainly a lot more complicated that the Java equivalent. I guess this is one of the cases where using Java classes in Scala can be tricky.

Written by lizdouglass

December 15, 2010 at 9:24 am

Posted in Uncategorized

Tagged with ,

Scala Singleton Objects

with one comment

There are two types of singleton objects in scala – companion objects and standalone objects. According to Programming in Scala:

“A singleton object definition looks like a class definition, except instead of the keyword class you use the keyword object… When a singleton object shares the same name with a class, it is called that class’s companion object. You must define both the class and its companion object in the same source file. The class is called the companion class of the singleton object. A class and its companion object can access each other’s private members… A singleton object that does not share the same name with a companion class is called a standalone object. You can use standalone objects for many purposes, including collecting related utility methods together, or defining an entry point to a Scala application”

Whilst I like the pattern of separating what would otherwise be static methods in Java into companion objects, I do find at times that they are a bit unintuitive, specifically in these two cases:

1. As shown in the brief example below it’s necessary to include the name of the companion object when invoking one of its methods, even from within the companion class. Coming from a Java background and therefore not being used to having static methods in a separate object, I’ve found that I’ve on occasion forgotten to do this.

class Foo {

  def bar {
    println(Foo.createName)
  }

}

object Foo {

  def createName: String = Math.random.toString

}

Programming in Scala has an example with a slightly neater way of handling this situation – including an import statement at the top of the companion class:

class Foo {
  import Foo._

  def bar {
    println(createName)
  }

}

object Foo {

  def createName: String = Math.random.toString

}

2. Along a similar line, I have also been caught out trying to invoke the methods of a companion object via an instance of the associated companion class. The code below shows how its still necessary to call the createName method from the Foo companion object even though we have the someFoo Foo instance:

class Foo {

  def bar(name: String){
    println(name)
  }

}

object Foo {

  def createName: String = Math.random.toString

}

class Consumer {

  val someFoo = new Foo
  someFoo.bar(Foo.createName)

}

Written by lizdouglass

December 15, 2010 at 9:24 am

Posted in Uncategorized

Tagged with ,

Overloaded methods in Scala

leave a comment »

I recently discovered that it’s not possible to have default parameters on more than one overloaded method alternative in Scala. The methods that I was working on looked a bit like this:

class Bar() {
  def foo(data: Map[String, String], list: String, field: String = "username"): Boolean = { 
      //do some processing
      true
  }

  def foo(data: String, list: String, field: String = "username"): Boolean = {
      //do some processing
      true
  }
}

Given that it’s possible to work out which alternative needs to be called from the type of the first argument, I incorrectly assumed that these declarations would be ok . As a work-around I created another pair of overloaded methods (see below). If you know of a better alternative then please let me know.

class Bar() {
  def foo(data: Map[String, String], list: String, field: String): Boolean = {
      //do some processing
      foo(data, list, field)
  }

  def foo(data: String, list: String, field: String): Boolean = {
      //do some processing
      true
  }

  def foo(data: Map[String, String], list: String): Boolean =
      foo(data, list, "username")

  def foo(data: String, list: String): Boolean =
      foo(data, list, "username")
}

Written by lizdouglass

December 15, 2010 at 9:23 am

Posted in Uncategorized

Tagged with

Scalatra, Scalate and Scaml

with one comment

A while ago I set about creating a webapp for monitoring the status of the various Quartz jobs that we use to keep our main application ticking. It was put together quickly using Simple Build Tool and Scalatra. I needed a templating engine and noticed that Scalatra has support for Scalate (although at the time the support was only experimental). I decided to give it a try. The first step was to add the dependencies for Scalate and Scalatra-Scalate into our build file:

val scalatraScalate = "org.scalatra" % "scalatra-scalate_2.8.0" % "2.0.0.M1"
val scalateCore = "org.fusesource.scalate" % "scalate-core" % "1.2"

I soon ran into the Scalatra issue with using Scalate and Logback and instead decided to use Scalate on its own. Referring to the Scalate Embedding Guide, I created an instance of the Scalate TemplateEngine class in a trait that is extended by my Scalatra servlet:

trait ScalateTemplateEngine {
   def render(templatePath: String, context: Map[String, Any], response: HttpServletResponse) = {
       val templateEngine = new TemplateEngine
       val template = templateEngine.load(templatePath)

       val buffer = new StringWriter()
       val renderContext = new DefaultRenderContext(templateEngine, new PrintWriter(buffer))

      context.foreach({case (key, value) => renderContext.attributes(key) = value})
      template.render(renderContext)

      response.getWriter.write(buffer.toString)
   }
}

Then, referring to this excellent blog post, I created an index.scaml file. The scaml file (below) is concise and reasonably readable. The only hiccups I encountered in making it were:

  1. Figuring out the syntax and correct indenting of the for loop
  2. Realising that all the values the map are automatically converted to Options (hence the many get calls).
-@ val title: String = "Scheduler"
-@ val header: String = "Scheduler"
-@ val quartzStatus: String = "No quartz status information available"
-@ val scheduledTasks: List[Map[String, String]] = Nil
-@ val quartzInformation: String = "No quartz information available"
!!! 5
%html
  %head
    %title= title
  %body
    %h1
      != header
    %h5
      != quartzStatus

    %table{:border => 1}
      %thead
        %td Task
        %td Status
        %td Quantity
        %td Last Attempt Start Time
        %td Last Attempt Finish Time
        %td Last Successful Run Start Time
        %td Last Successful Run Finish Time
        %td Last Error
        %td Action
      = for(scheduledTask <- scheduledTasks)
        %tbody
          %td= scheduledTask.get("name").get
          %td= scheduledTask.get("status").get
          %td= scheduledTask.get("quantity").get
          %td= scheduledTask.get("lastRunStart").get
          %td= scheduledTask.get("lastRunFinish").get
          %td= scheduledTask.get("lastSuccessStart").get
          %td= scheduledTask.get("lastSuccessFinish").get
          %td= scheduledTask.get("lastException").get
          %td
            %form(method="post")
              %input(type="submit" name={scheduledTask.get("action").get} value="Run now")

Written by lizdouglass

December 15, 2010 at 9:22 am

Posted in Uncategorized

Tagged with , ,

Using Mockito in a Scala unit test

leave a comment »

Our project has been using ScalaTest for unit and integration testing. For some of unit tests we have been using the Mockito mocking library. Recently I was caught out on two occasions when I was writing a Scala unit test that had some mocked classes.

The first head scratching moment was caused by a verification statement like this one:

verify(myClass).createSomething(org.mockito.Matchers.eq(name), anyString)

I had added an import statement for the Matchers class and expected to be cooking with gas….. but then this error appeared:

[error] MyClass.scala:47: type mismatch;
[error]  found   : Boolean
[error]  required: String
[error]     verify(myClass).createSomething(eq(name), anyString)
[error]                                      ^
[error] one error found

Why was the eq method returning a Boolean and not the type of the name variable (ie String)? I had written similar things in Java many times before. I realised that the eq method being used is the one that is defined in the Scala AnyRef class:

def   eq  (arg0: AnyRef)  : Boolean

This was simply fixed by explicitly calling the eq method from the Matchers class:

verify(outboundEmailService).createSomething(org.mockito.Matchers.eq(listName), anyString)

A little later on I ran into another problem with a verification. Once again, I thought there was not much to it:

verify(myClass).sendToGroup(org.mockito.Matchers.eq("foo"), startsWith(name), org.mockito.Matchers.eq(something))

But then….

org.mockito.exceptions.misusing.InvalidUseOfMatchersException:
Invalid use of argument matchers!
0 matchers expected, 3 recorded.
This exception may occur if matchers are combined with raw values:
//incorrect:
someMethod(anyObject(), "raw String");
When using matchers, all arguments have to be provided by matchers.
For example:
//correct:
someMethod(anyObject(), eq("String by matcher"));

I eventually realised that this meant that I had not provided matchers for all the parameters. In fact the method does have a fourth parameter, but it has a default value. I hadn’t expected to need to provide a matcher for the fourth parameter but as the docs state “If you are using argument matchers, all arguments have to be provided by matchers.”

Written by lizdouglass

December 15, 2010 at 8:58 am

Posted in Uncategorized

Tagged with ,

Migrating to Scala 2.8

leave a comment »

A few months ago Scala 2.8.0 was released. We migrated our project from version 2.7.7 quite soon after the announcement. Making the switch was quite straight forward, in fact all we needed to do was change the build.scala.versions property in our Simple Build Tool build.properties file. Admittedly it took several hours to fix all the compilation errors, but once this was done we found we found we were able to make our code base more readable because of two shiny new things in particular:

1. Collections:

As I’ve written about before, we are using MongoDB for persistence. We are also using the MongoDB Java driver. This library makes heavy use of BasicDBObjects. These are the simplest implementation of the DBObject interface. This interface represents “A key-value map that can be saved to the database”. BasicDBObjects are used for more than just inserting data. In fact they are also used extensively in many of the methods defined on the DBCollection class. For example, this is the definition of the update method:

WriteResult update(DBObject q, DBObject o)

(Source: class com.mongodb.DBCollection)
(q is the query used to find the correct element and o is the update operation that should be performed)

We often use the BasicDBObject constructor that takes a Java map. When were using Scala 2.7.7 we had a library to handle the conversion of maps from Scala to Java. We ended up with a lot of repository methods that were littered with calls to asJava, like the one below:

import org.scala_tools.javautils.Imports._
import com.mongodb._
import org.bson.types.ObjectId

def updateLastLogin(personId: String) {
   val criteria = new BasicDBObject(Map("_id" -> new ObjectId(personId)).asJava)
   val action = new BasicDBObject(Map("$set" -> new BasicDBObject(Map("lastLogin" -> new Date).asJava)).asJava)
   collection.update(criteria, action)
}

Now in Scala 2.8 there is a new mechanism for converting collection types from Scala to Java and thankfully all the asJava calls have disappeared from our codebase. Now we have cleaner functions like this one:

def markLogin(memberId: String) = set(new MemberId(memberId).asObjectId, "lastLogin" -> new Date)

Where:

def set(id: ObjectId, keyValues: (String, Any)*) = collection.update(Map("_id" -> id), $set (keyValues: _*), false, false)

Note that we are now also using the Casbah library. The update method above is defined on the MongoCollectionWrapper in the Casbah library.

2. JSON

We have a Scala backend API and a Django frontend. The backend serves JSON to our frontend. We are using the Lift Json library to do both the serialising in the backend webapp, as well as the de-serialising in our integration tests. Some of our tests use the Lift Json Library “LINQ” style JSON parsing, while others use the “Xpath” style (both of which are described here):

LINQ style:

import net.liftweb.json.JsonParser._
import net.liftweb.json.JsonAST._
...

val (data, status) = ApiClient.get(myUri)
val json = parse(data)

val myValues = for{JField("something", JString(myValue)) <- (json \ "somethingElse")} yield myValue
myValues should have length (3)
myValues should (
    contain("frog")
    and contain("tennis")
    and contain("phone")
)

Xpath style:

scenario("succesfully view something") {
   val json = getMemberLiftJson(someId)
   assert((json \ "bar").extract[String] === "ExpectedString")
}

The Xpath style is readable and compact and we’ve retained quite a few of these type of tests in our code base. Personally I find the LINQ style quite confusing. I need to refer to the Lift project github site every time I try to use it. In Scala 2.8 there is now JSON support and we have instead been using this for parsing in the tests. Specifically, we have been using the parseFull method like so:

scenario("bar") {
    val (content, status) = ApiClient.get(someUri)
    JSON.parseFull(content) match {
        case Some(data : List[Any]) => {
            data.length should be(4)
            data should not contain("Internet")
        }
        case json => fail("Couldn't find a list in response %s".format(json))
    }
}

Go Scala 2.8!

Written by lizdouglass

November 1, 2010 at 9:48 pm

Posted in Uncategorized

Tagged with