Liz Douglass

Archive for July 2010


with 2 comments

Recently I started on a project that is using some interesting technologies including Scala, MongoDB and Django. Some are quite new to me and I’ve learnt a great deal. Here are some observations of the things I’ve learnt.


Mongo is a schema-less document-oriented database that stores data in binary encoded JSON documents – BSON documents. The online documentation is quite good and there is a good tutorial and an online interactive shell.

How have we been using Mongo so far?

We have two projects that interact with with Mongo; one is a RESTful API back-end and the other is a tool for populating a Mongo database with data from a MySQL database. Both these projects are written in Scala and use the Mongo Java API.

Why Mongo?

РThe JSON-like documents allows us to store data about in a way that is obvious because they read like plain English.  We need to store information about the users of our system. In nearly all cases, the information available is different for each person Рsome people have no contact phone numbers, others have 6 children. Mongo has allowed us to build up profiles for our users and only include the pieces of information that we actually have available for them.

– The schema-less and denormalised nature of the database means that we can modify the structure frequently. We only started this project a couple of weeks ago and have already made several quite large changes. These include how we organise the Mongo documents that we’ve been generating from a legacy MySQL database. This sort of flexibility is fantastic, especially at the beginning of a new project.

Populating the Mongo database

Our data migration project extracts data for each record in the MySQL database using a SQL query. This data is then used to create a Person domain object, which is composed of microtypes like the one below. The Person type, as well as all of these types, implement the ConvertableToMongo trait:

class NextOfKin(val relationship: Relationship, val person: Person) extends ConvertableToMongo {
  def toMongoObject: DBObject = {
    new BasicDBObject(Map(
      "relationship" -> relationship.toMongoObject,
      "person" -> person.toMongoObject).asJava)


class Relationship(val description: String) extends ConvertableToMongo {
  def toMongoObject(): DBObject = {
    new BasicDBObject(Map("relationship" -> description).asJava)

ConvertableToMongo is a trait:

trait ConvertableToMongo {
  def toMongoObject: DBObject

Note that we need to use the asJava method from the scala-javautils library convert the Scala map to the requisite Java map required by the Mongo API.

The ConvertableToMongo trait has a single method that returns a Mongo DBObject. These are inserted into a Mongo collection like this:

val usersCollection = mongo.getCollection("user")
members foreach(user  => usersCollection insert(user toMongoObject))

The end result is a Mongo document like this one:

	"_id" : ObjectId("4c29f7fdbe924173a47a759f"),
	"firstName" : "Joe",
	"surname" : "Bloggs",
	"gender" : "Male",
	"nextOfKin" : {
		"relationship" : "Son",
		"person" : {
			"name" : "John Bloggs"

Note that unless specified every document added to a Mongo collection will automatically be assigned an ObjectId with the key _id.


Written by lizdouglass

July 19, 2010 at 8:31 am

Posted in Uncategorized

Tagged with