Designing your DynamoDB tables efficiently and modelling mixed data types with Kotlin

AWS (Amazon Web Services) offers a pretty neat NoSQL database called DynamoDB. It is fast and it can scale, what more can you wish for? The thing is, as a developer you are still responsible for designing your tables in such a way, that you actually make appropriate use of the benefits DynamoDB has to offer. If you simply apply your knowledge gained using other databases, you might end up wasting money and performance.

Choosing the best key to query

DynamoDB can be described as a key-value datastore. Each row of data consists of a key, which is indexed, and one or more values, which are not necessarily indexed. Since it is most efficient to query the database using this indexed key, choosing the best key to query becomes important. For some data models this is quite straightforward. For example, if you want to store orders or customers in a table and they have a unique id, this field will naturally be what you use as primary key.

However, when modelling one-to-many data relationships it is not so straightforward anymore. There are various ways to solve this, but I’ll demonstrate one strategy which I think works well. You can easily find more information on the different approaches, for example here.

Composite primary key

In DynamoDB the primary key of a table can be composed of multiple columns. Let’s consider the following example, with a composite primary key consisting of a partition and a sort key.

This example table models the processing capacity of certain facilities and consists of four columns:

• facility number: a unique identification number of each facility;
• moment: either a day of the week for regular capacity, or a specific date, for exceptional capacity;
• capacity: the amount of products which can be processed at the facility, on the given day;
• description: an optional field to describe why there is an exceptional capacity on that date.

Mixing data types in one column

If you look at the second column, moment, you might think: “wait a second, why would you ever mix multiple data types in a single column?” This is not how you would model the data in a relational database, such as MySQL. Neither is it how you would put it in a document database, such as MongoDB.

The first column is not unique by itself, in this table. It can be the partition key (also called hash key), but not the primary key. However, together with the sort key (also called range key), it is always unique, because each day of the week or each date only occur once. If we would have put the day of the week and the date in separate columns, our primary key would have to consist of three columns, which is not allowed. That’s why we chose this particular primary key consisting of the facility number as partition key and a mix of weekday and date, as sort key.

From theory to practice

In theory, this is how you could design this DynamoDB table. Now let’s take a look at how to implement this in practice. Eventually it will be your code which reads and writes records to your DynamoDB table. The structure of the table must be expressed in the code as well. I will demonstrate this with Kotlin using the AWS SDK for Java, but other programming languages could be used as well.

Modelling the DynamoDB table in Kotlin

How do we model this in Kotlin? Let’s look at the following code snippet.

import com.amazonaws.services.dynamodbv2.datamodeling.DynamoDBAttribute
import com.amazonaws.services.dynamodbv2.datamodeling.DynamoDBHashKey
import com.amazonaws.services.dynamodbv2.datamodeling.DynamoDBRangeKey
import com.amazonaws.services.dynamodbv2.datamodeling.DynamoDBTable
import com.amazonaws.services.dynamodbv2.datamodeling.DynamoDBTypeConverted
@DynamoDBTable(tableName = "facility-capacity")
data class FacilityCapacity(
  @DynamoDBHashKey
  var facilityId: String? = null,
  @DynamoDBRangeKey
  @DynamoDBTypeConverted(converter = MomentConverter::class)
  @DynamoDBAttribute
  var moment: Moment? = null,
  @DynamoDBAttribute
  var capacity: Long? = null,
  @DynamoDBAttribute
  var description: String? = null,
)

The table is modelled as a data class and the columns as fields of the class. The field facilityId is annotated as hash key (or partition key) and the field moment is annotated as range key (or sort key). The other fields are simply attributes.

Normally we would use the val keyword for fields in data classes, to mark them as immutable, but due to constraints on how the AWS Java SDK initializes this data, that’s unfortunately not so easy.

Model union type in Kotlin

Let’s take a closer look at the moment field. From our table design we know it can be either a day of the week (such as Tuesday) or a date (such as 13 March 2021). When one field can be multiple types, we call this a union type. You might have encountered this concept in other programming languages, such as in Typescript. How do we model this in Kotlin?

Using sealed classes

The answer is, using sealed classes. Let’s take a look at the following code snippet.

import java.time.DayOfWeek
import java.time.LocalDate
sealed class Moment
data class WeekDay(
  val dayOfWeek: DayOfWeek
) : Moment()
data class ExceptionDate(
  val date: LocalDate,
) : Moment()

The sealed class is sort of an abstract class, which can have multiple implementations. Our sealed class Moment has two implementations here, the regular WeekDay which holds a day of the week and the ExceptionDate, holding a specific date.

Reading from and writing to the column with mixed data

DynamoDB stores our moment column simply as a string. It is our responsibility to read from it and write to DynamoDB, and do the necessary type conversion. This code snippet shows what our converter looks like.

import com.amazonaws.services.dynamodbv2.datamodeling.DynamoDBTypeConverter
import org.slf4j.Logger
import org.slf4j.LoggerFactory
import java.time.DayOfWeek
import java.time.LocalDate
class MomentConverter : DynamoDBTypeConverter<String?, Moment?> {
  override fun convert(moment: Moment?): String? {
    return moment?.let {
      when (it) {
        is WeekDay -> it.dayOfWeek.toString()
        is ExceptionDate -> it.date.toString()
      }
    }
  }
  override fun unconvert(momentAsString: String?): Moment? {
    return momentAsString?.let {
      try {
        ExceptionDate(
          date = LocalDate.parse(it)
        )
      } catch (e: Exception) {
        try {
          WeekDay(
            dayOfWeek = DayOfWeek.valueOf(momentAsString)
          )
        } catch (e: Exception) {
          logger.error("Could not parse input '$momentAsString' to Moment.", e)
          null
        }
      }
    }
  }
  companion object {
    val logger: Logger = LoggerFactory.getLogger(this::class.java)
  }
}

The MomentConverter class extends from the DynamoDBTypeConverter class and can convert data between a String (how it’s stored in DynamoDB) and a Moment (how it’s represented in our code). The MomentConverter class implements two functions:

• convert (from Moment to String)
• unconvert (from String to Moment)

The convert function uses Kotlin’s switch expression to call the appropriate toString() method, depending on the type of Moment (since if it is a LocalDate, we want a proper ISO-8601 formatted string).

The unconvert function is a bit more complicated. We have a string and have to figure out what type it is. First we try to parse the string as LocalDate. If the input is not a LocalDate, we try to parse it as a DayOfWeek. If that doesn’t work either, we log an error.

Conclusion

That’s it, implementing this database table design is as easy in practise, as it is in theory. To summarize, we looked at how we can model our data structure in a DynamoDB table, using a composite primary key of two columns, where one column contains multiple data types. We saw how to implement this union type in Kotlin using a sealed class. Lastly, the converting and unconverting of this mixed data type column was demonstrated.

I hope this gave you some new ideas on how to use DynamoDB efficiently. Be aware that this is just the tip of the iceberg. If you want to go deeper into the rabbit hole, I would recommend watching one of Rick Houlihan talks, such as this one.