Functional Programming etc.

Simple Typed Maps

Tue, 16 Jun 2015 04:10:00 +0000

If you’ve ever wanted to store and access values of different types using an ordinary Map, here’s what you need :

It’s just 20 lines of code, with no dependencies. You can drop it into your project and start using it right away. Don’t worry if the code doesn’t make sense, we’ll go over the implementation in detail later. But let’s try it out first, starting with a Key (you might want to follow along in a REPL of your own) :

In the key k1, apart from the string "a", we’re also storing information about the type of value it will point to in the Map using a TypeTag. You can learn about TypeTags and type erasure here, but the basic idea is that they let you work with types at runtime.

Key is a simple case class, except that the equals method is overridden to compare the underlying key as well as the value type correctly (you can’t compare TypeTags using ==). Let’s try to create a Map using k1 :

Surprise! It doesn’t compile. Why is that? Well, the -> operator in scala is not a primitive. It is implemented by means of an implicit class, and creates a Tuple2 of the arguments around it. We would like to exercise some control over the type of values a Key can be paired with, so we have disabled the implicit -> operator by defining it as a method on Key. This method returns None, which cannot be provided as a constructor argument to Map.

To pair a Key with a value, we shall use the type safe arrow method ~> :

Since k1 is of type Key[String, Int], ~> expects an Int, and compilation fails if we try to give it anything else. Let’s create a valid Map and try to access the value stored in it :

We were able ensure that k1 points an Int value, but when we try to access it, we get back a value of type Any, which is no good! Let’s fix this using TypedMap, which is a wrapper around Map that lets you get back values with the right types :

Voila! The value we get out of the map m using k1 magically has the right type (we’re casting it under the hood, but let’s keep that between us). Here’s a map with different types of values :

We can recover each of the values with the right type using just the keys :

And we can work with keys that aren’t present in the map just as easily :

At this point, you might want to scroll back up all the way and go through the implementation of Key and TypedMap, and it should make a lot more sense. Now you can use a Map just like you would a case class or a tuple. Or you could just use a case class, right? Well, case classes and tuples are great, except when they’re a huge pain the ass. Specifically :

Tuple elements have to be accessed as ._1 , ._2 etc., which is just plain ugly. Case classes solve this by letting you name the elements. TypedMaps let you use pretty much anything as a key, and you can decide which ones to use at runtime.
You cannot build a tuple or a case class in steps, you have to build it all in one go. You might wrap the elements in Options, but then you find yourself calling .get (or pattern matching) all over the place. TypedMaps can be built in steps by adding key-value pairs one by one.
Case classes and tuples are unbelievably rigid, to the point that it is annoying. They don’t compose at all! You can’t concatenate tuples, and if you want to combine case classes you will have to write a ton of boilerplate to do it. TypedMaps are ordinary Maps, so you can combine them quite easily.
Sometimes you need a traditional Map i.e. a set of key-value pairs, with keys and values that can be passed around independently and used across multiple maps. A case class is simply too restrictive in such situations.

While TypedMap and Key are good to use as defined above, there’s bit of boilerplate involved in using them. Let’s define some helper classes to help clean up the syntax :

With the help of these implicit classes, it becomes much easier to create and use TypedMaps :

That’s pretty much it! Feel free to use it, but keep in mind that this is a simple (albeit functional) implementation and has several shortcomings. For instance, you can ask for a key that doesn’t exist and it will blow up at runtime, like an ordinary Map would. Also, since you have access to the underlying Map, you can call methods like .map, .flatMap etc. on it and totally mess up the whole thing and get all kinds of runtime exceptions. This can be avoided by making the underlying Map private and exposing only a limited set of methods on TypedMap. If you need better compile time safety and more features, you should check out scala records or shapeless records. I would suggest that you use case classes whenever you can, but when you find that your head hurts trying to do what you want, use TypedMaps (or something similar). Here’s the full implementation (raw) :

Scrap Your Type Class Boilerplate (2/2)

Mon, 13 Apr 2015 11:30:14 +0000

For the TL; DR version, go straight to the Summary.

Recap
More Improvements
Summary

Recap

The previous post covered some ways to make type classes easier to use. For a quick recap, see the summary. In this post, we will attempt to eliminate some of the boilerplate associated with creating and using type classes and their instances. Here is the implementation of the Group type class that we had at the end of the previous post :

Inside the Group companion object, we define instances of Group for Int, Double and pairs :

And here, the type class is used to define some generic functions :

More Improvements

One of the concerns raised earlier but not addressed was that the code for defining type class instances is rather verbose and repetitive, and if the type class has lots of abstract methods or if you’re defining many instances, this can lead to a lot of boilerplate. I alluded to something called the pain of overriding, which is the root of much of this boilerplate. Here is the definition ~~from Wikipedia~~ :

The pain of overriding : The #Scala compiler can infer batshit crazy types, still demands the full signature to override/implement methods.
— Aakash N S (@aakashns) April 4, 2015

It is annoying to write all these types, and it becomes much more annoying when you have to change something, because you have to make the same change in SO many places, and so you end up copy-pasting things all over the place, but that leaves you with a bunch of compilation errors, and then you spend a day figuring out the types to fix those errors, and then you finally fix them, and you reward yourself with a cookie or whatever, but then two days later you realize you’ve made an error in the logic because you were too busy trying to tell the compiler what it already knows!

Helper Class

To help eliminate some of this boilerplate, let’s define a class GroupAux as follows :

This class does nothing special. It simply moves the methods to be implemented into constructor arguments. Using this helper class, the code for defining instances of Group for Int and Double looks like this :

Isn’t this much nicer? By using anonymous functions as constructor arguments to GroupAux, we’re letting the compiler do all the hard work of figuring out the right types. By the way, anonymous functions are also called lambdas (λ). And here’s the definition for pairGroup :

You can make the anonymous functions more descriptive, if you’re not into the whole brevity thing :

Admittedly, this is not a huge improvement if you only have a few instances of the type class. Some people would argue that this not an improvement at all, because it compromises on readability. Here are a few arguments in favor of this style :

It avoids the pain of overriding. You no longer have to tell the compiler what it can figure out on its own. If the instances of your type class go into dozens, or if you have many abstract methods to be implemented, this can save you the trouble of having to write all the types all the time.
Anonymous functions are not unreadable if you already know the types. That’s precisely why we use them. When both the reader and the compiler already know the types, they’re just noise in the code. Moreover, you can optionally provide the types, if you really want to.
It’s not all or nothing. You don’t have to use it all the time. Use it when it saves you precious lines without compromising on readability, like in the case of IntGroup and DoubleGroup. If you think the definition of PairGroup feels unreadable, don’t use GroupAux! Extend Group and override the methods directly.

Note that this is a general technique that you can apply to any trait / abstract class. Just define a helper class that takes function values as constructor arguments, and uses them to implement the abstract methods.

The TypeClassCompanion Trait

This is another minor improvement, that can help you avoid having to write the apply method for your type class companion object. We define a trait called TypeClassCompanion that companion objects can extend, to automatically get the apply method :

Note that the apply method is marked as @inline, which tells the compiler to replace calls like Group[Int] with implicitly[Group[Int]]. Not surprisingly, the implicitly method in Predef is inlined by the complier too, so the compiler will replace implicitly[Group[Int]] with an implicit value of type Group[Int] summoned from the nether world. Long story short, the compiler will replace the expression Group[Int] with IntGroup.

Using the TypeClassCompanion trait, the Group companion object looks like this :

There isn’t much more to it than that. It saves you one line of code for each of your type classes.

Implicit Class for Pretty Syntax (or not)

With context bounds and the apply method on the companion object, the syntax for using type classes is already pretty clean. However, it is often the case that the methods defined in the type class are unary (a function of type T => T) or binary (a function of type (T, T) => T) operations. This is certainly true for Group : plus and minus are binary operations, and inverse is a unary operation. Wouldn’t it be nice if we could simply write (1, 2) + (3, 4) instead of Group[(Int, Int)].plus((1, 2), (3, 4))? Here’s some advice I just made up :

As a rule of thumb, the answer to every question of the form “Can I do XYZ in #Scala?” is “Yeah, totally man! Just use implicits.”
— Aakash N S (@aakashns) April 13, 2015

Armed with this newfound wisdom, let’s define a class called GroupOps that wraps objects and adds methods for group operations on the underlying objects :

By making GroupOps an implicit class, we’re simply telling the compiler to add an implicit conversion from T to GroupOps for types T that are members of the type class Group. The above piece of code is equivalent to the following :

Since we cannot write a def in the global scope, implicit classes can only be defined inside another class / trait / object, in this case GroupSyntax. Let’s use the new syntax to simplify the code for pairGroup and the sum functions :

Clearly, this technique substantially cleans up the interface for using group operations. But despite the benefits, you may not want to do this all the time, because :

It uses an implicit conversion. It is not easy to tell where the operator |+| came from unless you know where to look. Many people find this confusing, and are going to avoid using it, no matter what you tell them.
It may potentially create a new instance of the type class each time you use it. If you find yourself using Group[(Int, Int)] 10 times in a block, you’re better off defining a local val intPairGroup = Group[(Int, Int)] and using that, because Group[(Int, Int)] results in a call to pairGroup[Int, Int], which instantiates a new object each time it is called. However, if you’re writing a |+| b 10 times in a block, there is no easy way to avoid calling pairGroup[Int, Int] that many times.

Although the pretty syntax is nice to have, it comes at a cost. You should consider the use-cases of your type classes, and evaluate how they are affected by these costs. This might upset a few people, but I personally think that explicit is indeed better that implicit in this particular instance. So think twice before doing this.

Summary

Type classes are awesome and you should use them everywhere. Make sure you do them right, and everyone will be happy. In addition to the techniques described in the previous post :

Define a helper class to help reduce boilerplate for defining type class instances.
Use a companion trait to automatically add the apply method to your type class companion objects.
You may define implicit classes to provide some nice syntax and pretty operators, but be aware of the costs, and also be aware that many people just won’t use them. This is best provided as an optional feature.

That’s all folks!

Scrap Your Type Class Boilerplate (1/2)

Sun, 29 Mar 2015 11:30:14 +0000

For the TL; DR version, go straight to the Summary.

Introduction
Improvements
Summary

Introduction

For a short introduction to type classes, see my previous post. Type classes are a very useful pattern in Scala, that help keep your code base modular, decoupled and open to extension. Here is a somewhat ~~accurate~~ inaccurate description of type classes :

Type classes in #Scala : Write less code, so that you can spend more time compiling it.
— Aakash N S (@aakashns) March 29, 2015

Writing and using type classes, if not done properly, can entail some boilerplate and some fairly odd syntax for the user. Apart from being something to feel bad about, this can scare people away from using them, by making their heads blow up (at worst) and making code difficult to read or understand (at best). This post will try to identify and address some shortcomings in the implementation of a type class for groups and make it easier to use.

Here is implementation of the Group type class and its instances for a few types, as discussed in the previous post :

And here, the type class is used to define some generic functions :

Improvements

There are some problems with the above implementation :

Any function that uses group operations needs type class instances to be passed in as implicit parameters.
- This makes the signature of the function unnecessarily long, especially for functions like sumNonEmpty, which do not directly use any group operations, but simply call other functions that do. It also makes the code a little mysterious to read, because an argument is passed in but never used.
- It is not immediately clear by looking at the function signature that the implicit parameter is a type class instance. There is no easy way to differentiate it from implicit parameters used for other purposes.
- Implicit parameters have something of a bad reputation, mainly because they are easily misused / overused, leading to confusing code. It may seem crazy, but many people have an aversion to implicits, and will avoid writing methods with implicit parameters at all costs. Like this guy :

The problem with implicit parameters in #Scala is that you often have no idea where they come from or where they're going.
— Aakash N S (@aakashns) March 29, 2015

The code for defining type class instances is rather verbose and repetitive (due to something I like to call “the pain of overriding”), as can be seen in the implementation of IntGroup and DoubleGroup, which are nearly identical, except for the types. If your type class defines lots of abstract methods or if you’re defining many instances of the type class, this can lead to a lot of boilerplate.

Let’s try and fix these problems, and also other problems that arise in the process of fixing these.

Context Bounds

Scala provides syntactic sugar called context bounds to make the type class pattern slightly more visually appealing. With context bounds, the signature of pairGroup becomes :

And the signatures of the sum functions become :

The only difference is that the annoying (implicit tGroup: Group[T]) in the parameter list is replaced by a much nicer [T: Group] in the type parameter list. This is called a context bound, and it is simply syntactic sugar for passing in an instance of Group[T] as an implicit parameter to the function. However, the new function signature is more readable, and it is immediately clear that T is a member of the type class Group.

With the implicit parameter gone from the signature, everyone will be happy, and we can all go home. Except we can’t, because we still need to access the parameter to call the group operations plus, minus and inverse on it. But how do we find the ~~man~~ parameter with no name?

We find ~~him~~ it using the implicitly method, which, to quote the official documentation (not kidding), is used “for summoning implicit values from the nether world”. So you better not type implicitly[Satan] or implicitly[Hitler] or something like that (alright, you can type it, I just did, but please don’t compile it).

Using implicitly, the new implementation of pairGroup looks like this :

And the sum functions look like this :

Great, so we replaced implicit parameters in function signatures with implicitly[Group[T]] spewed all over the code base. Now you know I wasn’t kidding about heads exploding. I’ll let you decide which of the two styles is uglier, but the good news is, we can get rid of them both.

Companion Object’s Apply Method

Let’s add an apply method to the Group companion object :

As you might already know, the apply method is given special treatment in Scala, so that instead of writing Group.apply[T] to call the method, you can write Group[T] and the compiler will replace it with Group.apply[T].

The method itself is so simple that it feels almost unnecessary to even define it, let alone devote an entire section of this blog post to it (after all, the signature of the method is longer that its body). But bear with me for a moment. Using this apply method, the implementation of pairGroup now looks like this :

And the sum functions look like this :

I encourage you to take a moment and marvel at what we have achieved here :

The most obvious difference is that there is no more implicitly[Group[T]]. But if you look a little more closely, you will notice that there are no signs of anything implicit anywhere outside the definition of the type class. Someone using the type class in their functions does not need to know that the implementation uses implicits. In fact, they do not even need to know what implicits are, let alone “summon them from the nether world”.
With the context bound syntax [T: Group] in the function signature and the apply method syntax Group[T] to get the Group for type T, type classes looks like a feature of the language itself. Even your editor will congratulate you on this accomplishment, by syntax highlighting Group[T].plus(...) with pretty colors.

So, with context bounds and the apply method, you can make the lives of the users of your type classes much easier, by letting them bask in the unexpected virtue of ignorance (or Birdman). Also, they can no longer make lame excuses for not using your type classes.

Better Compilation Error Messages

Okay, here’s the thing, I lied to you earlier about the user not needing to know about implicits (but only because the truth wasn’t good enough, and you deserved to have your faith rewarded). Trying to compile the following piece code :

will lead to the following compilation error :

Ah! The ~~Empire~~ Implicit Strikes Back! This the kind of cryptic error message that will make people go “What? Implicit what? Evidence? What? Which parameter? :-( Screw this! I hate Scala!” And even if you do know that the implementation of type classes uses implicits, this error is message it not very helpful because it takes an additional mental step to figure out that it’s basically saying that the compiler couldn’t find an instance of the type class Group for Boolean. Fortunately, Scala gives us a way of fixing this too (is there anything Scala does NOT give?), using a compiler annotation :

That’s it! The error message when trying to compile sum(Seq(true, false)) will now read :

No more cryptic error messages! No more implicit parameters! No more implicitly and no more nether world-summoning! Type classes FTW! Developers be like :

Summary

People don’t like implicits. People don’t like unreadable code. People don’t like cryptic error messages. Make your type classes easy to use. Don’t give people reasons not to use your them.

Define an apply method on the companion object to allow easy access to type class instances :
Use and encourage people to use context bounds (and not implicit parameters) :
Use the implicitNotFound annotation to provide better compilation error messages :

That’s it for the first part of this post. As you might have noticed, we haven’t addressed the second concern raised earlier. We will address it in the second part, and we will also cover a other few techniques to make your type classes even better, especially for you, who’s doing the hard work of defining all the instances.

Type Classes in Scala

Fri, 27 Mar 2015 11:30:14 +0000

Consider the following functions :

sum : which takes a list of integers and returns their sum.
sumDifference : which takes two lists of integers and returns the difference of the their sums.
sumNonEmpty : which takes a list of integers and returns an Option[Int], which is None if the list is empty, and a Some containing the sum of the members otherwise.

Suppose we want to generalize these functions to work on lists of some other type, such as Doubles, pairs of integers i.e. (Int, Int) with (x1, x2) + (y1, y2) defined as (x1 + y1, x2 + y2), or pairs of Doubles. Take a moment and think about how you would do it. All these types seem to naturally support the operations addition and subtraction, but this fact is not encoded in their inheritance hierarchy, which makes it extremely difficult to write generic functions that work for all of them.

To solve this problem, let’s define a trait called Group :

Group is a generic trait, which takes a type parameter T and declares the operations plus, minus and inverse for objects of type T, as well as an element zero. Note that minus is defined using plus and inverse because x - y = x + (-y)(how clever!).

Groups are a real thing, and the operation plus needs to be associative i.e. (x + y) + z should be equal to x + (y + z), otherwise we may get weird results.

Instances of the trait Group should ideally be defined as implicit members of the object Group, which lives is the same file as the trait. It is called the companion object of the trait. When the Scala compiler needs to supply an implicit parameter of type Group[T], it looks inside the Group companion object. So, we won’t need to pass around Group instances explicitly while writing and using functions that depend on group operations.

Let’s define an instance of the trait for integers (inside the Group object, of course):

The instance for Double looks almost identical, except for the types :

Let’s try something a little more interesting. Here’s a method that produces a an instance of Group for the pair (T1, T2), if it can find instances of Group for T1 and T2 :

This single method will automatically provide Group instances for (Int, Int), (Int, Double), (Double, Int) and (Double, Double) whenever we need them. Isn’t that neat?

Alright, now that we’ve done all the hard work, let’s use the trait to make our sum functions generic :

With these small modifications, you can now use these functions on lists of any type T for which the compiler can find an implicit object of type Group[T]:

That’s it! That’s the type class pattern. Group is called a type class (because it represents a class of types that support group operations), and the types Int, Double, (Int, Int) etc. are called members of the type class Group because the compiler can find instances of the type class for these types.

This post merely scratches the surface of the use cases and benefits of using type classes in Scala. Type classes are awesome and you should use them all over your code base. There are many great resources (much better than this one) for learning about type classes in Scala, like this, this and this. The only reason this post exists is so that I can point out the deficiencies in this naive implementation, and show you how to make it much better in my next post.