Setup Ensime with Sublime in OS X: Scala Development with Sublime

This post shows how to setup Ensime with Sublime Text. In particular, we use Sublime 3 to demonstrate this setup process. Therefore, to follow along, we have to make sure that we have Sublime 3 already installed.

This post shows how to setup Ensime with Sublime Text. In particular, we use Sublime 3 to demonstrate this setup process. Therefore, to follow along, we have to make sure that we have Sublime 3 already installed.

Among others, we are going to use the following tools:

Setting up Ensime-Sublime

To get started, first we have to clone Ensime-sublime, a Sublime plugin to interact with Ensime as follows.

$ cd  ~/Library/Application\ Support/Sublime\ Text\ 3/Packages
$ git clone https://github.com/ensime/ensime-sublime.git Ensime
$ git checkout ST3

Since we are using Sublime 3, we have to checkout ST3 branch of the github repository.

We refer to this directory as $ENSIME_SUBLIME:

$ export ENSIME_SUBLIME="/Users/adilakhter/Library/Application Support/Sublime Text 3/Packages/Ensime"

An alternative to this approach is to clone it in any directory and afterwards, create a symbolic link as follows:

$ ln -s ensime-sublime ~/Library/Application\ Support/Sublime\ Text\ 3/Packages/Ensime

Configuring ensime-sbt

Next, we setup ensime-sbt, an SBT plugin. It can be configured at the project-level (by adding the following reference in project\plugins.sbt) or at a global level as outlined next.

$ cd ~/.sbt/0.13/plugins
$ echo "addSbtPlugin("org.ensime" % "ensime-sbt" % "0.1.7")" >> plugins.sbt

Running Ensime Server

First, we have to make sure coreutils is already installed. If not, we use brew to install it:

$ brew install coreutils

Now, to run Ensime server, we execute the following commands at that root of a SBT project:

$ sbt gen-ensime

It generates all the relevant configuration (see the generated .ensime) to interact with the Ensime server from Sublime.

Consequently, we can start ensime-server as follows:

$ $ENSIME_SUBLIME/serverStart.sh .ensime
...
13:02:15.536 DEBUG None o.e.i.SearchService - committing index to disk...
13:02:15.536 DEBUG None o.e.i.SearchService - ...done committing index
13:02:15.539 DEBUG akka://ENSIME/user/project o.e.c.Project - indexed 0 and removed 0
13:02:16.489 DEBUG akka://ENSIME/user/project/analyzer o.e.c.Analyzer - received handled message FullTypeCheckCompleteEvent in state loading
13:02:16.490 DEBUG akka://ENSIME/user/project/analyzer o.e.c.Analyzer - Analyzer ready in 0.012 seconds.

Configuring Sublime

Last step of this setup is to configure Sublime editor for our project. To do so, first we have to modify the Ensime-related configuration that we can find by navigating at Sublime Text > Preferences > Package Settings > Ensime > Settings - User and update it as follows and restart Sublime:

{
    "connect_to_external_server": true,
    "external_server_port_file": "[project-root]/.ensime_cache/port"
}

Note that, in the future version of Ensime-sublime plugins, external_server_port_file probably will not be required (see discussion at this pull-request). However, at the time of writing, it was indeed required.

If we have followed so far, we are done with configuring Ensime with Sublime. Next, we start development with Sublime.

Beginning Development with Sublime

To start developing with Sublime, we open the root directory of the project for which we have started the server (see “Running Ensime Server” section) and invoke Ensime:Startup in Sublime’s Command Palette (i.e. cmd+shift+p). Afterward, we can also the project with Ensime:Build command, which leads to running relevant sbt command to compile the project.

With respect to features, among others, ensime-sublime supports syntax completions (as shown next), go to definition and etc.

syntax completion

These features are also accessible from the context menu, as illustrated below.

context menu

That’s it for today! If you have any remarks or suggestion, please post. I look forward to your feedback.

PS. Photo credits: Chris McFarland

Inverting a Binary Tree with Scala

The problem of Inverting a Binary Tree has got some hype after following tweet.

As a problem-solver, I was wondering how to approach this problem, as it seems to be a great application of structural recursion. In this post, let’s see how to solve it with functional programming and Scala.

Problem Definition:

The problem can be specified as follows.

Given a Binary Tree t1:

4
/ \
2 7
/ \ / \
1 3 6 9
view raw actual-tree.txt hosted with ❤ by GitHub

We have to implement a function to transform t1 to a Binary Tree t2:

4
/ \
7 2
/ \ / \
9 6 3 1
view raw invertedTree.txt hosted with ❤ by GitHub

Thus the function invertTree essentially has following signature.

invertTree: Tree => Tree

Solution

First, we define a Binary Tree ADT.

sealed trait Tree[+A]
case object EmptyTree extends Tree[Nothing]
case class Node[A](value: A , left: Tree[A], right: Tree[A]) extends Tree[A]
view raw BinaryTree.scala hosted with ❤ by GitHub

In order to conveniently encode tree instances, we add following methods in the companion object of Tree.

object Tree{
def empty[A]: Tree[A] = EmptyTree
def node[A](value: A, left: Tree[A] = empty, right: Tree[A] = empty): Tree[A]
= Node(value, left, right)
}
view raw BinaryTree.scala hosted with ❤ by GitHub

As a result, we can define an instance of a tree in Scala REPL as follows.

scala> val t1 = node(4,
node(2
, node(1)
, node(3)
) ,
node(7
, node(6)
, node(9)
)
)
t1: Tree[Int] =
Node(4,
Node(2,
Node(1,EmptyTree,EmptyTree),
Node(3,EmptyTree,EmptyTree)),
Node(7,
Node(6,EmptyTree,EmptyTree),
Node(9,EmptyTree,EmptyTree)))
view raw BinaryTree.scala hosted with ❤ by GitHub

Next, in order to facilitate structural recursion, we define fold function for the binary tree as follows:

def fold[A, B](t:Tree[A] , z:B)(f:(B,A,B) => B): B = t match {
case EmptyTree => z
case Node(x,l,r) => f ( fold( l , z )(f) , x , fold( r , z )(f) )
}
view raw fold.scala hosted with ❤ by GitHub

It allows to traverse the tree, perform transformations and accumulate the result. For instance, we can define a function to count the length of the tree in a generic manner–

def size[T] (tree: Tree[T]) =
fold(tree, 0: Int){(l,x,r) => l + r + 1}
scala> size(t1)
res11: Int = 7
view raw BinaryTree.scala hosted with ❤ by GitHub

Also, we can define a map function that applies a function f: A ⇒ B on the value of each Node. Note that the application of map is always structure-preserving, that is, it retains the existing shape as it was before application (unlike the aforementioned size function) and perform only local transformations.

import Tree._
def map[A,B](tree: Tree[A])(f: A => B): Tree[B] =
fold(tree, Tree.empty[B]){(l, x, r) => Node(f(x), l,r)}
scala> map (t1) ( x => x * 10)
res11: Tree[Int] =
Node(40,
Node(20,
Node(10,EmptyTree,EmptyTree),
Node(30,EmptyTree,EmptyTree)),
Node(70,
Node(60,EmptyTree,EmptyTree),
Node(90,EmptyTree,EmptyTree)))
view raw BinaryTree.scala hosted with ❤ by GitHub

As you have guessed, we can similarly define the invertTree function in a generic manner as follows:

def invertTree[A](tree: Tree[A]): Tree[A] =
fold (tree, Tree.empty[A]){(leftT, value, rightT) => Node(value, rightT, leftT)}
view raw BinaryTree.scala hosted with ❤ by GitHub

In essence, invertTree simply swaps left node with the right node, and thus derives the resultant tree with the generic fold.

scala> invertTree(t1)
res12: Tree[Int] =
Node(4,
Node(7,
Node(9,EmptyTree,EmptyTree),
Node(6,EmptyTree,EmptyTree)),
Node(2,
Node(3,EmptyTree,EmptyTree),
Node(1,EmptyTree,EmptyTree)))
/*
4
/ \
7 2
/ \ / \
9 6 3 1
*/
view raw BinaryTree.scala hosted with ❤ by GitHub

Neat..uh? By the way, this problem can be solved in several ways. This post particularly demonstrates the application of structural recursion in a generic manner (e.g., with fold), which is the essence of #fp, imho ;).

If you have any question/suggestion or a different idea to solve this problem, please free to post it as a comment; I highly appreciate that! Thanks for visiting!

References

  1. LeetCode OJ: Invert Binary Tree

Applying Type Lambdas with Scala

Type Lambda, a must-have when utilizing higher-kinded types with Scala programming language, is the topic of this post. In this post, we discuss the application of type lambda in solving specific problems.

Background

In this section, we enumerate several concepts that are imperative to our today’s discussion.

Type Members

Scala allows defining type members in a trait or class as follows.

trait HList{
  type Hd
  ....
}

We can define an abstract type member Hd and we can also give a concrete meaning to Hd from its enclosing context, as shown next.

class IntList extends HList {
  type Hd = Int
  ....
}

As noted in #1, the above definition of Hd can also be used as an alias for type Int. For instance, we can define a type alias for Map[Int, V] asIntMap`–

type IntMap[V] = Map[Int, V]

Note that, in the above code, V is a parametric type parameter and can vary based on the context of IntMap.

Type Projection

We can access the type members of a class or trait with the # operator (similar to . operator in case of value members), e.g., IntList#Hd. For instance, the following line asserts that IntList#Hd is indeed an Int type.

implicitly[Int =:= IntList#Hd]

Type members of a class or a trait can be reused into different context and the relevant type checks are performed statically —

val x: IntList#Hd = 10

Design Principle

From the design perspective, type members are particularly interesting when they are evolved along with their enclosing type to match the behavior accordingly. As noted in #2, it is regarded as family polymorphism or covariant specialization.

Application of Type Lambda

Type Lambda is often regarded as Type-Level Lambda. As its name suggests, it is similar to anonymous functions, but for types. In fact, type lambdas are used to define an anonymous/inner types for a given context. It is particularly useful when a type constructor has fewer type parameters compared to the parameterized type that we want to apply in that context.

Next, we explain the concept of type lambda with an example.

Problem Description

Consider the following excellent example of a Functor from #2:

trait Functor[A, +M[_]]{
  def map[B] (f: A => B): M[B]
}

Notice that M[_] type constructor only accepts one type parameter. So, it is quite straightforward if we want extend the Functor defined above with a Seq types.

case class SeqFunctor[A](seq: Seq[A])
  extends Functor[A, Seq]{

  override def map[B](f: (A) => B): Seq[B] =
    seq.map(f)
  }
}

And we can use SeqFunctor as follows:

> val lst = List(1,2,3)
> ListFunctor(lst).map(_ * 10)
// prints List(10, 20, 30)

However, in case of Map[K, V], it would be tricky to extend, since Map[K,V] has two type parameters while M[_] type constructor only accepts one.

Solution

To solve this, we apply type lambda, which handles the additional type parameter as shown below.

case class MapFunctor[K,V](mapKV: Map[K,V])
  extends Functor[V, ({type L[a] = Map[K,a]})#L]{

  override def map[V2](f: V => V2): Map[K, V2] =
    mapKV map{
      case (k,v) => (k, f(v))
    }
}

> MapFunctor(Map(1->1, 2->2, 3->3)).map(_ * 10)
// Result: Map(1 -> 10, 2 -> 20, 3 -> 30)

Here, we simply apply f on the values of mapKV. Thus, type variable are indeedV and V2.

Informally, we extend Functor for Map with a type lambda as follows–

Functor[A, +M[_]] ==>
Functor[V,
  (
    {type L[a] = Map[K, a]} // structural type definition
  )
  #L // type projection
]

Here {type L[a] = Map[K, a]} denotes a structural type. It essentially specifies an inner/annonymous type alias L[a], which is then matched against the (outer) #L. Type parameter K is partially applied and has been resolved from the context in this case. By applying type projection, #L, we get the type member out of the structural type, and thus define an alias for Map[K,_] in effect.

As noted in #3, an empty block in the type position (as in M[_]) essentially creates an anonymous structural type and the type projection allows us to get the type member from it.

Additional Tricks

Type lambda seems a bit intimidating; and handling multiple of them leads to somewhat difficult-to-comprehend code. To avoid that, Dan Rosen proposed a trick.

To demonstrate that, lets consider the previous example. Revisiting the type lambda from MapFunctor, we note that the only varying type is V and V2. We can define a Functor for Map by avoiding type lambda and use type member Map[K] as shown below.

case class ReadableMapFunctor[K,V](mapKV: Map[K,V]){
  def mapFunctor[V2] = {
    type `Map[K]`[V2] = Map[K, V2]
    new Functor[V, `Map[K]`] {
      override def map[V2](f: (V) => V2): `Map[K]`[V2] = mapKV map{
        case (k,v) => (k, f(v))
      }
    }
  }
}

> ReadableMapFunctor(Map(1->1, 2->2, 3->3)).mapFunctor.map(_*10)
//Map(1 -> 10, 2 -> 20, 3 -> 30)

This trick handles the extra type parameter of Map[K, V]. Also note the backticks; they permit the use of [] in the name of an identifier #4.

Following gist (#5) outlines code examples used in this post.

Outlook

Overall, it seems to be an excellent feature with specific use-case, yet quite verbose. Sometimes, it is quite difficult to head around it. But surely, it has its applications and is a nice-to-have tool for Scala developers.

Let me know if you have any question. Thanks for visiting this post and going through this rambling.

References

  1. Programming in Scala by Odersky et al.
  2. Programming Scala by Dean Wampler and Alex Payne.
  3. Stackoverflow answer by Kris Nuttycombe regaring the benefits of Type Lambda in Scala.
  4. A More Readable Type Lambda Trick by Dan Rosen
  5. Gist outlining the code

Building ODataURI Parser with Scala Parser Combinators

Objective

Open Data Protocol (ODATA) facilitates end-users to access the data-model via REST-based data services by utilizing Uniform Resource Identifiers (URIs). In this post, we present the result of our recent experiment to build an abstraction on ODATA URIs to generate AST (Abstract Syntax Tree).

Note that this experiment is in its initial stage; hence, the implementation does not support complete feature-set of ODATA URI specification outlined at odata.org. It states a set of recommendations to construct these URIs to effectively identify data and metadata exposed by ODATA services.

To give an example of ODATA URI, consider following URI:

http://odata.org/service.svc/Products?$filter=Price ge 10

It in essence refers to a service request to return all the Product entities that satisfies the following predicate: Price greater than or equal to 10.

Motivation

Primary motivation of building such abstraction is to promote separation-of-concern and consequently, to allow the underlying layers of ODATA service implementation to process query expression tree and yield the result-set in a more efficient manner.

Approach

To implement this parser, we use Parser Combinators, which is in essence a higher-order function that accepts a set of parsers as input and composes them, applies transformations and generates more complex parser. By employing theoretical foundations of function composition, it allows constructing complex parser in an incremental manner.

Scala facilitates such libraries in its standard distribution (see scala.util.parsing). In this implementation, we in particular, use JavaTokenParsers along with PackratParser.

class ODataUriParser extends JavaTokenParsers with PackratParsers {
//...
}

Implementation

ODATA URI contains three fundamental parts, namely Service Root URI, Resource Path and Query Options as below and as per the documentations at [1].

http://host:port/path/SampleService.svc/Categories(1)/Products?$top=2&$orderby=Name
\______________________________________/\____________________/ \__________________/
| | |
service root URL resource path query options
view raw odata_uri.txt hosted with ❤ by GitHub

If we consider the ODATA URI mentioned previously, following illustrates the stated three parts of ODATA request:

odata-uri-parts

Hence, we can construct this parser by building combinators for the three sub-parts in a bottom-up manner and then compose them to construct the complete parser as listed below.

lazy val oDataQuery: PackratParser[SourceNode]={
serviceURL ~ resourcePath ~ opt(queryOperationDef) ^^ {
case uri ~ path ~ None => ODataQuery(uri, path, QueryOperations(Seq.empty))
case uri ~ path ~ Some(exp) => ODataQuery(uri, path,QueryOperations(exp))
}
}
view raw main.scala hosted with ❤ by GitHub

To gets started with an example, lets consider following URI:

http://odata.io/odata.svc/Schema(231)/Customer?$top=2&$filter=concat(City, Country) eq 'Berlin, Germany'

and we are expecting an expression tree based on a pre-defined model as follows:

ODataQuery(
URL("http://odata.io/odata.svc"),
ResourcePath("Schema",Number("231"),ResourcePath("Customer",EmptyExp(),EmptyExp())),
QueryOperations(
List(Top(Number("2")),
Filter(
EqualToExp(
CallExp(
Property("concat")
, List(Property("City"), Property("Country"))
)
, StringLiteral("'Berlin, Germany'"))))))
view raw expressionTree hosted with ❤ by GitHub

Building a parser combinator for Service Root and Resource Path are considerably simpler compared to that of Query Options (the third part). Let’s build them first.

We are using this convention (see ODATA specification) that a ODATA service root should always be ended by .svc. The following snippet can parse for instance http://odata.io/odata.svc to URL("http://odata.io/odata.svc").

lazy val serviceURL: PackratParser[Expression] =
"""^.*.svc""".r ^^ {
case s => URL(s)
}
view raw serviceRoot.scala hosted with ❤ by GitHub

Next we are defining a resource path which can parse for instance Schema(231) to ResourcePath("Schema",Number("231"),ResourcePath("Customer",EmptyExp(),EmptyExp())) expressions. A compound resource path can be augmented with multiple resources.

lazy val resourcePath: PackratParser[Expression] =(
"/" ~> idn ~ ("(" ~> predicate <~ ")") ~ opt(resourcePath) ^^ {
case Property(name) ~ keyPredicate ~ None => ResourcePath(name, keyPredicate, EmptyExp())
case Property(name) ~ keyPredicate ~ Some(expr) => ResourcePath(name, keyPredicate, expr)
}
| "/" ~> idn~ opt(resourcePath) ^^ {
case Property(e)~None => ResourcePath(e, EmptyExp(), EmptyExp())
case Property(e)~Some(expr) => ResourcePath(e, EmptyExp(), expr)
}
)
view raw resourcePath.scala hosted with ❤ by GitHub

After that we have reached to the crux of the problem: to build a parser that can handle the query operators defined in the OData specification. To solve it, we apply bottom up approach in conjunction with top-down realization.

First we define a basic parser that can parse arithmetic expressions as follows.

lazy val expression: PackratParser[Expression] =
expression ~ ("add" ~> termExpression) ^^ {case l ~ r => PlusExp(l, r)} |
expression ~ ("sub" ~> termExpression) ^^ {case l ~ r => MinusExp(l, r)} |
termExpression
lazy val termExpression: PackratParser[Expression] =
termExpression ~ ("mul" ~> factor) ^^ {case a ~ b => MultiplyExp(a, b)} |
termExpression ~ ("div" ~> factor) ^^ {case a ~ b => DivideExp(a, b)} |
termExpression ~ ("mod" ~> factor) ^^ {case a ~ b => ModExp(a, b)} |
factor
lazy val factor: PackratParser[Expression] =
("not" ~> factor) ^^ NotExp |
factor ~ ("(" ~> expressionList <~ ")") ^^ {case id ~ param => CallExp(id, param)} |
number |
boolean |
string |
idn |
"(" ~> predicate <~ ")"
lazy val expressionList: PackratParser[Seq[Expression]] = repsep(predicate, ",")
lazy val propertyList: PackratParser[Seq[Property]] = repsep(idn, ",")
lazy val idn: PackratParser[Property] = ident ^^ Property
lazy val number: PackratParser[Number] = floatingPointNumber ^^ Number
lazy val string: PackratParser[StringLiteral] = ("\'" + """([^"\p{Cntrl}\\]|\\[\\/bfnrt]|\\u[a-fA-F0-9]{4})*""" + "\'").r ^^ StringLiteral | stringLiteral ^^ StringLiteral
lazy val boolean: PackratParser[Expression] = "true" ^^^ TrueExpr() | "false" ^^^ FalseExpr()
view raw ExpressionParser.scala hosted with ❤ by GitHub

Then we incrementally augment support for handling relational operators, and thus can handle logical and, or and similar operation.

lazy val predicate: PackratParser[Expression] =
predicate ~ ("and" ~> relExpression) ^^ {case l ~ r => AndExp(l, r)} |
predicate ~ ("or" ~> relExpression) ^^ {case l ~ r => OrExp(l, r)} |
relExpression
lazy val relExpression: PackratParser[Expression] =
relExpression ~ ("gt" ~> expression) ^^ {case l ~ r => GreaterThanExp(l, r)} |
relExpression ~ ("lt" ~> expression) ^^ {case l ~ r => LessThanExp(l, r)} |
relExpression ~ ("eq" ~> expression) ^^ {case l ~ r => EqualToExp(l, r)} |
relExpression ~ ("ne" ~> expression) ^^ {case l ~ r => NotEqualToExp(l, r)} |
relExpression ~ ("ge" ~> expression) ^^ {case l ~ r => GreaterOrEqualToExp(l, r)} |
relExpression ~ ("le" ~> expression) ^^ {case l ~ r => LessOrEqualToExp(l, r)} |
expression
view raw RelationalExpression.scala hosted with ❤ by GitHub

The above two code listings form the basis to provide support for the query operations such as $filter and $select. See below.

lazy val queryOperationDef: PackratParser[Seq[Expression]] =
"?" ~> repsep(filter | select | top | skip | orderBy, "&")
lazy val filter: PackratParser[Expression] =
"$filter" ~> "=" ~> predicate ^^ Filter
lazy val top: PackratParser[Expression] =
"$top" ~> "=" ~> number ^^ Top
lazy val skip: PackratParser[Expression] =
"$skip" ~> "=" ~> number ^^ Skip
lazy val orderBy: PackratParser[Expression] = (
"$orderby" ~> "="~> propertyList <~ "asc" ^^ OrderByAsc
| "$orderby" ~> "="~> propertyList <~ "desc" ^^ OrderByDesc
| "$orderby" ~> "="~> propertyList ^^ OrderByAsc
)
lazy val select: PackratParser[Expression] =
"$select"~>"="~> propertyList ^^ Select
view raw QueryOperatorDef.scala hosted with ❤ by GitHub

Thus, it allows to parse the URI to expression tree as shown below.

test("Parse /Customers?$top=2&$filter=concat(City, Country) eq 'Berlin, Germany'"){
val uri = "http://odata.io/odata.svc/Schema(231)/Customer?$top=2&$filter=concat(City, Country) eq 'Berlin, Germany'"
val actual = p.parseThis(mainParser,uri).get
println(uri + "=>" + actual)
val expectedAst=
ODataQuery(
URL("http://odata.io/odata.svc&quot;),
ResourcePath("Schema",Number("231"),ResourcePath("Customer",EmptyExp(),EmptyExp())),
QueryOperations(
List(Top(Number("2")),
Filter(
EqualToExp(
CallExp(
Property("concat")
, List(Property("City"), Property("Country"))
)
, StringLiteral("'Berlin, Germany'"))))))
assert(actual == expectedAst)
}
view raw ODataUriParser.scala hosted with ❤ by GitHub

Or, as follows:

test("Parse /Products?$select=Name"){
val uri = "http://services.odata.org/OData.svc/Products?$select=Name,Price&quot;
val actual = p.parseThis(mainParser,uri).get
val expectedAst =
ODataQuery(
URL("http://services.odata.org/OData.svc&quot;),
ResourcePath("Products",EmptyExp(),EmptyExp()),
QueryOperations(List(Select(List(Property("Name"), Property("Price"))))))
assert(actual == expectedAst)
}
view raw ODataUriParserTests_Select.scala hosted with ❤ by GitHub

Conclusion

The complete source of this project is available at github repository. Please feel free to browse and if there is any question, please post.

See More:

  1. OData URI Specification
  2. External DSLs made easy with Scala Parser Combinators
  3. DSLs in Action