Behaviour Driven Development with Elasticsearch

Elasticsearch has been riding on top of the hype for a while now, and I expect it to hit even harder with the release of 1.0 – We will continue to see a massive growth in various fields throughout the tech world, and even more use cases will be discovered and put to production in stunning speed.

While it’s all hot and fresh I want to urge every Developer to try to include proper craftmanship techniques in his daily work with Elasticsearch. We all strive to ensure great results without regression – In this post I want to talk about a behaviour driven approach when it comes to Elasticsearch, something we at codecentric have had tremendous success so far.

Let’s imagine you’re having an Elasticsearch cluster up and running, and you’re trying to improve your search results for a specific use case, maybe by using the fantastic function score query – would you feel save pushing that change into production? Are you sure all the queries your customers throw at you will be answered sufficiently? If your answer is “HELL NO!” then you know you have a problem.

It’s not unsolvable though: This can be approached by having a decent set of tests that will provide accurate safety against regressions and support agile development of new features: Acceptance Tests! We are able to execute our tasks fast by starting up a whole ES node with the NodeBuilder in the java API and with a JUnit Rule ( as described by Florian Hopf here ) :

So let’s write our first test for it! Let’s create an Index, index a document and retrieve it again – in only a couple of clean lines!

Run the test and we’ll see in the console log that the node boots up, actually handles the request and shuts down gracefully, awesome!

So we could be done right here and commence happy TDD – but let’s crank it up a notch and

1. add JBehave to our stack
2. create a custom mapping within our code that we want to test

Let’s imagine we are building the next Twitter application and after careful consideration we come up with the follwing story:

 Scenario:  Basic Tweet retrieval

Given A user Chris submitted a tweet I luv tweeting

When We list all tweets for the user Chris

Then A tweet with the text I luv tweeting will be found

To introduce JBehave I can really recommend the fantastic JUnitReportingRunner from my workmates, grab it from Maven Central and create a Story Class that wires our story with some sane defaults. For further explanation check out Andreas’ post here .

Here you can see we’re loading our previous story called “Tweet.story” and a test called “TweetRetrievalTest”. This test maps our story to actual executable code and takes care of the Elasticsearch node bootup:

As a side-note, see how easy it is to inject a custom mapping into the whole setup! Feel free to experiment with it:

• provision your Elasticsearch production nodes with a custom mapping from a .yml file
• make use of the API: boost values, give it a custom scoring, try out different filters or analyzers
• run the test and know that it’s going to work. Awesome!

Happy testing folks! You can grab the code for this small example on our company github account here .

Google