Ruby Meets Elasticsearch

We use Elasticsearch here at Belly to implement this thing we call “search.” It comes in handy for a few things, such as analyzing your logs via ELK (which stands for Elasticsearch, Logstash, & Kibana) or just trying to figure out which businesses in Chicago have the word “pizza” in the name. At its core, Elasticsearch is just Apache Lucene. But it has a lot of other desirable features, like highly-available node clustering, aggregate search, and a JSON Query DSL.

But interacting with Elasticsearch on an application level is easier said than done. While Search Lite is simple to use in development, constructing meaningful queries can get complicated real quick. Since Elasticsearch isn’t as common as MySQL or Postgres, the odds are that other people on your team won’t know the internals of Elasticsearch. And that makes writing readable code really difficult.

omg funny no

Luckily, if you’re developing a Ruby app, there’s Elasticsearch Rails to the rescue!

Not only does it use the official Elasticsearch Ruby client (don’t write your own), but it also provides tools that solve some common problems. Plus, it has a bunch of syntactic sugar that feels familiar to Rails development. If you want a direct interface into your Elasticsearch cluster that feels similar to ActiveRecord, check out elasticsearch-persistance. But often times, you’ll have a database with a bunch of stuff that you need to search, so you need a way to get it all into Elasticsearch in a manageable way. That’s where elasticsearch-model comes in.

Let’s look at some sample code:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
# if you don't have autoloading, i.e. not-a-Rails-or-Napa-app, do:
# require 'elasticsearch/model'

class Bizniz < ActiveRecord::Base
  include Elasticsearch::Model
end

# index your Bizniz records from your database into Elasticsearch
Bizniz.import

# search it
response_from_elasticsearch = Bizniz.search 'pizza'

# show me the results!
response_from_elasticsearch.records.to_a

# => [#<Bizniz _id: 1, id: 1, name: "Combination Pizza Hut and Taco Bell">]

If your reaction is,

Not bad!

You’re right! Because the alternative is all like:

1
2
3
4
5
6
7
http = Net::HTTP.new('http://es-hostname/')
create_index = Net::HTTP::Put.new('/bizniz')
response = http.request(create_index)

if response.status == '200'
  document = Bizniz.take.to_json
  create_document = Net::HTTP::Put(' # BYE 30 YEARS OF PROGRAMMING FEHUWAIFLHEWAULIFHEWAUILFHAWEUILFHEWAUILF

If you want more tutorials on getting started and features, check out the README for elasticsearch-model. It’s awesome and really, really helpful. Some pieces you should look at are using ActiveRecord Concerns and indexing asynchronously during model callbacks.

Here are some tips & best practices that we’ve found helpful here at Belly.

Bop it, Twist it, Spec it

Test all the things! But make it easy for yourself by tagging your rspec examples. This way you’re not creating/refreshing/deleting indices between each spec, and you get to have a “clean state” like you would when you use DatabaseCleaner.

You can use this drag-and-drop code for your spec_helper.rb, or something equivalent:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
config.before :each, elasticsearch: true do
  ActiveRecord::Base.descendants.each do |model|
    if model.respond_to?(:__elasticsearch__)
      model.__elasticsearch__.create_index! force: true
      model.__elasticsearch__.refresh_index!
    end
  end
end

config.after :each, elasticsearch: true do
   ActiveRecord::Base.descendants.each do |model|
    if model.respond_to?(:__elasticsearch__)
      model.__elasticsearch__.delete_index!
    end
  end
end

And that will let you do stuff like this in your specs:

1
2
3
4
5
6
7
8
9
# in your bizniz_spec.rb

context 'nah, not using search here' do
  # too cool for ES
end

context 'search for love', elasticsearch: true do
  # huge Elasticsearch party over here
end

Use JSON Builders. Please.

Please

Jbuilder is a DSL that helps you create JSON structures in far more readable fashion. It also helps a ton with conditional logic, which you find yourself doing a lot when you’re writing queries with the Elasticsearch DSL, like so:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
query = Jbuilder.encode do |json|
  json.query do
    json.match do
      json.message do
        json.query: 'mad men game of thrones'
        if use_and_operator?
          json.operator 'and'
        else
          json.operator 'or'
        end
      end
    end
  end
end

Bizniz.search(query)

Though, it gets tricky to have a nested array of Jbuilder objects, which is needed for certain Elasticsearch queries. A simple way to work around this is to use the #attributes! method:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
def jbuild(*args, &block)
  Jbuilder.new(*args, &block).attributes!
end

# then you can do this

def donut_filter
  jbuild do |json|
    json.give_me_only_donuts
  end
end

query = Jbuilder.encode do |json|
  json.buncha_filters [
    donut_filter
  ]
end

Bizniz.search(query)

Search on!

Hope this helped you figure out how to use Elasticsearch in your Ruby app. If you have any questions (or tips), please let me know on The Twitter or The Email. Thanks for reading!

Ask a question or share this article, we’d love to hear from you!