We use Elasticsearch here at Belly to implement this thing we call “search.” It comes in handy for a few things, such as analyzing your logs via ELK (which stands for Elasticsearch, Logstash, & Kibana) or just trying to figure out which businesses in Chicago have the word “pizza” in the name. At its core, Elasticsearch is just Apache Lucene. But it has a lot of other desirable features, like highly-available node clustering, aggregate search, and a JSON Query DSL.
But interacting with Elasticsearch on an application level is easier said than done. While Search Lite is simple to use in development, constructing meaningful queries can get complicated real quick. Since Elasticsearch isn’t as common as MySQL or Postgres, the odds are that other people on your team won’t know the internals of Elasticsearch. And that makes writing readable code really difficult.
Luckily, if you’re developing a Ruby app, there’s Elasticsearch Rails to the rescue!
Not only does it use the official Elasticsearch Ruby client (don’t write your own), but it also provides tools that solve some common problems. Plus, it has a bunch of syntactic sugar that feels familiar to Rails development. If you want a direct interface into your Elasticsearch cluster that feels similar to ActiveRecord, check out elasticsearch-persistance. But often times, you’ll have a database with a bunch of stuff that you need to search, so you need a way to get it all into Elasticsearch in a manageable way. That’s where elasticsearch-model comes in.
Let’s look at some sample code:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17
# if you don't have autoloading, i.e. not-a-Rails-or-Napa-app, do: # require 'elasticsearch/model' class Bizniz < ActiveRecord::Base include Elasticsearch::Model end # index your Bizniz records from your database into Elasticsearch Bizniz.import # search it response_from_elasticsearch = Bizniz.search 'pizza' # show me the results! response_from_elasticsearch.records.to_a # => [#<Bizniz _id: 1, id: 1, name: "Combination Pizza Hut and Taco Bell">]
If your reaction is,
You’re right! Because the alternative is all like:
1 2 3 4 5 6 7
http = Net::HTTP.new('http://es-hostname/') create_index = Net::HTTP::Put.new('/bizniz') response = http.request(create_index) if response.status == '200' document = Bizniz.take.to_json create_document = Net::HTTP::Put(' # BYE 30 YEARS OF PROGRAMMING FEHUWAIFLHEWAULIFHEWAUILFHAWEUILFHEWAUILF
If you want more tutorials on getting started and features, check out the README for elasticsearch-model. It’s awesome and really, really helpful. Some pieces you should look at are using ActiveRecord Concerns and indexing asynchronously during model callbacks.
Here are some tips & best practices that we’ve found helpful here at Belly.
Bop it, Twist it, Spec it
Test all the things! But make it easy for yourself by tagging your rspec examples. This way you’re not creating/refreshing/deleting indices between each spec, and you get to have a “clean state” like you would when you use DatabaseCleaner.
You can use this drag-and-drop code for your spec_helper.rb, or something equivalent:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16
config.before :each, elasticsearch: true do ActiveRecord::Base.descendants.each do |model| if model.respond_to?(:__elasticsearch__) model.__elasticsearch__.create_index! force: true model.__elasticsearch__.refresh_index! end end end config.after :each, elasticsearch: true do ActiveRecord::Base.descendants.each do |model| if model.respond_to?(:__elasticsearch__) model.__elasticsearch__.delete_index! end end end
And that will let you do stuff like this in your specs:
1 2 3 4 5 6 7 8 9
# in your bizniz_spec.rb context 'nah, not using search here' do # too cool for ES end context 'search for love', elasticsearch: true do # huge Elasticsearch party over here end
Use JSON Builders. Please.
Jbuilder is a DSL that helps you create JSON structures in far more readable fashion. It also helps a ton with conditional logic, which you find yourself doing a lot when you’re writing queries with the Elasticsearch DSL, like so:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16
query = Jbuilder.encode do |json| json.query do json.match do json.message do json.query: 'mad men game of thrones' if use_and_operator? json.operator 'and' else json.operator 'or' end end end end end Bizniz.search(query)
Though, it gets tricky to have a nested array of Jbuilder objects, which is needed for certain Elasticsearch queries. A simple way to work around this is to use the
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19
def jbuild(*args, &block) Jbuilder.new(*args, &block).attributes! end # then you can do this def donut_filter jbuild do |json| json.give_me_only_donuts end end query = Jbuilder.encode do |json| json.buncha_filters [ donut_filter ] end Bizniz.search(query)