Thinking Sphinx

Welcome to Thinking Sphinx version 3 – a complete rewrite from past versions, built for Rails 3.1 or newer only.

Installation

It’s a gem, so install it like you would any other gem. You will also need to specify the Mysql2 gem as well (this is not an inbuilt dependency because JRuby, when supported, will need something different):

gem 'mysql2',          '0.3.12b4'
gem 'thinking-sphinx', '3.0.0'

The mysql2 gem is required for connecting to Sphinx, so please include it even when you’re using PostgreSQL for your database.

As for Sphinx itself, you need to make sure it’s compiled with MySQL support, even if you’re using PostgreSQL as your database – which is normally the default, but Homebrew can be too smart sometimes. If that’s your compiling tool of choice, make sure you install with the --mysql flag:

brew install sphinx --mysql

Usage

Indexes are no longer defined in models – they now live in `app/indices` (which you will need to create yourself). Each index should get its own file, and look something like this:

# app/indices/article_index.rb
ThinkingSphinx::Index.define :article, :with => :active_record do
  indexes title, content
  indexes user.name, :as => :user
  indexes user.articles.title, :as => :related_titles

  has published
end

You’ll notice the first argument is the model name downcased and as a symbol, and we are specifying the processor – :active_record. Everything inside the block is just like previous versions of Thinking Sphinx. Same goes for most settings in config/thinking_sphinx.yml (formerly config/sphinx.yml) unless noted below.

When you’re defining indices for namespaced models, use a lowercase string with /’s for namespacing as the model reference:

# For a model named Blog::Article:
ThinkingSphinx::Index.define 'blog/article', :with => :active_record

Other changes:

class Person < ActiveRecord::Base
  include ThinkingSphinx::Scopes

  sphinx_scope(:date_order) { {:order => :created_at} }
  default_sphinx_scope :date_order
  # ...
end
ThinkingSphinx::Index.define :article, :with => :active_record, :delta => true do
  # ...
end
ThinkingSphinx::Deltas.suspend :article do
  article.update_attributes(:title => 'pancakes')
end
ThinkingSphinx.search 'foo',
  :excerpts => {:chunk_separator => ' -- '}
ThinkingSphinx.search 'pancakes',
  :skip_sti => true,
  :classes => [User, AdminUser, SupportUser]
ThinkingSphinx.search(
  :select => '@weight * 10 + document_boost as custom_weight',
  :order  => :custom_weight
)
Article.search :sql => {:include => :user}
Article.search :group_by => :user_id
Article.search(
  :group_by       => :user_id,
  :order_group_by => 'created_at DESC'
)
search = Article.search('pancakes', :select => '*, @weight')
search.masks << ThinkingSphinx::Masks::WeightEnumeratorMask

search.each_with_weight do |article, weight|
  # ...
end

You’ll also note here that I’m specifying the internal weight attribute. This is necessary for edge Sphinx post 2.0.5.

batch = ThinkingSphinx::BatchedSearch.new
batch.searches << Article.search('foo')
batch.searches << Article.search(:conditions => {:name => 'bar'})
batch.searches << Article.search_for_ids('baz')

# When you call batch#populate, the searches are all populated with a single
# Sphinx call.
batch.populate

batch.searches #=> [[foo results], [bar results], [baz results]]
class OccasionalDeltas < ThinkingSphinx::Deltas::DefaultDelta
  # Invoked via a before_save callback. The default behaviour is to set the
  # delta column to true.
  def toggle(instance)
    super unless instance.title_changed?
  end

  # Invoked via an after_commit callback. The default behaviour is to check
  # whether the delta column is set to true. If this method returns true, the
  # indexer is fired.
  def toggled?(instance)
    return false unless instance.title_changed?

    super
  end
end

# And in your index definition:
ThinkingSphinx::Index.define :article, :with => :active_record, :delta => OccasionalDeltas do
  # ...
end
indexes events.eventable.name

polymorphs events.eventable, :to => %w(Page Post User)

Search Middleware

This section needs information – go hunting in the source for the moment if you’re keen on adding a layer around querying/result population process.

Search results, Glazes and Panes

In versions of Thinking Sphinx prior to v3, each search result object had many methods inserted into it – for direct access to the weight, distance, sphinx attributes and excerpts. This is no longer the case, but there is a more modular approach available.

Search results may now have a glaze object placed around them, which can then delegate methods to any number of panes the glaze has available. By default, there are no panes added (and thus, no glazing), but this can be modified:

# For every search
ThinkingSphinx::Configuration::Defaults::PANES << ThinkingSphinx::Panes::WeightPane

# Or for specific searches:
search = ThinkingSphinx.search('pancakes')
search.context[:panes] << ThinkingSphinx::Panes::WeightPane

The available panes are as follows:

All panes namespaced to ThinkingSphinx::Panes, and the DistancePane is automatically added when you provide latitude/longitude values via the :geo option.

If you wish to add your own panes, go ahead. The only requirement is that the initializer must accept three arguments: the search context, the underlying search result object, and a hash of the raw values from Sphinx.

Deployment with Capistrano

Thinking Sphinx comes with several Capistrano tasks to help ease deployment of your applications. Just require the recipes:

# config/deploy.rb
require 'thinking_sphinx/capistrano'

When running cap deploy:setup to get your server ready for deployments, Thinking Sphinx will also set up a shared folder for your indexes. Before finalizing a deployment, these indexes will be symlinked into the release path for use. When you deploy your application for the first time using cap deploy:cold, your indexes will be built for you and the search daemon will be started. Run cap -T to see all of the deployment tasks.

Limitations

Almost all functionality from v2 releases are implemented, though it’s worth noting that some settings haven’t yet been brought across, and a handful of the smaller features don’t yet exist either. Some may actually not return… we’ll see.

May or may not be added:

Sphinx Versions

TS 3 is built for Sphinx 2.x only. You cannot use 1.10-beta, 0.9.9 or anything earlier than that. 2.0.5 or newer is recommended.

Rails Versions

Currently TS 3 is built to support Rails/ActiveRecord 3.1 or newer. If you’re using Sinatra and ActiveRecord instead of Rails, that’s fine – just make sure you add the :require => 'thinking_sphinx/sinatra' option when listing thinking-sphinx in your Gemfile.

TS 3 does not support Rails 3.0, Rails 2.x or earlier, or Merb – please refer to the TS 1.x and 2.x releases in those situations.

Ruby Versions

Built on MRI 1.9.3 and tested against MRI 1.9.2 as well. No plans to support MRI 1.8, but would like to support Rubinius and JRuby (the one catch with the latter is the different MySQL interfaces).

There’s also the complication that Sphinx 2.0.x releases don’t work with JDBC (as JDBC sends several MySQL-specific commands through when initializing a connection). So, JRuby support won’t appear until there’s a stable Sphinx release that can interact with JDBC.

Database Versions

MySQL 5.x and Postgres 8.4 or better are supported.

Contributing

You’re brave! To contribute, clone this repository and have a good look through the specs – you’ll notice the distinction between acceptance tests that actually use Sphinx and go through the full stack, and unit tests (everything else) which use liberal test doubles to ensure they’re only testing the behaviour of the class in question. I’ve found this leads to far better code design.

If you’re still interested in helping evolve this, then write the tests and then the code to get them passing, and send through a pull request. No promises on merging anything, but we shall see!

For some ideas behind my current approach, have a look through sketchpad.rb in the root of this project. If you can make sense of that, you’re doing very well indeed.

Licence

Copyright © 2007-2012, Thinking Sphinx is developed and maintained by Pat Allan, and is released under the open MIT Licence. Many thanks to all who have contributed patches.