Let's talk Open™

What's Hired Got?

Reactor - pub/sub message bus for Sidekiq

Stretchy - Flexible query builder for Elasticsearch

Fortitude - Rock-solid SCSS framework for teams

Puma Stats Logger - Rack middleware for lawgz

lledci - Turn lights red/green from your CI

...and 41 more!

I'll Focus Here

Reactor - pub/sub message bus for Sidekiq

Stretchy - Flexible query builder for Elasticsearch

should you Open™?

what are the challenges?

how did Open™ affect our team?

what did Open™ teach us about Ruby & Rails?

Q&A

Real Talk

bestgems ranks out of 130k

Similar Rankings

rave - build bots for Google Wave

pirate_bay_ruby - CLI for dead web site

activetrail - ActiveAdmin + Trailblazer = ???

hedbergism - Random Mitch Hedberg quote

Worth it?

let's see...

Reactor

Enhanced Sidekiq Interface

Sidekiq is cool! Thanks, Mike!

Problem:

def update
  candidate.save!
  WorkerOne.perform_async(candidate)
  WorkerTwo.perform_async(candidate)
  WorkerThree.perform_async(candidate)
  # ...
  WorkerTwentySeven.perform_async(candidate)
end

⌘+C, ⌘+V

Worse:

# WorkerOne
def perform(candidate)
  WorkerTwo.perform_async(candidate)
  WorkerThree.perform_async(candidate)
end

# WorkerTwo
def perform(candidate)
  WorkerOne.perform_async(candidate) # OOPS
end

Reactor

Reactor::Event.publish :my_event, some: :data

candidate.publish :profile_updated

Kicks off abstracted Sidekiq worker

Locates "subscriber" workers & enqueues their jobs

Each job gets all event data

def action_event(event_name, options = {})
  current_user.publish(event_name,
    {controller: params[:controller]}.merge(options))
end

Subscribers

class CandidateCacheBuster
  include Reactor::Subscribable

  on_event :profile_updated do |event|
    id = event.actor.id # User id
    REDIS.del "candidate-profile-cache-#{id}"
  end
end

class ActivityLogger
  include Reactor::Subscribable

  on_event '*' do |event|
    # log it to places
  end
end

Moar Options!

class AdminEmailer < ActionMailer::Base
  include Reactor::Subscribable

  on_event :profile_updated, :activity_email

  def activity_email(event)
    # ... send the admin an email
  end
end

class ReminderMailer < ActionMailer::Base
  include Reactor::Subscribable

  on_event :interview_request, delay: 2.days do |event|
    # send reminder that this happened
  end
end

Subscriptions in yer DB!

class AdminNotifier < Reactor::Subscriber
  # t.string   "event_name", limit: 255
  # t.string   "type",       limit: 255
  # t.json     "data"
  # t.datetime "created_at",             null: false
  # t.datetime "updated_at",             null: false

  on_fire do
    notify_admin! if should_notify?
  end

  def notify_admin! ; end
  def should_notify? ; end
end

Scheduled DB Subscribers

class Interview < ApplicationRecord
  # t.string "status", limit: 255
  # t.datetime "end_at", limit: 255

  publishes :interview_ended, at: :end_at,
    if: -> { status == 'scheduled' }

  publishes :interview_start_reminder, at: :interview_reminder_time,
    watch: :start_at, if: -> { status == 'scheduled' }
end

will auto-reschedule jobs if end_at changes

Did Open™ Help?

reactor code is out-of-sight, out-of-mind

extra friction discourages arbitrary changes

keeps pattern & convention clear

open visibility adds pressure to do it "right"

pinning versions and releasing thoughtfully

Different context forces thinking about the pattern

What's the minimum info Reactor needs, vs what our app will use?

requiring Rails is harder than it sounds!

Railties

Concerns

API design & gang-of-four

message bus: subscribe to "channels," data is arbitrary

different from event delegation

ActionCable - subscriber_map.rb

ActiveSupport::Notifications - subscriber.rb

didn't make us famous

helped our team establish a convention

quietly kept working

smaller, loosely-coupled code

learning cool pattern stuff

Stretchy

Composable query builder for Elasticsearch

this'll never take off without a clever name

this'll never take off without a clever name

Goals

ActiveRecord-ish syntax, immutable queries

just a query builder; use Reactor for indexing

flexible enough for complicated queries

no dependencies except Elasticsearch

composable scopes for common query functions

Query returning Lynn Conway

Candidate.search
  .match(university: 'Michigan')
  .where(
    recognized_enough: false,
    born_between: 100.years.ago..50.years.ago
  )
  .boost.match(study_field: 'microchips')
  .results

ActiveRecord-ish query syntax

Abstracts away this stuff

{"function_score" : {
  "functions" : [{
      "filter" : { "match" : { "study_field" : "microchips" } },
      "weight" : 1.5
  }],
  "query" : {
    "bool" : {
      "must" : [{"match" : { "university" : "Michigan"}}],
      "filter" : [
        { "terms" : { "recognized_enough" : [ false ] } },
        { "range" : { "born_between" : {
              "gte" : "1917-04-13T16:16:59.883-08:00",
              "lte" : "1967-04-13T16:16:59.884-08:00" }
        }}
]}}}}

Example Methods

api = Stretchy.query(index: 'candidates', type: 'candidate')

# range filter - remember, immutable! api = api.query_method...
api = api.where created_at: (2.days.ago)..(1.days.ago)

# match query
api = api.match name: 'Grace Hopper'

# multiple should filters (OR-like)
api = api.should.where language: 'ruby', awesome: true

# function_score functions
api = api.boost.near field: 'coords', origin: [lon, lat]

# raw query
api = api.query multi_match: {name: 'Adele Goldberg', fields: [...]}

LOTS to support

Match Phrase Prefix Query

Regexp Query

has_child query?!

More Like This?!?!?!

OMG... HALP!

Think about the pattern, not our needs right now

this 2010 post from John Nunemaker is still relevant

def add_nodes(additional)
  self.class.new(opts.merge(
    nodes: collector.nodes + Array(additional),
    root:  root,
    body:  body,
    context: {}
  ))
end

new self with added things (collect into array)

Visitor Pattern

1. build a stack
2. compile into tree
3. visitor traverses tree, reduces

Arel's collector classes

tree_manager

MySQL visitor - Postgres visitor - Oracle visitor

Visitor Pattern

1. build a stack
2. compile into tree
3. visitor traverses tree, reduces

Stretchy doesn't need multiple visitors

only one dialect

Some Challenges

confusing: other gems do much more

API too like ActiveRecord hides ES complexity, features

hard to tell what common query types to extract

updating the gem shouldn't block new features

Solutions

make it more flexible for internal customization

Solutions

make it more flexible for internal customization

...

several times

Solutions

make it more flexible for internal customization

...

several times

...

then some more

flexible architecture needs less code!

learned cool new pattern

excuse to dive deep on framework code

core query interface simple, business logic complex

keep ES concerns out of models

a little more friction can be good

thinking off the Rails can make code clearer

learn more about computers

be more thoughtful

Q&A

Thanks all!

Use Hired - or come work for us!

slides at atevans.com/rc2017 - it me: @agius