a t e v a n s . c o m

(╯°□°)╯︵ <ǝlqɐʇ/>

I love Mike Gunderloy's "Double Shot" posts on his blog A Fresh Cup. Inspired by that, here's a link roundup of some stuff I've read lately, mainly from my Pinboard.

How to monitor Redis performance metrics - Guess what I've been up to for the last week or two?

redis-rdb-tools - Python tool to parse Redis dump.rdb files, analyze memory, and export data to JSON. For some advanced analysis, load the exported JSON into Postgres and run some queries.

redis-memory-analyzer - Redis memory profiler to find the RAM bottlenecks through scaning key space in real time and aggregate RAM usage statistic by patterns.

LastPass password manager suffers ‘major’ security problem - They've had quite a few of these, recently.

0.30000000000000004.com - Cheat sheet for floating point stuff. Also, a hilarious domain name.

Subgraph OS - An entire OS built on TOR, advanced firewalls, and containerizing everything. Meant to be secure.

When I get paged, my first step is to calmly(-ish) asses the situation. What is the problem? Our app metrics have, in many cases, disappeared. Identify and confirm it: yep, bunch of dashboards are gone.

Usually I start debugging at this point. What are the possible reasons for that? Did someone deploy a change? Maybe an update to the metrics libraries? Nope, too early: today's deploy logs are empty. Did app servers get scaled up, which might cause rate-limiting? Nah, all looks normal. Did our credentials get changed? Doesn't look like it, none of our tokens have been revoked.

All of that would have been a waste of time. Our stats aggregation & dashboard service, Librato, was affected by a wide-scale DNS outage. Somebody DDoS'd Dyn, one of the largest DNS providers in the US. Librato had all kinds of problems, because their DNS servers were unavailable.

We figured that out almost immediately, without having to look for any potential problems with our system. It's easy for me to forget to check status pages before diving into an incident, but I've found a way to make it easier. I made a channel in our Slack called #statuspages . Slack has a nifty slash command for subscribing to RSS feeds within a channel. Just type

/feed subscribe http://status.whatever.com/feed-url.rss

and boom! Any incident updates will appear as public posts in the channel.

Lots of services we rely on use StatusPage.io, and they provide RSS and Atom feeds for incidents and updates. The status pages for Heroku and AWS also offer RSS feeds - one for each service and region in AWS' case. I subscribed to everything that might affect site and app functionality, as well as development & business operations - Github, npm, rubygems, Atlassian (Jira / Confluence / etc), Customer.io etc.

Every time one of these services reports an issue, it appears almost immediately in the channel. When something's up with our app, a quick check in #statuspages can abort the whole debugging process. It can also be an early warning system: when a hosted service says they're experiencing "delayed connections" or "intermittent issues," you can be on guard in case that service goes down entirely.

Unfortunately not all status pages have an RSS feed. Salesforce doesn't provide one. Any status page powered by Pingdom doesn't either: it's not a feature they provide. I can't add Optimize.ly because they use Pingdom. C'mon y'all - get on it!

I've "pinned" links to these dashboards in #statuspages so they're at least easy to find. Theoretically, I could use a service like IFTTT to get notified whenever the page changes - I haven't tried, but I'm betting that would be too noisy to be worth it. Some quick glue code in our chat bot to scrape the page would work, but then the code has to be maintained, and who has time?

We currently have 45 feeds in #statuspages . It's kind of a disaster today with all the DNS issues, but it certainly keeps us up-to-date. Thankfully Slack isn't down for us - that's a whole different dumpster fire. But I could certainly use an RSS service as an alternative, such as my favorite Feedbin. That's the great things about RSS: the old-school style of blogging really represented the open, decentralized web.

I'm not the first person to think of this, I'm sure, but hopefully it will help & inspire some of you fine folks out there.

[Updated May 24, 2016: now with more salt]

There's been some chatter about how Ruby on Rails is dying / doomed / awful.

Make Ruby Great Again

...can a programming language continue to thrive even after its tools and core libraries are mostly finished? What can the community do to foster continued growth in such an environment? Whose job will it be?

Rails is Yesterday's Software

We need new thinking, not just repackaging of the same old failures. I should be spending time writing code, not debugging a haystack of mutable, dependency-wired mess.

My Time with Rails is Up

As a result of 9 freaking years of working with Rails and contributing like hell to many ruby OSS projects, I’ve given up. I don’t believe anything good can happen with Rails. This is my personal point of view, but many people share the same feelings.

Ruby in decline...

The significance of the drop in May and the leveling of is that "Ruby does not become the 'next big programming language'".

Whoops! That last one is from 2007. To summarize the various comments I've seen on Reddit, Hacker News, blogs, articles, and talks lately, here are some examples stuffed with the finest straw:

Rails Doesn't Scale

This canard hasn't aged well. My uninformed college-student argument in 2006 was: "You don't scale your language or framework, you scale your application." I had never scaled anything back then, but over ten years later I still agree with young-dumb-and-loud college me.

Your language or framework is rarely the bottleneck: your database, network structure, or background services are where things will slow down. ActionCable may or may not get two million plus connections per box like a trivial Elixir app, but you can always deploy more app servers and workers. And I'd worry those Elixir connections will end up waiting on Redis or Postgres or something else - your language or framework isn't a magic bullet.

You can also extract services when Ruby and Rails aren't the right tools for the job. Many Ruby gems do this by binding to native C extensions. Extracting a service for machine learning models can take advantage of Python's different memory model and better ML libraries. Your CRUD app or mobile API doesn't have to do it all.

Ruby is a Messy Language

In Ruby you can re-open any class from anywhere. You can access any object's private methods. Duck typing is everywhere. People make weird DSLs like RSpec. Rails is chock-full of magic in routing, database table names, callbacks, and mysterious inheritence. Ruby and Rails are easy, but they're not clear or simple.

This blast from the past is a 2007 argument. Since then we've learned tons about how to write expressive-but-clear Ruby. Rails isn't going to keep you safe by default, but you can and should try to write clear Ruby on top of it. I've found the following super helpful when refactoring and designing code:

Solnic in particular felt like his attempts to reduce complexity, tight coupling, and shenanigans in Rails were actively mocked by DHH and others in the community. If true, that's awful, and I would hope most of our community isn't the ActiveMocking type. DHH has certainly said Rails won't trade convienence for cleanliness by default. I don't think that precludes cleanliness and safety when the code and team get big enough to need it, though.

A programming language isn't a magic bullet here, either. I've seen people hold up Java as an example of explicit, clear code with sanity checks and few surprises. Well, here's some Java code for an Elasticsearch plugin, with comments stripped, spacing cut down, and truncated for brevity:

public class LookupScript extends AbstractSearchScript {
    public static class Factory extends AbstractComponent implements NativeScriptFactory {
        private final Node node;
        private final Cache<Tuple<String, String>, Map<String, Object>> cache;

        public Factory(Node node, Settings settings) {
            this.node = node;

            ByteSizeValue size = settings.getAsBytesSize("examples.nativescript.lookup.size", null);
            TimeValue expire = settings.getAsTime("expire", null);
            CacheBuilder<Tuple<String, String>, Map<String, Object>> cacheBuilder = CacheBuilder.builder();

Now here's a "magical" plugin for ActiveRecord, Paperclip. Truncated for brevity, but nothing stripped out:

module Paperclip
  require 'rails'

  class Railtie < Rails::Railtie
    initializer 'paperclip.insert_into_active_record' do |app|
      ActiveSupport.on_load :active_record do

      if app.config.respond_to?(:paperclip_defaults)

    rake_tasks { load "tasks/paperclip.rake" }

The latter seems vastly more readable to me. You may need to read the docs on what some of the methods do, but I had to read an awful lot more to grok the Elasticsearch plugin. Since we're in the era of musty old arguments, I'll bring up the amazing AbstractSingletonProxyFactoryBean again.

Ruby gives you enough rope to hang yourself. Javascript helpfully ties the noose and sets up a gallows for you. Java locks you in a padded room for your own good. Elixir uses a wireless shock colla... wait, sorry. That metaphor got weird when a functional language jumped in. Point is, you can write terrible code in any language, and some languages make that a little easier or harder depending on your definition of "terrible." Ruby strikes a balance that is my favorite so far.

Rails Jobs are Drying Up

Here's some data I found by asking colleagues, Googling and poking around:

Rails isn't achieving the thunderous growth percentage that Javascript and Elixir are, but percentage growth is much harder to achieve when your base is already huge.

On further discussion and analysis, this is actually insanely hard to measure. Javascript is on the rise, for sure - but how many Javascript StackOverflow questions are for front ends that exist inside a Rails, Django, or Phoenix app? How many Node.js apps are tiny background services giving data to a Java dumpster fire? Where do you file a full-stack job?

Languages and frameworks don't just disappear. You can still find a decent job in COBOL. Ruby and Rails will be with us for a long time to come.

The Cool Kids™ Have Moved On

These are people I look up to, and significantly contributed to my understanding & development in Ruby.

Of course, many of the "movers-on" are still part of the Ruby + Rails communities. Sidekiq for Ruby isn't going anywhere - Sidekiq Enterprise is pretty recent, and Mike has a commendable goal of making his business sustainable. Yehuda has been at RailsConf the last two years, and works at Skylight - a super cool performance monitoring tool with great Rails defaults. Searls has also been at RailsConf the last two years, and as he said to me on Twitter:

...my favorite thing about programming is that we can use multiple languages and share in more than one community 😀

~ @searls

Sanid Metz and Aaron Patterson have been fixtures in the community, and they don't seem to be abandoning it. Nick Quaranto is still at RailsConf. And of course, Matz is still on board.

Plus, a mature community always has new Cool Kids™ and Thought Leaderers™. I've found lots of new people to follow over the last few years. I can't be sure if they're "new" or "up and coming" or "how the heck am I just finding out now." Point is: the number of smart Ruby & Rails people I follow is going up, not down.

The above-mentioned Katrina Owen, Sarah Allen, Godfrey Chan, Richard Schneeman and others are all people I wish I'd known about earlier in my career.

I'm Still Pro-Skub

Ruby and Rails make a fantastic environment for application development. Rails is great at rapid prototyping. It's great at web apps. It's great at APIs - even better now that Rails 5 has an API-only interface. The Ruby community encourages learning, thoughtfulness, and trying to get things right. Resources like books, tutorials, screencasts, and boot camps abound. Extensions to Rails to help keep code clean, DRY, and organized are out there. Integrations with SaaS products are plentiful.

If you're looking to learn a new language or just getting started in web dev, I'd still recommend Ruby and Rails before anything else. They're easy enough to get started in, and the more you read and work with them, the better you'll get at OOP and interface design.

I've been working a lot with Hubot, which our company is using to manage our chat bot. We subscribe to the ChatOps mantra, which has a lot of value: operational changes are public, searchable, backed up, and repeatable. We also use Hubot for workflows and glue code - shortcuts for code review in Github, delivering stories in Pivotal Tracker when they are deployed to a demo environment, various alerts in PagerDuty, etc.

Hubot is written in CoffeeScript, a transpiles-to-javascript language that is still the default in Rails 5. CoffeeScript initially made it easy and obvious how to write classes, inheritence, and bound functions in your Javascript. Now that ES6 has stolen most of the good stuff from CoffeeScript, I think it's lost most of its value. But migrating legacy code to a new language is low ROI, and a giant pain even with tools like Decaffeinate. Besides, most of the hubot plugins and ecosystem are in CoffeeScript, so there's probably some advantage to maintaining compatibility there.

Hubot has a relatively simple abstraction over responding to messages in Slack, and has an Express server built-in. It's basically writing a Node application.

Writing Clean Code

A chatbot is not external, and is often not super-critical functionality. It's easy to just throw in some hacks, write very minimal tests (if any), and call it a day. At Hired we have tests for our Hubot commands, but we've never emphasized high-quality code the way we have in our main application. I'm changing that. Any app worth making is worth making well.

I've been trying to figure out how to break hubot scripts into clean modules. OO design is a hard enough problem in Ruby, where people actually care about clean code. Patterns and conventions like MVC provide helpful guidelines. None of that in JS land: it's an even split whether a library will be functional, object-oriented, or function-objects. Everything's just a private variable - no need for uppercase letters, or even full words.

While Github's docs only talk about throwing things in /scripts, sometimes you want commands in different scripts to be able to use the same functionality. Can you totally separate these back-end libraries from the server / chat response scripts? How do you tease apart the control flow?

Promises (and they still feel all so wasted)

Promises are a critical piece of the JS puzzle. To quote Domenic Denicola:

The point of promises is to give us back functional composition and error bubbling in the async world.

~ You're Missing the Point of Promises

I started by upgrading our app from our old library to Bluebird. The coolest thing Bluebird does is .catch(ErrorType), which allows you to catch only for specific errors. Combine that with the common-errors library from Shutterstock, and you get a great way to exactly classify error states.

I'm still figuring out how to use promises as a clean abstraction. Treating them like delayed try/catch blocks seems to produce clean separations. The Bluebird docs have a section on anti-patterns that was a good start. In our code I found many places people had nested promises inside other promises, resulting in errors not reaching the original caller (or our test framework). I also saw throwing exceptions used as a form of flow control, and using the error message of the exception as a Slack reply value. Needless to say, that's not what exceptions are for.


NodeJS comes with eventing built in. The process object is an EventEmitter, meaning you can use it like a global message bus. Hubot also acts as a global event handler, so you can track things there as well. And in CoffeeScript you can class MyClass extends EventEmitter. If you've got a bunch of async tasks that other scripts might need to refer to, you can have them fire off an event that other objects can respond to.

For example, our deploy process has a few short steps early on that might interfere with each other if multiple deploys happen simultaneously. We can set our queueing object to listen for a "finished all blocking calls" event on deploys, and kick off the next one while the current deploy does the rest of its steps. We don't have to hook into the promise chain - a Deploy doesn't even have to know about the DeployQueue, which is great decoupling. It can just do its waterfall of async operations, and fire off events at each step.


Hubot comes with a Brain built-in for persistent storage. For most users, this will be based on Redis. You can treat it like a big object full of whatever data you want, and it will be there when Hubot gets restarted.

The catch is: Hubot's brain is a giant JS object, and the "persistence" is just dumping the whole thing to a JSON string and throwing it in one key in Redis. Good luck digging in from redis-cli or any interface beyond in-app code.

Someone (not me) added SQLite3 for some things that kind of had a relational-ish structure. If you are going to use SQL in your node app, for cryin' out loud use a bloody ORM. Sequelize seems to be a big player, but like any JS framework it could be dead tomorrow.

Frankly, MongoDB is a much bigger force in the NodeJS space, and seems perfect for a low-volume, low-criticality app like a chatbot. It's relational enough to get the job done and flexible enough with schema-less documents. You probably won't have to scale it and deal with the storage, clustering, and concurrency issues. With well-supported tools like Mongoose, it might be easier to organize and manage than the one-key-in-Redis brain.

We also have InfluxDB for tracking stats. I haven't dived deep into this, so I'm not sure how it compares to statsd or Elasticsearch aggregations. I'm not even sure if they cover the same use cases or not.


Whooboy. Testing. The testing world in JS leaves much to be desired. I'm spoiled on rspec and ruby test frameworks, which have things like mocks and stubs built in.

In JS, everything is "microframeworks," i.e. things that don't work well together. Here's a quick rundown of libraries we're using:

  • Mocha, the actual test runner.
  • Chai, an assertion library.
  • Chai-as-promised, for testing against promises.
  • Supertest-as-promsied, to test webhooks in your app by sending actual http requests to Who needs integration testing? Black-box, people!
  • Nock, for expectations around calling external APIs. Of course, it doesn't work with Mocha's promise interface.
  • Rewire, for messing with private variables and functions inside your scripts.
  • Sinon for stubbing out methods.
  • Hubot-test-helper, for setting up and tearing down a fake Hubot.

I mean, I don't know why you'd want assertions, mocks, stubs, dependency injection and a test runner all bundled together. It's much better to have femto-frameworks that you have to duct tape together yourself.

Suffice to say, there's a lot of code to glue it all together. I had to dive into the source code for every single one of these libraries to make them play nice -- neither the README nor the documentation sufficed in any instance. But in the end we get to test syntax that looks like this:

describe 'PING module', ->
  beforeEach ->
    mockBot('scripts/ping').then (robot) =>
      @bot  = robot

  describe 'bot ping', ->
    it 'sends "PONG" to the channel', ->
      @bot.receive('bot ping').then =>

The bot will shut itself down after each test, stubs and dependency injections will be reverted automatically, Nock expectations cleaned up, etc. Had to write my own Chai plugin for expect(bot).to.send() . It's more magical than I'd like, but it's usable without knowledge of the underlying system.

When tests are easier to write, hopefully people will write more of them.


Your company's chatbot is probably more important than you think. When things break, even the unimportant stuff like karma tracking, it can lead to dozens of distractions and minor frustrations across the team. Don't make it a second-class citizen. It's an app - write it like one.

While I may have preferred something like Lita, the Ruby chatbot, or just writing a raw Node / Elixir / COBOL app without the wrapping layer of Hubot, I'm making the best of it. Refactor, don't rewrite. You can write terrible code in any language, and JS can certainly be clean and manageable if you're willing to try.

I was running through tutorials for Elixir + Phoenix, and got to the part where forms start showing validation failures. Specifically this code, from Pragmatic Programmers' "Programming Phoenix" book:

<%= form_for @changeset, user_path(@conn, :create), fn f -> %> 
  <%= if f.errors != [] do %>
    <div class="alert alert-danger">
      <p>Oops, something went wrong! Please check the errors below:</p>
        <%= for {attr, message} <- f.errors do %>
          <li><%= humanize(attr) %> <%= message %></li>
        <% end %> 
  <!-- snip -->
<% end %>

I got this error: no function clause matching in Phoenix.HTML.Safe.Tuple.to_iodata/1

Couldn't find a bloody solution anywhere. Took a long time to find IO.inspect . The message thing turned out to be a tuple that looked made for sprintf - something like {"name can't be longer than %{count}", count: 1} , so I spent forever trying to figure out if Elixir has sprintf , looked like there might be something in :io.format() , then I had to learn about Erlang bindings, but that wasn't...

Ended up on #elixir-lang IRC channel, and the author of the book (Chris Mccord) pointed me to the Ecto "error helpers" in this upgrade guide. It's a breaking change in Phoenix 1.1 + Ecto 2.0. The book is (and I imagine many tutorials are) for Phoenix 1.0.x , and I had installed the latest at 1.1.0 .

Major thanks to Chris - I have literally never had a question actually get answered on IRC. It was a last-resort measure, and it really says something about the Elixir community that someone helped me figure this out.

Stretchy is an ActiveRecord-esque query builder for Elasticsearch. It's not stable yet (hence the <1.0 version number), and Elasticsearch has been moving so fast it's hard to keep up. The major change in 2.0 was eliminating the separation between queries and filters, a major source of complexity for the poor gem.

For now my machine needs Elasticsearch 1.7 for regular app development. To update the gem for 2.0 2.1, I'd need to have both versions of Elasticsearch installed. While I could potentially do that by making a bunch of changes to the config files set up by Homebrew, I thought it would be better to just run the specs on a virtual machine and solve the "upgrade problem" indefinitely.

Docker looked great because I wanted to avoid machine setup as much as possible. I've used Vagrant before, but it has its own configuration steps beyond just "here's the Dockerfile." I already have boot2docker docker-machine installed and running for using the CodeClimate Platform™ beta, and I didn't want to have multiple virtual machines running simultaneously, eating RAM and other resources. Here's the setup:

  • Docker lets you run virtual machines inside "containers," using a few different technologies similar to LXC
  • boot2docker docker-machine manages booting virtual machine instances which will run your Docker containers. I'm using it to keep one virtualbox machine around to run whatever containers I need at the moment
  • fig docker-compose lets you declare and link multiple containers in a docker-compose.yml file, so you don't need to manually run all the Docker commands
  • The official quickstart guide for rails gives a good run-down of the tools and setup involved.

It's a bit of tooling, but it really didn't take long to get started; maybe an hour or two. Once I had it up and running for the project, I just modified the docker-compose.yml on my new branch. I had to do a bit of fiddling to get Compose to update the elasticsearch image from 1.7 to 2.1:

# modify the docker-compose.yml to update the image version, then:
docker-compose stop elastic
docker-compose pull elastic
docker-compose rm -f -v elastic
docker-compose run web rpsec # boom! builds the machines and runs specs

Once there, the specs started exploding and I was in business. Let the updates begin! After that, just a matter of pestering our CI provider to update their available versions of Elasticsearch so the badge on the repo will look all nice and stuff.

I wanted to try out basscss. Ended up changing the fonts, color scheme, and fixing syntax highlighting all over the place. Now using highlightjs, which looks like the only well-supported syntax highlighter than can guess which language your snippet is in.

This blog's been around for 5 years now. It's mostly thanks to Jekyllrb -- whatever I want to change about the site, I can do it without having to migrate from one database or format to another. If I need to code something myself, I can do that with Ruby or Javascript or plain 'ole shell scripts.

atevans.com has run off at least 3 different servers. It's been on Github Pages, Heroku, Linode, and more. It will never be deleted because some blogging company with a great platform ran out of VC, or added some social dingus I didn't want.

The git repo has been hosted four different places. When I got pwned earlier this year, the git repo was on the infected server. I just deleted it since my local copy had everything. Benefits of decentralized version control.

I had blogs all over the place before this, but this one has stuck around. I think even if Jekyll dies off somehow, a parser & renderer for a bunch of flat files should be easy in any language. I wonder what this will look like in another 5 years?

atevans.com in basscss

Web widgets are something everyone needs for their site. They've been done a hundred ways over the years, but essentially it's some bundle of HTML, CSS, and JS. The problem is that there are so many ways of doing this badly. The worst is "semantic" classes and JS hooks. Semantic-ish markup is used in quick-start frameworks like Bootstrap and Materialize, and encouraged by some frontend devs as "best practices."

Simaantec Makrpu

Semantic markup: semantic to who? It's not like your end-users are gonna read this stuff. Google doesn't care, outside of the element types. And it's certainly not "semantic" for your fellow devs. Have a look at the example CodePen linked from this post.

This CodePen represents the html, css, and js for two sections of our Instagram-clone web site. We have a posts-index page and a post-page ; two separate pages on our site that both display a set of posts with an image and some controls using semantic patterns. Some notes on how it works:

  • Does the post class name do anything? It's semantic, and tells us this section is a post. But that was probably obvious because the html is in _post.html.erb or post.jsx or something, so it's not saying anything we didn't already know.
  • What does a post look like? What styles might it have applied? Can't tell from reading the html. Is it laid out using float: left or text-align or inline-block or flexbox ? I'd have to find the css and figure it out.
  • Once I do figure it out, I need to look even further to see what post featured changes. Is featured something that only applies to post, or can featured be applied to anything? If I make user featured will that have conflicts?
  • Why are post and featured at the same level of CSS specificity? Their rules will override each other based on an arcane system no one but a dedicated CSS engineer will understand.
  • The .post{img{}} pattern is a land mine for anyone else. What if I want to add an icon indicating file type for that post? It's going to get all the styles of the post image on my icon, and I won't know about these style conflicts until it looks weird in my browser. I'll have to "inspect element" in the browser or grep for it in the CSS, and figure out how to extract or override them. What if I want to add a "fav" button to each post? I have to fix / override .post{button{}} . Who left this giant tangled mess I have to clean up?
  • Does .post have any javascript attached to it? From reading the html, I have no idea. I have to go hunting for that class name in the JS. Ah, it does - the "hide" behavior on any <button> inside the post markup. Again, the new "fav" button has to work around this.
    • What happens on the featured post? For the big post on your individual post page, a "hide" button doesn't even make sense, so it's not there. Why is the JS listening for it?
    • If I add my "fav" button, where do I put that JS? We'll want that on both the featured and the regular post elements.
    • What if we want "hide" to do different things in each place? For example, it should get more images from the main feed on the posts-index page, but more images from the "related images" feed in the related-images section. Do we use different selector scoping? Data attributes? Copy + paste the JS with minor alterations? The more places we use this component, the more convoluted the logic here will get.

Two steps forward, three back

Okay, we can apply semantic class names to everything: hide-button , post-image , featured-post-image , etc. Bootstrap does it, so this must be a good idea, right? Well, no. We haven't really solved anything, just kicked all these questions down another level. We still have no idea where CSS rules and JS behaviors are attached, and how they're scoped is going to be even more of a tangled maze.

What we have here is spaghetti code. You have extremely tight coupling between what's in your template, css, and js, so reusing any one of those is impossible without the others. If you make a new page, you have to take all of that baggage with you.


In the rest of our code, we try to avoid tight coupling. We try to make modules in our systems small, with a single responsibility, and reusable. In Ruby we tend to favor composable systems. Why do we treat CSS and JS differently? CSS by its nature isn't very conducive to this, and JS object systems are currently all over the place (function objects? ES6 Class ? Factories?). Still, if we're trying to write moar gooderer code, we'll have to do something different.

I'm not the first one to get annoyed by all this. Here's Nicholas Gallagher from Twitter on how to handle these problems. Here's Ethan Muller from Sparkbox on patterns he's seen to get around them.

I've found a setup that I'm pretty happy with.

  1. Use visual class names as described by Ethan above.
    • Decouples CSS rules from markup content: an image-card is an image-card is an image-card
    • Makes grouping and extracting CSS rules easy - any element using the same CSS rules can use the same class names
  2. Name classes with BEM notation, and don't use inheritence.
    • No squabbles over selector specificity
    • Relationships between your classes are clear: image-card__caption is clearly a child of image-card just from reading the markup
    • Prevents class name clobbering: image-card highlighted could be clobbered or messed up when someone else wants a highlighted class, but image-card image-card--highlighted won't be
  3. Use .js-* classes as hooks for javascript, not semantic-ish class names.
    • Decouples JS from html structure - your drop-down keyboard navigator widget can work whether it's on a <ul> , an <ol>, or any arbitrary set of elements, as long as they have .js-dropdown-keyboarderer and .js-dropdown-keyboarderer-item
    • Decouples JS from styles. Your .js-fav-button can be tiny on one screen and huge on another without CSS conflicts or overrides
    • Clearly indicates that this element has behavior specified in your JS - as soon as you read the markup, you know you have to consider the behaviors as well
    • data-* attributes have all the same advantages, but they are longer to type and about 85% slower to find (at least, on my desktop Chrome)

This was brought on by using Fortitude on the job, which has most of these solutions baked-in. It had a bit of a learning curve, but within a month or two I noticed how many of the problems and questions listed above simply didn't come up. After using Bootstrap 3 for the previous year and running into every. single. one. multiple. times. I was ready for something new. I quickly fell in love.

The minute anyone decided to go against the conventions, developing on that part of the site got 10x harder. Reusing the partials and components with "semantic" markup was impossible - I had to split things up myself to move forward. Some components were even tied to specific pages! Clear as day: "do not re-use this, just copy+paste everything to your new page."

I'd much rather be shipping cool stuff than decoupling systems that should never have been tightly bound in the first place.

I made a bloggity post for Elastic, the company behind Elasticsearch, Logstash and Kibana.

Our list page got 35% faster. Then we took it further: Angular was making over a dozen web requests to get counts of candidates in various buckets - individuals who are skilled in iOS or Node development, individuals who want to work in Los Angeles, etc. We dropped that to a single request and then combined it with the results request. From 13+ HTTP round-trips per search, we got down to one.

Chef seemed like a big hack when I first started with it. Chef "cookbooks" have "recipes" and there's something called "kitchens" and "data bags" and servers are called "nodes." Recipes seem like the important things - they define what setup will happen on your servers.

Recipes are ruby scripts written at the Kernel level, not reasonably contained in classes or modules like one would expect. You can include other recipes that define "lightweight resource providers" - neatly acronymed to the unpronouncable LWRP. These define helper methods also at the top level, and make them available to your recipe. What's to keep one recipe's methods from clobbering another's? Nothing, as far as I can tell.

Recipes run with "attributes," which can be specified in a number of different ways:

  • on the node itself: scoped for a specific box
  • inside a "role:" for every server of an arbitrary type
  • inside an "environment:" for overriding attributes on dev/stg/prd
  • from the recipe's defaults

Last time I used Chef, attributes had to be defined in a JSON file - an unusual choice for Ruby, which usually goes with YAML. Now apparently there's a Ruby DSL, which uses Hashies, which also appear to run at the Kernel level. I couldn't get it to work in my setup. Chef munges these different levels together with something like inheritence - defaults get overridden in a seemingly sensible order. Unless you told them to override each other somewhere. Then whatever happens, happens.

"Data bags" are an arbitrary set of JSON objects. Or is the Ruby DSL supposed to work there, too? I dunno. Anyway, they store arbitrary data you can access from anywhere in any recipe, and who doesn't love global state? They seem necessary for things like usernames and keys, so I can forgive some globalization.

This seems like a good enough structure / convention, until you start relying on external recipes. Chef has apparently adopted Berkshelf, a kind of Bundler for chef. You can browse available cookbooks at "the supermarket:" are you tired of the metaphors yet?

The problem here is that recipe names are not unique or consistent! I was using an rbenv recipe. But then I cloned my Chef repo on a new machine, ran berks install, and ended up with a totally different cookbook! I mean, what the hell guys? You can't just pull the rug out like that. It's rude.

Sure, I could vendor said recipes and store them with my repo. Like an animal. But we don't do that with Bundler, because it seems like the absolute bloody least a package manager can do. Even Bower can handle that much, and basically all it does is clone repos from Github.

These cookbooks often operate in totally different ways. Many cookbooks include a recipe you can run with all the setup included; i's dotted and t's crossed. They install something like Postgres 9.3 from a package manager or source with a configuration specified in the munged-together attributes for your box. Others rely on stuff in data bags, and you have to specify a node name in the data bag attributes or something awful. Some cookbooks barely have any recipes and you have to write your own recipe using their LWRPs, even if attributes would be totally sensible.

Coming back to Chef a few months after doing my last server setup, it seems like they are trying to make progress: using a consistent Ruby DSL rather than JSON, making a package manager official, etc. But in the process it's become even more of a nightmarish hack. The best practices keep shifting, and the cookbook maintainers aren't keeping up. You can't use any tutorials or guides more than a few months old - they'll recommend outdated practices that will leave you more confused about the "right" way to do things. Examples include installing Berkshelf as a gem when it now requires the ChefDK, using Librarian-Chef despite adoption of Berkshelf, storing everything in data bags instead of attributes, etc, etc, etc.

Honestly, I'm just not feeling Chef any more. Alternatives like Ansible, Puppet, and even Fucking Shell Scripts are not exactly inspiring. Docker is not for system configuration, even though it kinda looks like it is. It's for isolating an app environment, and configuring a sub-system for that. Maybe otto is the way to go? But damn, their config syntax is weirder than anything else I've seen so far.

I'm feeling pretty lost, overall.