SuperPumpup (dot com)
General awesomeness may be found here.

Ruby on Rails

Articles on Ruby on Rails

14 December 2014

Internalizing Dependencies

So I saw an interesting conversation today on Twitter between @DHH and @thijs this morning. They were discussing the NewRelic IPO and the subject got around to running ancient Rails versions, and the fact that a lot of the migration projects from Rails 2.3 have been driven in large part because of the movement of the surrounding ecosystem (read, gem updates).

Continue reading

12 August 2014

Mode Analytics Is Helping Me Become A Better Developer

Like most other "software writers" who primarily write Rails code I've met with, I don't have any formal CS training - or any formal training at all, really. I just started learning how to build things, which was hard and horrible in PHP and spectacularly easy in Rails. Love it or hate it, it's SO EASY to build systems in Rails when you have no idea what you're doing. I'll always argue that, and think it's amazing.

That said, I never learned SQL well...

Continue reading

25 February 2014

Agh! People are starting to use my App! Now what?

This is a slightly beefed up version of the lightning talk I gave at Austin on Rails last night - Feb 25, 2014.

I had intended to give this talk on things I've learn about infrastructure as I've been able to help stabilize and grow at a my last two jobs. I've had a lot of experience working on deployment infrastructure, from Dreamhost, to Rackspace, to Heroku, AWS, and now am mananging our infrastructure on Amazon OpsWorks using Chef (which I've been writing about recently).

However, as I thought about it, the problems that have been hindering our growth have not been as much server infrastructure related, as much as they were visibility-related...

Continue reading

13 February 2014

Deploying a Multi-Rails-App OpsWorks Stack

I've written before about how OpsWorks kind of pushes you into a weird architecture because of their default behaviors for applications in a stack.

Namely, the system tries to push all the applications on all of the layers, and chaos ensues.

But, it turns out I needed to finally move our various apps into one stack, and after a few days of poking, prodding, waaaaaaaaaaiting for machines, I got it working, so I thought I'd document it.

Continue reading

10 December 2013

Paperclip and The Legend of Zelda

Figuring out where a Paperclip attachment is a bit like playing the Legend of Zelda sometimes.

These are things you need to figure out, from three different sources:

1 - The filename of the actual file (face.png) - source your database

1a - The name of the model and model id that you have attached this to. This is different information, but generally in the DB

2 - The pathing strategy that Paperclip uses - this is typically in a global paperclip config file (maybe config/initializers/paperclip.rb)

3 - Where the eff in the cloud it is - this is the biggest pain in the butt there. In a sloppily-configured prod setup, you'll be lucky enough to have this in a file like s3.yml or maybe paperclip.yml. But with good configuration-management, that file will only live on the server it's supposed to because, you know, security.

So once you have all three of these pieces of information from different sources, and have rescued the princess, what can you build? A string. The information to construct a string that's like 100 characters long is scattered across the corners of your application's world and that's just nuts.

WORSE! If someone ever decides to touch corner 2 of the triforce there and change anything about how paths are calculated, your application just forgets where to find its attachments. All your links are broken, and you're off trying to figure out how to set up "compatibility modes" for some of the files. Because apparently that string changed? Wut? The location of this file is mutable all of a sudden? (facepalm)

This came up for me as I'm building a Persistence as a Service module for an application and I started putting in Carrierwave (since it's a touch saner than Paperclip) and then realized, "Self, wtf are you doing?" Shouldn't this just be holding a string? And maybe (maybe?!) aren't you using a modern database that will let you have a field hold an array of strings? And can't that array of strings hold all of the immutable data?

Yep. So that's what I've got now. If someone wants the Foo object to hold some attachments, they send something like this:

"foo": {
  "name": "The ultimate Foo",
  "attachments": [
      "type": "image",
      "image_content_type": "image/png",
      "small_url": "some_absolute_path_on_s3/picture_small.png",
      "medium_url": "some_absolute_path_on_s3/picture_medium.png",
      "large_url": "some_absolute_path_on_s3/picture_large.png",
      "original_url": "some_absolute_path_on_s3/picture_small.png"

So there we go. That's at that end, but why can't that level of treatment of the location of these files as immutable be pushed down into the actual image processing library? What if I decide to start saving new pictures in a different bucket? Can't my database just point two different things at two different buckets? What if I want to push some of the files immediately out to a CDN and only reference that location of theirs? Piece of cake! It's just strings! URI's are just strings. The only thing your application is ever going to care about in most cases is "what is the URI of this file." So keep those concerns separate.

14 October 2013

OpsWorks and System Architecture

TL;DR Moving to OpsWorks has been very helpful for us to get our system infrastructure represented and reproducible as code. I don't think it's a great tool for us to stay on as things grow in complexity, or we just want better ease-of-use. It's a great stepping stone.

At OwnLocal, I've been involved in porting our application infrastructure from EC2 instances controlled by Capistrano to Amazon's new product, OpsWorks.

I was impressed by the Capistrano scripts, though some things caused us a lot of pain:

  • We did not always get consistent deploys
  • We had to hard-code in IP addresses in our deploy files
  • Server configuration was unreliable (some of our servers had customized configurations that were problematic to rebuild)

Understanding what I do about Opsworks has taken a lot of effort, and I'm certainly no expert (yet?) but we now have servers that spin up on weekdays as our system comes under load and down as the load abates without any intervention.

The first time I saw my load-based servers spin up was amazing

However, I have felt some tension between what I consider good system design (an ecosystem of small applications) and the way that OpsWorks "Stacks" work. Oddly, when you add an App to this server that is deployed using their friendly GUI, it adds the app to all the servers, and then a deploy command is triggered on ALL instances by default. Which is odd. There's no obvious way to deploy several different Rails app layers within the same stack (so communicating between service applications require hard-coding addresses).

This design makes it seem easier to leave all your logic in One Big App

I also have been frustrated that though I have to write my own cookbooks to run other than the most trivial vanilla "Stack", I do not have access to the most powerful Chef concepts like "Search". Plus, debugging is an absolute nightmare. Amazon does a lot of "crafting" of the settings used on lifecycle events that are not easily (possibly?) replicated in something like a Vagrant environment. Therefore the feedback cycle on customizing scripts is ungodly slow at times (tens of minutes for some apsects of the life cycle).

Now I'm at a place where I have the set of cookbooks written to be able to deploy my infrastructure as code (GO ME!) and OpsWorks helped me get there. However, I'm really looking for a justifiable way to jump ship and go to a hand-rolled cloud (maybe VPC) that runs Chef (probably paying Opscode) with more modern tools like knife and search. In fact, I'll probably achieve this the way all great change is done - slowly and deliberatly, one service at a time.

Goals for moving forward:

  • Search for nodes (or settings) in a sane way
  • Reconfigure multiple "stacks" more quickly (to install package the "OpsWorks way", I click a lot, which is "convenient", though it's a pain to do on two stacks - like Production and Staging)
  • Have better visualization tools
  • Have a better deployment tool ("cap -S staging deploy branch=test_something_dangerous")

So in all, OpsWorks is great. If you want an easy-ish way to move to infrastructure as code, I encourage you to check it out. However, it's not the be-all and end-all, and it won't save you from learning Chef. So if your application has some significant complexity already, it may not be right for you.

09 July 2013

Form Matters

So a little snippet about why form is important. This system I inherited definitely has a "God object". Like pretty much every other order-processing/e-commerce project, that's "Order". I've whittled it from about 900 LOC to ~600 so far, and every bit is a battle (but one worth doing!). A virtue of having smaller objects that do one (or at least, for today, n-1) things is that it's actually practical to test that one thing.

The "unit" tests on the order object take about 30 seconds to run (it's also about 600 lines) and expensive to maintain. Today I am finally extracting a chunk of logic out of it (since I'm going to be extending functionality of that bit of logic) and I found that of the about 30 LOC I'm extracting, there are, I'd say six different behaviors. Those are:

  1. Check one of the order's parents to decide if it needs external reporting
  2. Convert the order to a hash
  3. Replace one of that hash's attributes with a "processed" version
  4. Replace another of that hash's attributes with a "processed" version
  5. Initialize a reporter
  6. Send a report

Of those, the following behaviors have any testing at all: 1, and 2. The hash attribute manipulation is actually somewhat hairy logic, but I'm sure it's a huge pain in the butt to test while it's buried there in the order object (I'm certainly not going to try).

So now I've extracted the entire external data reporting code to:

def complete_order_delayed

Which can be tested as such:

should "send itself off to the SubtleDataReporter" do
  job = @order.complete_order_delayed

Simple, complete, and effective. Now I can start building a testing framework around the hairy code in isolation and extend it.

So why is form important? Because once you've sacrificed form, the costs of good discipline (testing, readability, etc) begin to outweigh the benefits.

03 March 2013

Leveling up in RSpec - DRY

Repetitive tests suck. Just like repetitive code sucks. Repetitive tests are a pain to maintain so are likely to get stale. I very clearly know that repetitive code is To Be Avoided (and tools like CodeClimate make repetitive code glaring to me), and so don't do it. However, I remember being SO PROUD of my first text editor extension (a Sublime Text 2 Snippet):

Continue reading

14 February 2013

The Wrong Way To Do STI

I'm almost through an epic legacy data migration (that's quite interesting and I think I'll write something up about when I'm done). I'm on the very last customer data model. I have found a good deal of clunkiness in the old application, but nothing too egregious.

Then I Find This Gem

def set_type
  self.type = == 33 ? "SomeSubtype" : "SomeOtherSubtype"

Oof. Talk about coupling your data to your application.

06 February 2013

Cucumber vs Capybara/Selenium

At the beginning of the project I'm currently on (Nov 2012), I had a disagreement with the (then) lead dev on the project regarding using Cucumber for acceptance testing. He was a big proponent of the "mythical customer who reads and writes specs for you", which, to be fair, has not been even close to happening on this project, while I argued that "Cucumber just puts a DSL between you and your RSpec code. You have to interpret it all yourself, and it's not worth the double-effort there."

Rank was pulled and I grudgingly set up the application using Cucumber for integration testing. However, the other dev left the company about 3 days later and before the proverbial chair was cold,

How's that for a cliffhanger? Read more!

Continue reading

06 February 2013

Interface Testing With RSpec

I figured out how to do a pretty cool "interface test" (like described in Chapter 9 of POODR) in RSpec.

In this app I have a polymorphic association from a PurchaseOrder to one of many PurchaseOrderProperties (I know, the names are kind of awful, but that's what they are for now). Simple to set up, but how do I test the interfaces in RSpec? In MiniTest Sandi Metz made it look SO EASY AND AWESOME to just include a module of tests.

Well, I searched around a bit, and this is what I came up with:

Continue reading

05 February 2013

Translation In An API Client Gem

I went to the austin.rb meeting last night and enjoyed the talk. The speaker, @benhamill, asked for feedback and the thing that I've been thinking about since but is too long to tweet is his concept of Translation vs Transliteration of an API.

Briefly: most things you get out of an API are JSON objects, and accessors on JSON objects don't "feel" very "rubyish". For example:

Continue reading

01 February 2013

Testing Rails 4 Strong Parameters

I'm currently implementing Strong Parameters into a Rails application I'm building (3.2.11), and ran into a snag - well, honestly I've been battling it a fair amount for different reasons, but this is a new one, and in the vein of the Destroy All Software: Debugging With Tests, I'm getting this behavior to emerge at the lowest level I can.

That means actually testing the PermittedParameters system that I set up (a la Railscast 371). Oops, I probably should have done it as I built it but since it wasn't 100% clear what was needed to do the testing, I just skipped it and made sure that my "integration" tests - the controllers and acceptance tests - were behaving properly. I hereby deduct from myself 10 TDD points.

Maybe I can redeem myself through this post, though...

Continue reading

30 January 2013

I Love Fast Tests

Since my post a couple days ago My First Sub-millisecond Test, I've been using this technique more and more. I can't emphasize enough how much more pleasant it is when test output looks like this:

Wow. TDD is not just a possibility when it you have this instant feedback, it almost feels like a necessity.

Again, it is SO easy to start using this. You don't have to go back and rewrite your tests, you don't have to spend weeks "building the project from scratch", you just start moving your new features (and consequently testing) outside of rails.

So awesome.

It's such an improvement that I decided to come in here and compose and publish this blog post while the "old" test suite is running for that application...

28 January 2013

My First Sub-millisecond Test

This weekend I had the opportunity to watch Corey Haines's excellent talk: Fast Rails Tests from the 2011 GoGaRuCo. Maybe I'm a bit late to the party, but better late than never, right? I decided that yes. I'll do it. I'll accept your challenge, Corey, and write that first fast test.

And of course, you're reading about it now because I'm going to document it.

I'm currently working on a project to manage operations of construction companies, and the application is nearing a deliverable state and currently our test output looks like this:472 green dots of glory It's not bad. Honestly, it's the best test suite I've built, and what most of what doesn't get covered by these "unit" tests gets covered by integration tests (we're using Rspec/Cabybara/Selenium to run them rather than cucumber).

But why not push it?

Continue reading

20 January 2013

Building Antifragile Software

Imagine an application to run a business that is one method, 10,000 lines long. Worst thing in the world, right? Just thinking about such a think makes me shudder because you can look at it and know it's bad. But let's play with the idea a little bit.

So let's say the function of this application is to generate an invoice for a purchase with lots of line items. Cool, you may think if you like thinking about how software solves problems, an obvious thing that we could separate (to make this system a little more robust) is have the invoice built in two "phases". Phase 1 is preparing individual components (header, lines, footer) and then Phase 2 is composing them into an invoice.

Now you have a dramatically more robust system. You can fiddle with one part (say, the footer) feeling pretty safe that you are not going to change the "guts" of the invoice.

No problem there, let's go a little further -

Continue reading