Merbcamp - notes from the edge

Oct 11, 2008

A recap on the presentations from the first day of the very first Merbcamp. If you want to read about day to, check this page Merbcamp - Day 2.

Ezra's Keynote

Ezra gave a great speech talking about merb, some of the design decisions that are directing it, but the two most interesting points for me were:

Minigems

A really useful drop-in replacement for rubygems. Instead of loading the full gem list into RAM (and into every mongrel instance of your app as a result) minigems loads them as required. The reported saving is around 10-20MB of RAM, depending on how many gems you have installed on your system. And using it is simple, instead of:

require 'rubygems'


you do:

begin 

  require 'minigems'

rescue LoadError 

  require 'rubygems'

end


Thanks to Fabien for all his hard work, go grab it from his github repo.

Nanite

A "self-assembling fabric of daemons" that Ezra has written. It's based on RabbitMQ, and essentially you create agents and mappers. Agents do the work, mappers keep track of who does what. And to keep running with what is written on the tin, agents are transient, persistent, and transactional.

Essentially the agents ping the mappers on a regular basis with information on who they are, what they can do, and their current/average load. If the mapper doesn't hear from the agent within the timeout period, it's removed from the pool of workers. When new work comes in and is passed to the mapper, it can decide who to pass the work to based on any arbitrary rules. It can go to the agent with the lowest load, it can be broadcast to all agents, a random one, or any rule you decide. This is awesome if you're wanting to push work off into the cloud. Just dial up a new agent, it informs the mapper of who it is and what it can do, and there is zero configuration requirement...; just keep dialing them up as required. Additionally, you can just turn them off and the mapper will handle it gracefully. Ezra is looking to add in support for the merb rack interface so the same can be done for the actual merb process, just dial up merb web servers as nanite agents and scale your site on demand at the click of a button.

Go check it out

The slides and a video of the presentation is now also available.

Merb plumbing: The router

This one ran way too quick for me to take any detailed technical notes on, but suffice to say that the routing in merb is absolutely amazing. Handling both domains and subdomains is a breeze, with code like the following:

match(:subdomain => /(.*)/).

to(:account => ":subdomain[1]") do

  #application routes here...

end



r.match(%r[^/movies/(\d+)-(\d+)-(\d+)$]).

to(:controller => "movies", :movie_id => "[1][2][3]", :action => "show")


As you can see, you can do pattern matching within the route definition, and then use the captures as parameters. You can also add in requirements on the parameters much like rails:

match("/artices(/:year)", :year => "[\d]*").


Or specify a route that has optional components:

match("/artices(/:year(/:month(/:day)))/:title").


It may be too subtle to notice here so I'll point it out; unlike rails the optional parts of the route don't have to be at the end. For this rule to be met, the requested path must start with /artcies and have a :title at the end, :year, :month, and :day are all optional.

Particularly impressive was Yehuda mentioning in the Q&A session how routes can initiate DB calls, and then pass the found objects directly into the controller. One call for matching the route, and then re-using the same object in the controller. Nifty!! Some more detailed usage examples have been put up in this pastie.

Merb Slices

They're kinda like engines in rails, without as much voodoo because of the way merb has opened up the necessary hooks to support them natively within core. Install commonly used pieces of functionality in your apps as a slice, and away you go

What if you want to customize?

You have a few options, and it depends on what it is you want to customize but your options essentially are:

  • monkey patch it (using overrides)
  • use the "patch" rake task
  • update settings within the config hash

On that last one, each merb slice has it's own configuration has available at

Merb::Slices::config[:blog_slice]


Over-riding behaviour

Much like engines, you can use slices to create custom controller actions, custom models, or custom views. What happens if you don't like the way the behaviour in the slice works? Well go change it. If you want to natively support an action or format that the slice didn't come with, just go create the template. Your own monkey-patches exist in the /slices directory within your app. Just add in the new models, views, controllers definitions, etc. They'll take precedence over any conflicting definitions within the slice.

Initializer

When a slice initializes, it gets quite a number of hooks available to determine what it should load and add in its own behaviour. The main load points are:

  • Declaring dependencies/boot loader
  • Defining routes
  • Event hooks (initializing engine, engine loaded, etc.)

Handling URLs

The URL helper method is handled by slice_url for your slices, so you use it like this:

slice_url(:controller => ...., :action => ...)

slice_url(:merb_auth_slice_password, :login)


The latter would give you the URL to the login route, as defined within the merb_auth slice

Being a good citizen

A few other points to keep in mind, you should namespace everything to prevent conflicts with the actual application that is including your slice. This includes models. You just namespace models like so:

class MySlice::Post

end


Also, package up any assets you depend on. The end-user can include them in their app with a rake task:

rake slices:myslice:copy_assets


And now the assets will be available in /myslice/images/...;

download the presentation here

Yehuda's Testing Philosophy

Basically the gist of this talk was that TDD (at least how most people do it) is useless as a means of regression testing. If you've have to go back and change tests because you've changed code then you no longer have any proof that your code still works. There is no existence of a test that tests your original intention. That being said, TDD is still good and Yehuda still uses it for development, but his approach is somewhat different.

Yehuda's approach to testing rests on 3 fundamental principles:

  • If you have broken application something should fail
  • If you have a working application all tests should pass
  • Test what you care about

The latter point needs some clarification to prevent the "Well, duh!" response. You don't care about which methods are called, you care about the end result. Do you really care that you've called find on your model? Or that the output you're expecting has been displayed as intended to the end user?

If you change views, helpers, etc. then usually tests fail because you've changed code. The tests are incorrectly claiming that things are broken, but they're not. The site still works. Refactor tests into such a way that they don't break so easily, and you still have proof that everything still works.

One point that came up in the Q&A: He thinks you should never mock your own code. If you've got your own code, just test directly against it. Mocks should be used for web services or a 3rd party API. 3rd party providers should provide a mock framework for testing against.

All very interesting, and I need to give it some more thought. I can see the underlying benefits from the number of times a minor change has taken a few hours to implement because of the impact on testing, not because of the complexity of the change. I do wonder what impact it has on tests also serving as documentation of your implementation though.

MerbAuth: Darwinian authentication

MerbAuto was originally the first plugin with multi-ORM support, and was a direct port of Restful Authentication from rails but it was awfully complex. Then slices came along, and merb-auth was born.

Authenticating the session

In the interest of reducing some of this complexity, merb-auth was re-written with a pretty major implementation change. Don't authenticate controller, we don't really care about it. Authenticate the session. It's the at the session level that we can determine who a user is, not on an individual controller. Thankfully, merb routing makes session available within the router. Not only does that mean that we can authenticate the session, but we can do it in the routing!!

Design Benefits

Merb routing sets response codes correctly from raises, so rather than a 302 redirect on failure you can raise an unauthorized response. Authentication is not user management, and your underlying user requirements could be vast so this is just an authentication framework and you're left to implement user admin/creation yourself. And it's just that, a framework and not a plugin. It allows you to build your own authentication for your application.

Implementation options

merb-auth includes a password slice which allows you to have both form base authentication or regular HTTPbasic authentication. It also offers you two different ways to protect your application; either in the router or within a controller. To protect at the router level you just do the following

protect do

  resources :admin

end


Or in a controller:

before :ensure_authenticated


Why protect within a route?

The main benefit is that it is earlier in the request/response cycle, and so processing can stop right in the dispatcher and send the response straight back to rack. Within a controller you'll need to create a controller instance, possibly run some filters, etc.

Authentication strategies

A stratergy contains logic for authentication. You need to implement a run method, and then within it any logic you feel pertinent to authenticating a user. You can also declare many different strategies for various login types, say one for teachers and one for students.

More details and actual code implementations are included in the presentation

Damage control: A primer on Merb

This is an intro to merb and how to get everything installed and get up and running quickly. The usual things on how merb isn't opinionated and is very modular. If you want to get straight into it though, you can get opinionated and get a setup with the following franmeworks:

  • helpers
  • datamapper
  • rspec
  • merb-auth
  • sqlite3
  • jquery

To get up and running quickly these days Yehuda has written a new gem called thor. It's basically a cross between rake and sake, except that you can write the tasks in syntax that looks more like ruby as opposed to it's own custom DSL. Once you've installed thor, you need the merb tasks:

gem install thor

thor install http://merbivore.com/merb.thor


and now to install merb and it's dependencies you use thor

thor merb:edge --install


And a useful piece of advice for those that like to drink the kool-aid a little too much, if you've already got a perfectly good working app in rails there is really no point porting it to merb.

Keep a hot cache with merb-cache

The video stream died at this point so I've nothing to talk about, I'll come back and update it retrospectively once I've caught up. The same goes for the following topics. If anybody can kindly fill me in in the interim while I try and track down some streams...;

Using multiple databases in Merb: with special guest CouchDB

DataMapper: the Ruby ORM that kicks butt and takes names

Merb and Sequel

jQuery on Merb

Yay! Someone publically pointing out why using the built in rails ajax/javascript helpers is generally a bad idea. Especially remoteformfor. Javascript is part of the web stack, and you'll eventually get to a point where you actually need to know how to program in it so hiding behind the rails helpers can only help you for so long. Plus all the nasty violations you get re separating presentation from logic.

Testing with Javascript

For testing javascript there is ScrewUnit (check out this post for more detail)

which is a javascript port of rspec with before, after, and describe blocks. There is also a mocking framework for handling XHR so you can appropriately test your ajax requests.

Testing with selenium is good for the full coverage and acceptance testing that a person would normally be doing. It can be a difficult to test everything you care about though.

Why can we use ruby to generate HTML and SQL, but not Javascript?

SQL is a mismatch for what we're trying to do, hasmany, belongsto, and all the various helpers represent logic which isn't well defined in the SQL layer. And HAML is only a very very thin abstraction layer from HTML. Javascript is an entire language though, which isn't running on our environment it's running on the client PC and has it's own state. Most Javascript code generators don't work that well, and the point at which you need to actually know the language is hit much earlier in the development of a project than it is for something like SQL. That being said, it's still useful to have someone who actually knows SQL on your team.

Hi, I'm Glenn! 👋 I've spent most of my career working with or at startups. I'm currently the Director of Product @ Ockam where I'm helping developers build applications and systems that are secure-by-design. It's time we started securely connecting apps, not networks.

Previously I led the Terraform product team @ HashiCorp, where we launched Terraform Cloud and set the stage for a successful IPO. Prior to that I was part of the Startup Team @ AWS, and earlier still an early employee @ Heroku. I've also invested in a couple of dozen early stage startups.