On our puppet module design

Most puppeteers seem to work for either one company or only deploy one type of service to different companies. You can notice this in the way they build their modules. Most people tend to create one module that contains everything that is needed for a service or application, since they only need to have it (mostly) the same. We at Kumina work for a lot of different companies and try to be as flexible as possible. This makes our module design a fair bit different than most others, in my experience. This post tries to explain some of the choices we’ve made with regards to module design.

We work at three different levels, which we gave separate names:

• The first one is the most generic and we call it… generic.
• The second one builds on top of a generic module and implements our best practises. We call it “Kumina Best Practises” or kbp for short.
• The final one is the customer specific layer, which actually implements our kbp modules with the correct variables for a specific customer.

When designing our modules, we keep several guidelines in mind:

• Prefer to include instead of inheriting. Inheritance is no longer needed since we can pass parameters to classes.
• In generic, the modules name should be the name of the application. So if you’re creating a module for Apache, call it “gen_apache”.
• In generic, you should only add resources that are needed for that exact application, nothing else.
• In generic, you can only rely on functions that are part of puppet or the gen_common module. Nothing else.
• In kbp, you can only rely on functions in gen_common and any included class from generic or kbp.
• In kbp, make sure the module is totally self-contained. This means that if it uses defined types or functions from other modules, it should include those modules explicitely.
• The kbp module should setup monitoring and trending explicitely using the kbp_icinga and kbp_munin classes (or whichever modules we use per default for those).
• The customer specific modules are only allowed to include kbp modules or customer specific modules from the same customer (actually, each customer has her own environment, so they cannot access specific modules from other customers).
• The customer specific modules should be divided in actual service, where the class includes everything that’s needed for a service. So if you have multiple PHP websites, each site has a separate class and each class includes everything it needs, even if that means that both site classes will include the mysql server, for example.
• Anything slightly generic should be built in kbp, not in the customer specific modules.

That looks like a fairly long list, but most of it seems rather logical once you’re working with it. These guidelines make sure that our entire team can quickly and easily work on each customer’s setup, where needed. Keeping the environments separate from each other also allows us to easily see the impact certain changes will make. In practise, most of the resources will be defined in the kbp layer. But the generic layer is still important, because we try to create an API-like approach the applications of the same service type. I’ve described how to do that a while ago on the Puppet mailinglist (I should probably write a blog post about that too, but not today). The main advantage of this being that you should be able to easily replace for example apache with nginx.

Using the above guidelines, setting up our webserver is simply a block like this:

node 'web.kumina.nl' inherits 'kumina_default' {
  include site::www_kumina_nl
  include site::www_twenty_five_nl
  include site::blog_kumina_nl
  include mail::incoming
}

And everything is setup as we need it. The class itself looks as follows (for example):

class site::www_kumina_nl {
  include kbp_httpd
  include site::common

  kbp_httpd::simple_site { "www.kumina.nl":
    ensure => 'present',
    documentroot => '/srv/www/www.kumina.nl/',
  }
}

class site::www_twenty_five_nl {
  include kbp_httpd
  include site::common
  include kbp_httpd::php
  include kbp_mysql

  kbp_httpd::simple_site { "www.twenty-five.nl":
    ensure => 'present',
    documentroot => '/srv/www/www.twenty-five.nl/',
  }

  kbp_mysql::db_with_user { "tf_interface":
    password_hash => 'very_secret',
  }
}

class site::blog_kumina_nl {
  include kbp_httpd
  include site::common
  include kbp_httpd::php
  include kbp_mysql

  kbp_httpd::simple_site { "blog.kumina.nl":
    ensure => 'present',
    documentroot => '/srv/www/blog.kumina.nl/',
  }

  kbp_mysql::db_with_user { "kumiblog":
    password_hash => 'very_secret',
  }
}

If we decide to ever move the blog to a separate server, we can simply do:

node 'web.kumina.nl' inherits 'kumina_default' {
  include site::www_kumina_nl
  include site::www_twenty_five_nl
  include mail::incoming
}

node 'blog.kumina.nl' inherits 'kumina_default' {
  include site::blog_kumina_nl
}

Aside from manually moving the data in the database, everything should work as expected. This allows us to easily move sites (or other applications) from one machine to another.

This way of building your modules either appeals because of the flexibility or seems a horribly inefficient use of your time. We find it’s a nice way to keep some order without losing too much flexibility.