Partial Templates with Puppet

Partial templates are a design pattern that are pervasive pretty much every where templating is used. The concept is very simple - any sort of data that needs to be reused can be pulled out into a file by itself, and applied within other templates. This also means that if there’s a particularly complex set of templated behavior, that behavior can be isolated and maintained by itself. And of course if you’re using a partial template, if there’s a bugfix applied to that partial then you make the change in one place.

Partial templates are frequently implemented as one template calling out to the templating engine with another template, and embedding the result inside of that first template. For instance, Ruby on Rails considers any template that’s prefixed with an underscore to be a partial template. So given the following code fragment:

<%= render "menu" %>

When this trivial template is evaluated, the file _menu.html.erb will be rendered and inserted into this template, per Rails conventions.

Fun fact - Puppet is trivially capable of using partial templates. However, I’ve never seen this pattern implemented anywhere. This is a shame - partial templates are awesome and allow you to tidy up a lot of code.

Templating vhosts with nginx or apache is a perfect example of this pattern. In general vhost definitions share a lot of commonality, but you’re almost always bound to have one or two tweaks that have to be applied. This leads to having a different template for every single possible application or situation, and in this case you have a set of templates that are almost, but not quite, identical.

Let’s look at a real world example that I’ve implemented

nginx
├── README
├── manifests
│   ├── init.pp
│   ├── loadbalancer.pp
│   ├── server.pp
│   ├── unicorn.pp
│   └── vhost.pp
├── templates
│   ├── nginx.conf.erb
│   ├── vhost
│   │   ├── _listen.conf.erb      <-- Partial templates
│   │   └── _servername.conf.erb  <--
│   ├── vhost-default.conf.erb         <--
│   ├── vhost-loadbalancing.conf.erb   <-- Vhost templates
│   └── vhost-unicorn.conf.erb         <--
└── tests
    └── init.pp

On the manifest front, we have the loadbalancer.pp, unicorn.pp, and vhost.pp files. Each one is a fairly simple define; loadbalancer.pp provides for nginx backed load balancing, unicorn.pp serves a unicorn backed rack app fronted with nginx, and the vhost.pp implements a basic http file server. Each vhost is going to have to define how it is going to listen on ports, but this is going to vary wildly depending on if this is the default vhost, if SSL is enabled, if IPv6 is available, and so forth. This is a prime candidate for using a partial template.

To show how to use partial templates, I’m going to use the load balancer as an example. Here is the main template, nginx/templates/vhost-loadbalancer.conf.erb.

# Nginx load balancer configuration
upstream workers {
<% @workers.each do |worker| -%>
  server <%= worker %> max_fails=<%= @max_fails %> fail_timeout=<%= @fail_timeout %>s;
<% end -%>
}
server {

# *** This is where the partial template is included
<%= scope.function_template(['nginx/vhost/_listen.conf.erb']) %>

  root /usr/share/empty;

  location / {
    proxy_pass <%= @proto %>://workers;
    proxy_redirect        off;
    proxy_next_upstream   error timeout invalid_header http_500 http_503;
    proxy_connect_timeout 5;
  }
}

When you’re writing normal Puppet manifests, calling out to the template function means simply calling template(). However, ERB templates are evaluated as pure Ruby, so calling Puppet functions from Ruby means calling scope.function_myfunctionname(). So in the above example, all that’s happening is within a vhost template, we’re calling out to the template nginx/vhost/_listen.conf.erb. You’ll notice that the partial template is prefixed with an underscore - this is entirely my convention.

And now, for the partial template itself:

<%# Configuration fragment for listening on IPv4 and IPv6 with SSL %>
<% unless @sslonly -%>
  listen   <%= port %>;
<%   if scope.lookupvar('::ipaddress6') -%>
  listen   [::]:<%= port %>;
<%   end -%>
<% end -%>

<% if ssl -%>
  listen   <%= ssl_port %> ssl;
<%   if scope.lookupvar('::ipaddress6') -%>
  listen   [::]:<%= ssl_port %> ssl;
<%   end -%>

  ssl_certificate      <%= ssl_path %>/certs/<%= ssl_chain %>.cert;
  ssl_certificate_key  <%= ssl_path %>/private/<%= ssl_key %>.key;

  # Redirect non-SSL on SSL port to SSL
  error_page  497  https://$host$request_uri;
<% end -%>

This is obviously very gnarly code. This handles every single permutation of SSL, default vhosts, IPv6, and will even redirect HTTP traffic send to an HTTPS port to the appropriate HTTPS URL. Using the partial template pattern this complex code is isolated to a single location for easier maintenance, and it can be reused by any number of templates.

So, what does the final configuration file look like?

upstream workers {
  server 192.168.128.101:80 max_fails=3 fail_timeout=10s;
  server 192.168.128.102:80 max_fails=3 fail_timeout=10s;
}
server {

  listen   80 default;
  listen   [::]:80 default ipv6only=on;

  listen   443 default ssl;
  listen   [::]:443 default ipv6only=on ssl;

  ssl_certificate      /etc/ssl/certs/mycert.cert;
  ssl_certificate_key  /etc/ssl/private/mycert.key;

  # Redirect non-SSL on SSL port to SSL..
  # http://www.ruby-forum.com/topic/193781#844520
  error_page  497  https://$host$request_uri;


  root /usr/share/empty;

  location / {
    proxy_pass http://workers;
    proxy_redirect        off;
    proxy_next_upstream   error timeout invalid_header http_500 http_502 http_503 http_504;
    proxy_connect_timeout 5;
  }
}

By using partial templates, the main load balancing template just has to handle the details of the load balancing implementation. All the complexities of handling the listen configuration are entirely abstracted away, and Just Work(TM).

This partial template pattern can really improve the modularity and reusability of your modules. Instead of trying to write templates that can solve every single possible problem, or simply provide a barebones template, you could provide a set of partial templates instead. This allows people to compose things as they need while still rely on your module to handle the complex bits.