Just about one month ago, a group of us from Liberty Mutual open sourced a project that we had been working on for a long time internally. This project is called “Herman”, and lives here on GitHub. Herman is a tool that was originally built to help simplify the deployment of a Docker-ized application to AWS Elastic Container Service (ECS).
Up until this point, most of our AWS deployments had been at the IaaS level: using Cloudformation to spin up EC2 instances and deploy applications in AMIs, or using a Pivotal Cloud Foundry-based PaaS. While AWS makes IaaS deployments like this possible over a data center approach, it still isn’t as straightforward as it could be to help developers support large portfolios of microservices. ECS itself is relatively simple to operate and use, but developers were looking for a more PaaS-like approach to ideally manage many applications without worrying too much about the underlying infrastructure. Herman helps to enable this using a configuration file containing various parameters needed to deploy a container to ECS. This template file is meant to live alongside the application code, which means it’s easy to manage through version control and code review through whatever process a team wants to use.
Simplifying ECS deployments was a big step to help move our teams to AWS, but there were a few other pieces beyond just running a container that most applications need to be successful. Most of our microservices use a database of some sort, and many leverage things like queues and messaging services. To this end, a handful of “brokers” were added to Herman to provision AWS services (RDS, SQS, SNS, etc.) alongside application deployments in the same configuration file as the container deployment. These brokers make it easy to configure and deploy ancillary service while including many of our best practices and requirements baked in. There’s also a pattern in place for brokers that need to do some work outside of the Herman runtime, such as provisioning users in an RDS database or notifying NewRelic of an application deployment.
Herman was built using an “internal open source” model, where a small group built the foundation but others were free to contribute back as they wanted additional features. Many service brokers were actually built by contributors from our internal community who wanted to help move the project forward in their spare time, or felt that some new feature could make their lives easier. What I find incredible is that Herman never particularly showed up in our typical day-to-day work, much of it was contributed in spare or innovation time (Coincidentally, this is how I got my start on the project. Configurable timeouts!).
Originally, Herman was built as a plugin for Atlassian’s Bamboo continuous integration tool. That’s still how our teams use Herman today, and the code for the plugin is on GitHub. We know that lots of people don’t use Bamboo or just want options in general, so we’re working on migrating Herman to a standalone application that can be run from a CLI (or have a plugin wrapper built around it for integration with any other tool). Lots of work has been done already both to make Herman applicable to other AWS accounts outside of our installation and toward making it easy to use outside of a Bamboo context.
Today, Herman is open for use and contribution from the open source community. We’re continuing to make improvements on the tool both as we need them and as issues are raised. Check out the repo on GitHub or join us on Slack!
comments powered by Disqus