How Charming - Part 1

When I take a step back and look at it, I’m pretty happy with Honeycomb.

As part of the Writers’ Guild, I take part in weekly…well, meetings isn’t quite the right word. We call them Coffeehouse Chats, which is closer to what they are. They’re something between a formal meeting, in that we have a loose schedule to adhere to, and a group of friends meeting up for tea. We start with our accomplishments from the previous week and we end with our goals for the next week, but the intervening forty to fifty minutes are spent basically doing whatever. Sometimes we’re organized enough to come up with a topic to occupy us for that whole period, but we’re not too stressed if we don’t.

During our October 11th Coffeehouse Chat, one author raised a question that occupied us quite thoroughly for the whole allotted hour and quite a bit after.

Our existing solutions aren’t doing a good job of promoting authors and writing in the community because they’re so focused on visual art, and submitting written works is often very difficult. Everything non-visual falls by the wayside. There are reasons for that, but, if we had the ideal writing site, what would you expect it to be and what would you want out of it?

That’s the start of a spec if I’ve ever heard one, so I perked up and started asking questions. Authors, it turns out, have quite a bit to say on what it is that they would like in a writing site, and quite firm ideas about what is lacking in existing sites. Thank goodness for this being online, as it was easy to turn the logs from the chat into a loose spec.

We settled on the name ‘Honeycomb’, a reference to the cozy communal space within the rabbits’ warren in Richard Adams’ Watership Down.

Over the next few months, I hammered away in my off time on the project. It felt good to get back in the swing of writing a python app, really, and I was happy to see just how far Django had come since I last touched it (from 1.4 to 1.10). I picked Django primarily because it was popular and well documented, a good low barrier to entry for any other programmers in the community who would like to help out along the way.

Fast forward to today. I’m nearing my self-imposed 0.0.1 milestone, and I’ve got a functional site that’s got nearly all of the goals I set out to accomplish in place. I’ve got a QA instance up and running on a linode somewhere in Dallas that the other members of the Guild have been poking at now and then when they have the chance.

The QA instance is limited by necessity. It still uses a sqlite database, for instance, and the search and cache functionalities are disabled. I have Postgres running on that machine already, but with the schema being so in flux during development and migrations leading to some undesired results in the process, it felt less than worth it. I didn’t even try to get ElasticSearch or memcached up and running. Nevermind the fact that alopex, that linode in Dallas, is a little underpowered to be posting a site like that alongside all of the stuff it already hosts.

{% include tech/how-charming/part-1-alopex.dot.svg %}

I wanted something much more robust, though, for Honeycomb, and possibly some other projects. I wanted something that would be easy to scale as needed, and would be easy to spin up as a production-quality stack for even a lay-person. I wanted to figure out a way to make my pie-in-the-sky dreams of running a fancy-pants popular website possible, even if they weren’t actually a reality, basically. I wanted caching, searching, load balancing, logging…I wanted it all!

Unconstrained by reality, I sat down in front of a dot pad (and later a .dot file) and plotted out what it was that I really wanted:

{% include tech/how-charming/part-1-cloud.dot.svg %}

“Whoa whoa,” I hear you say. “Tone it down there, Maddy.”

Fine. Grumble grumble.

{% include tech/how-charming/part-1-goal.dot.svg %}

This is what I want, when it comes down to it: separation of concerns, scalability, existing solutions, and tooling to implement everything without needing to start from scratch.

Luckily, I work for Canonical on Juju, a DevOps solution that lets you encapsulate deployment methods in packages, called ‘charms’, which you can then deploy to a cloud provider. These charms can be composed into bundles and deployed easily to create your application stack - a ‘model’ - which is managed by juju from a central system - a ‘controller’ - that you or others control.

With that in mind, I started prowling around through the charm store for the stuff that I need. Postgres? Check. Apache? Check. ElasticSearch? Check. Memcached? Check. Honeycomb? …er, well, we don’t want to charm something so specific; charms are meant to be reusable. Django, maybe? Well…there’s a django charm, but it’s pretty out of date, and there’s no relation for ES or memcached. Likewise, there’s a uswsgi and a gunicorn charm, which may come in handy, but which require the existing Django charm.

Hmm.

Well, how about we give writing a WSGI app charm a stab? After all, I like writing in other WSGI compatible frameworks, such as Flask, so something more generic like that would be useful! Then my stack, proposed above, would apply to any WSGI project I’d like to write:

There’s our bundle, as Juju will see it. The only thing that’s missing is the wsgi-app charm that we want to write in the middle there. That’s our goal for the next little bit, here: writing a stable and reusable charm and pushing it to the charm store so that we, and anyone else, can use it to deploy our stack.

Next up, we’ll be taking a look at charms from two different points of view: hooks and layers.