23 Mar Understanding Open Source Communities – Part III: Netflix OSS Tracker

In the previous posts in this series, I covered how we started looking for a solution and eventually wrote a plugin for Count.ly, to help our product team track activity of our open source users.

In this third part, I will share our approach in monitoring Key Performance Indicators (KPIs) of our Restcomm Open Source Projects. We now do this through dashboards that show us a bird’s eye view of the overall status of the projects, but also a historic view of how the KPIs have been evolving.

First, let’s take a look at the KPIs we chose to monitor:

(Please note this is only v0.1 of this list, since we’re only getting started…)

• Number of Open Issues
• Number of Open Pull Requests
• Number of Stars
• Number of Forks
• Number of Committers
• Average time to close
• Days since last Commit

These kinds of KPIs are enough during this first phase where we are just beginning to form a high-level picture of the status and activity on the Restcomm FOSS projects in GitHub.

Clearly, having a dashboard with all the above metrics can offer a great picture of the current state of the projects. Where things really start getting interesting, though, is when we can start tracking how this activity evolves over time. In addition, we wanted a tool that would allow us to drill down into the data, so we can draw our own insights and investigate hot points.

How did other OSS teams solve this?

We knew were were not the first to address this sort of thing. As always in the Open Source world, we can learn from others and would like to stand on the shoulders of giants who have gone before us.

In speaking to a couple of fellow Developer Advocates / Community Evangelists. we learned that many teams are using proprietary or home-grown scripts that pull this information directly from the public APIs and store it into internal databases for their own use.

This gave us an idea. Ideally, we wanted to start with an Open Source project. This way, we could contribute whatever work we put into this kind of solution back into the community. So we started looking and came across the Netflix OSS Tracker project. In a recorded presentation by Andrew Spyke, we saw what we wanted to do. We highly recommend that you check out the video. It shows how Netflix is gardening the 100+ projects it has open sourced.

The Netflix OSS Tracker seemed to be a natural fit for our projects. Not only did it already provide almost out-of-the-box support for most of the above KPIs, but it was also built on Elastic Stack, which meant… Kibana for dashboards!!

If you’ve recently built a reporting or analytics system, and you haven’t rejected Kibana for a good reason, there’s an equally good chance you did something wrong. It’s an awesome tool that’s evolving at a very fast pace, allowing you to build your own visualisations on the data stored in Elasticsearch (though it’s not limited to just that) and combine those in custom dashboards.

But we note that Kibana may not be the best choice for non-technical users, so you’ll want to think about your audience first. In our case, that wasn’t a problem. Quite the opposite: I was particularly drawn to this flexibility offered by the more technically-oriented user interface.

Taking It Out For A Spin

The out-of-the-box experience with the OSS Tracker was great, because on deployment one immediately sees the stats for the Netflix OSS repos. There is no additional setup.

However, there was a hardcoded value for the GitHub organisation as `Netflix`, meaning some work was necessary to expose the GitHub Org as an environment variable. That’s the first thing we did so we could setup our own organization: `Restcomm`. Fortunately, the project maintainers were quick enough to merge my PR, so this now means you can all use the Netflix OSS Tracker for monitoring your own GitHub projects!

Deployment was pretty simple, since the Netflix folks were kind enough to provide a `docker-compose.yml` file. It just required some updates to the latest Elastic stack versions.

Another limitation of the `docker-compose.yml` file was the lack of persistence for the Cassandra and Elasticsearch data, which you clearly wouldn’t want to lose, however, the corresponding PR for this has also been merged, leading to a more production-ready deployment of the system.

Huge shout out to the Netflix OSS team for their help on this!

With all this in place, we were finally good to go!

Time For Deployment

Since the project comes with a `docker-compose.yml` deployment descriptor, all we needed was some (set of) server(s) with just Docker and Docker Compose installed on them. It’s possible to then choose to deploy either via Docker Compose (if on a single server), or through Docker Swarm, if you are aiming for more than a single node.

Configuration Management

To handle the installation of these dependencies, we used Ansible as our Configuration Management tool of choice. We chose to use two community playbooks that handle Docker and Docker Compose installation respectively.

Provisioning

For the servers and cloud infrastructure required, we use Terraform to create a separate VPC, (with the necessary subnets, security groups, etc.) and then spun up an EC2 instance on Amazon Web Services.

We have released all the scripts for this deployment in a public repo on GitHub.  

Success!

We can now finally start monitoring those KPIs. For more information on this project, or other Telestax products, please contact sales.