From data to action: Using metrics to improve open source communities

See how these reusable practitioner guides from the CHAOSS project can help improve open source communities.

Continuously working to improve the overall health of our open source projects is key to making our projects and the broader open source ecosystem more sustainable. An important starting point is measuring your open source project health to help you decide where you should improve.

We’ve already covered measuring open source project health in a previous blog post which talked about using the CHAOSS Starter Project Health Metrics Model to kick off your project health metrics efforts. But gathering data is only the first step. You also need to use what you learned from the metrics to improve the health and sustainability of your open source projects, which is the focus of this article.

Introducing reusable practitioner guides

If only it were as simple as it sounds. It can be difficult to generate meaningful insights from the tsunami of data generated by open source projects and use those insights to improve the health and sustainability of a project. This is why the CHAOSS project recently launched a series of MIT-licensed Practitioner Guides to help improve contributor sustainability, responsiveness and organizational participation for open source projects. These reusable Practitioner Guides are on the CHAOSS Data Science Working Group repository. You can contribute, suggest new ones, fork or customize them for your needs.

The series starts with an Introduction Guide that covers some common approaches to interpreting metrics and making improvements based on what you’ve learned. Because no two open source projects are alike, you might want to start by thinking about your goals to help you decide what to measure and how improving in certain areas will help you achieve your project’s unique goals and needs. Getting key people (e.g., maintainers) involved during the goals stage can help you make sure you’re on the right track before you start trying to make improvements. This guide contains many tips along with some cautions and considerations to keep in mind when using metrics.

The Contributor Sustainability Guide focuses on growing your contributor base and promoting contributors to become maintainers since contributor sustainability has a large impact on overall project sustainability. There are a lot of projects with a single maintainer and most projects struggle to find enough people to continue to maintain them over the long term. Contributor sustainability metrics can help you think about who you can encourage to contribute more or maybe find someone who could move into a leadership role (e.g., maintainer or approver).

Reaching out to someone specific and encouraging them to participate can help quite a bit with growing your contributor base. It’s also important not to underestimate how much time you can free up from maintainers if they can spend less time answering questions or redirecting people by focusing on a few relatively easy things that can help your project scale like onboarding/contributor documentation or good first issue/help wanted labels. The catch here and with many metrics is that you don’t want to just think about the code. Maintainers often spend more time than you might realize on other tasks and moving people into leadership positions to be responsible for documentation, community management, marketing and other important roles can help free up time from overworked and burnt-out maintainers.

Responsiveness is critical for new contributor recruitment and retention of existing contributors, so the Responsiveness Practitioner Guide focuses on ways to improve a project’s responsiveness in a sustainable way that doesn’t increase the pressure on already overworked contributors. This ties in closely with the Contributor Guide discussed earlier since one way to improve responsiveness is by recruiting and promoting more contributors, but there are also other ways to improve the responsiveness of your project.

One way to reduce reviewer friction and make it easier for maintainers to review contributions is by using issue and change request templates to help contributors make good contributions that require less work from maintainers. It’s also important to talk to your maintainers to understand where they are spending their time along with the challenges they are facing that might be impacting their ability to respond to contributions to make sure that any improvements you make are likely to help your existing maintainers.

We don’t always spend enough time thinking about how organizations impact the sustainability of open source projects and the Organizational Participation Practitioner Guide is designed to help you think about how it might impact open source project health.

The biggest challenge with identifying trends for organizations in open source projects is that the organizational affiliation data is rarely accurate enough to use without doing some manual cleanup, because in many cases it can be impossible to know where someone works. If most of the work on a project is being done by people at a single organization, the project might be riskier to use and harder to contribute to than a project with contributions that are spread out over many organizations with no single organization being dominant.

You can start by engaging within the community to better understand whether they want contributions, and if so, whether there are any restrictions on how you might contribute. If you work for that dominant organization, you might want to focus on getting contributors from some other organizations by reaching out to people you know who are using the project and might be interested in contributing. It’s also important to make sure that the work is happening transparently and in the open so that other people can more easily contribute.

We hope you find these guides useful and we would love to see you around the CHAOSS community to collaborate as we all work toward a more sustainable open source ecosystem.

---

This article is adapted from “From data to action: Using metrics to improve Open Source communities” by Dawn Foster, and is republished with permission from the author.