Measuring open source community health with Savannah

Track what matters to support growth, sustainability, and contributor experience.

tl;dr

Savannah is an open source Community Relationship Manager (CoRM) that helps track community health across repositories and discussion platforms. It’s brilliant for identifying new contributors, spotting disengagement early, and measuring the impact of your community efforts. Keep reading if you’re interested in why this matters and how it works.

Gut feelings aren’t metrics

Throughout my career, I’ve worked worked with organizations that create widely-used open source tools. The popularity of these tools is evident through their impressive download statistics, strong community presence, and engagement both online and at events.

But, in my experience, most project maintainers rely on gut feelings or anecdotal evidence when assessing their community health. “The Discord feels active” or “We’re getting more GitHub stars” aren’t really metrics you can track over time or use to make decisions.

What if there was a tool that could provide concrete metrics across all your community platforms? Enter Savannah, an open source Community Relationship Manager designed specifically for tracking and analyzing open source community health.

What does “healthy” even mean?

Community health goes beyond merely counting stars on GitHub or tracking download numbers. That’s like judging a car solely on its shiny paint job and ignoring the engine.

A genuinely healthy open source community demonstrates consistent engagement, shows a balanced mix of new and returning contributors, and maintains reasonable response times to questions and contributions.

The trouble is, measuring these factors can be challenging, especially when community interactions happen across multiple platforms. You might have GitHub for code, Discord for chat, a forum for longer discussions, Twitter/Mastodon for announcements…this fragmentation makes it nearly impossible to get a holistic view without something to tie it all together.

Read more: Why open source matters in your project management tool

Savannah: The multi-platform community tracker

Savannah takes a refreshingly practical approach to community health metrics by focusing on two key types of engagement:

• Conversations: Any discussion within the community, whether in code issues, discussion channels, or forum threads.
• Contributions: Tangible artifacts created by community members, such as code, documentation, or support for other users.

By tracking these elements across multiple platforms and cleverly de-duplicating community members, Savannah provides a unified view of your community’s engagement. No more jumping between six different dashboards trying to piece together what’s happening!

The visualizations that make you go “Aha!”

The engagement pyramid

One of Savannah’s most insightful visualizations is the engagement pyramid, which categorizes members into four distinct levels:

• Visitors – The drive-by folks who pop in with a single interaction, like reporting an issue
• Participants – Members who return and engage regularly
• Contributors – Those who provide tangible contributions to the project
• Core – Regular contributors who form the project’s backbone

The shape of this pyramid can tell you a lot about your community’s health:

• Too bottom-heavy? People aren’t sticking around after their first interaction
• Too top-heavy? You might be excluding casual participants
• Bottlenecks between levels? There are obstacles preventing community members from leveling up their engagement

I’ve personally seen communities where visitors rarely become participants—it turned out their onboarding documentation was confusing and intimidating. Metrics made this visible.

Connections network

Savannah also maps the connections between community members, showing who interacts with whom across platforms. This network visualization helps identify:

• Community anchors who connect different members
• Potential single points of failure where one person holds too much responsibility

It’s like seeing the social map of your community—fascinating and sometimes surprising!

Dashboard metrics

The main dashboard gives you the vital signs at a glance:

• Number of conversations and average response time
• New member growth rates
• Active member count trends
• Company participation (for corporate-sponsored projects)
• Notifications about new contributors or drops in activity

Real-world applications (not just theoretical)

Based on actual experience using Savannah, here are some ways it can enhance your community management:

Rolling out the red carpet

Savannah notifies you when a new contributor makes their first contribution, allowing you to respond promptly and make them feel welcome. This personal touch has dramatically increased the likelihood that first-time contributors will return.

Even if they don’t return immediately, their experience may live on via word-of-mouth recommendations to others looking for a project to contribute to. “Yeah, I sent in a small doc fix to that project and the maintainer personally thanked me within hours!” is the kind of reputation you want.

“Hey, where did everyone go?”

When a previously active contributor’s participation drops, Savannah highlights this change. This early warning allows you to reach out and head-off potential issues before they lead to contributor burnout or departure.

Sometimes, people stop contributing for personal reasons, be it starting a family, getting a new job, or just working through their Netflix recommendations. It’s good to know why if you care about your community. Often, it’s not you, it’s them—but occasionally there’s something you can fix!

Did that hackathon actually work?

By segmenting your community data, you can measure the impact of specific initiatives. For example, you can track engagement before and after implementing a mentorship program, documentation sprint, or that expensive conference sponsorship.

No more wondering if your community investment was worth it—the numbers will tell you.

Not all heroes commit code

Not all valuable contributions are code. Savannah recognizes and tracks various contribution types, ensuring that documentation writers, community supporters, and other non-code contributors receive appropriate recognition.

My first significant contribution to open source was answering support questions without contributing code. The project recognized and appreciated that effort, making me feel welcome and valued.

Getting started with Savannah

Savannah is an open source project that can be self-hosted with modest resources. The project lead also runs a Software as a Service (SaaS) solution if you’d rather not deal with the infrastructure:

• The SaaS option provides a straightforward way to start without managing infrastructure.
• Self-hosting offers complete control and is ideal for those comfortable with maintaining Python/Django environments.

Self-hosting is also a great stepping-stone to contributing to Savannah itself if you’re that way inclined.
To get started with either option:

1. Create a Savannah community
2. Add “sources” such as GitHub, GitLab, Slack, Discord, or Discourse
3. Configure “channels” for each source (repositories, Slack channels, etc.)
4. Let Savannah gather historical data and generate insights

While setting up API connections to various platforms can be somewhat complex (what isn’t these days?), the resulting insights make the effort worthwhile.

Real-world success: Universal Blue

The Universal Blue project recently adopted Savannah to monitor community health across over 60 channels spanning GitHub, Discourse, and Discord. Their annual report for 2024 showed impressive metrics: 3,313 new members, 207 new contributors, 37,410 conversations, and 715 contributors across all platforms.

The project leaders are also able to see immediately when new contributors arrive, to roll out the red carpet, and to make them feel welcome and their contributions valued. This level of holistic insight would be nearly impossible without a tool specifically designed to aggregate community data across platforms.

Not everything can be measured (and that’s okay)

Despite Savannah’s capabilities, measuring community health comes with inherent challenges. Qualitative aspects, such as a community’s general atmosphere or inclusivity, are difficult to quantify. Platform limitations mean some interactions may not be captured by APIs. Context matters when interpreting metrics; numbers alone don’t tell the whole story.

The most effective approach combines Savannah’s quantitative insights with qualitative community observation and direct feedback. Use the metrics to guide your attention, but still trust your instincts about the community’s feel.

Give it a whirl

Open source communities thrive when maintainers can see patterns in health and engagement. Savannah provides this visibility by aggregating data across platforms, identifying trends, and surfacing insights that might otherwise go unnoticed.

Whether you’re maintaining a small project or overseeing a large open source ecosystem, Savannah offers valuable metrics to guide your community-building efforts. The tool’s open source nature also means it can evolve alongside community needs, with contributions welcome from the broader developer community.

If you’re keen to gain deeper insights into the health of your open source community, visit the getting started with Savannah docs to learn more, or contribute to the project if you have Python/Django expertise.

Measuring what matters in your community, can assist in making informed decisions that foster growth, sustainability, and a positive experience for all contributors. And isn’t that what we all want for our open source projects?

More from We Love Open Source

• How Cody AI can supercharge your development workflow
• Why open source matters in your project management tool
• Tech career growth: Networking, AI tools, and the power of community
• What is prompt engineering?
• What is OpenTelemetry?