Measuring Communities: Protocols, not Algorithms

With the recent release of The Forrester Wave on Community Platforms, there's a lot of discussion about communities from a Product angle. But technology alone does not create value. Nor do traffic, buzz levels, or Authority scores demonstrate it.

I keep coming back to a line in Steven Pinker's piece on consumer genetics this past weekend in the NY Times.

To study something scientifically, you first have to measure it, and psychologists have developed tests for many mental traits. And contrary to popular opinion, the tests work pretty well: they give a similar measurement of a person every time they are administered, and they statistically predict life outcomes like school and job performance, psychiatric diagnoses and marital stability.

It was reaffirming to read this statement... not that I'm in the business of measuring mental traits (anymore), but the process of test construction and validation is a well-worn path. It's one that lends several hints toward creating frameworks to measure meaningful dimensions of social interactions, the currency of communities and collaboration.

The late Peter Kollock was a pioneer of social relationships. He studied, from a sociological perspective, which features contribute to the success and failure of online communities: cooperation and conflict. He wrote about things like the importance of "identity persistence," internal economies, and archives of information as design principles for online communities. But importantly, he concluded, "there is no algorithm for community."

Tying these two ideas together offers an important lesson in measuring value in communities. There are sophisticated methods and approaches to measurement, but no golden rules about what your scores should be. We can readily capture value in communities - with reliability and validity, but it's not going to look the same for everyone.