Go Back: The Future of Measurement

Last week, I asked a diverse group of researchers to share their thoughts on the future of measurement. The researchers (names below) loosely have 1-3 common denominators across psychology, artificial intelligence, text analysis, social media, and/or advertising research and analytics. As witnessed in the visualization of our discussion (courtesy of Wordle), data was the currency of conversation. Broadly, responses spanned tools, methods, metrics, constructs, and value.

I categorized responses into 4 major predictions about what 2009 will hold measurement-wise. In the pdf, available for download below, I've extracted verbatim from all respondents to support the predictions. The discussion took the form of a Google Group, so in some cases, it was necessary to remove substantial dynamics of conversation. I’ve done my best to capture the essence of themes represented and hope this aggregation will be insightful in understanding where things have been, and potentially where they are going.

I walk away with a sense that researchers are optimistic, yet there’s an undercurrent of “let’s get real.” What are we really measuring—what is a meaningful connection, what do we really pay attention to...

In other words, to paraphrase Diane Court (Ione Skye) in Say Anything:

I’ve glimpsed our future, and all I can say is: Go Back.

Let’s get back to basics in 2009 and measure something meaningful.

Predictions:

• We will substantially advance our understanding of individuals and the meaningful connections they have.

• We will identify methods to tap what people are *really* thinking, feeling, and paying attention to, meanwhile gaining insight on what a measurement is truly capturing.

• We will determine how to measure the value of social interactions and attach financial value, whether we’re monetizing attention or a new medium.

• We will build better tools to manage-- analyze and visualize-- massive volumes of data, primarily tapping the evolving social graph.

Thanks to the many researchers who contributed:

Neal Burns, University of Texas, Austin Center for Brand Research
Walter Carl, Chat Threads,
Maury Giles, GSD&M Idea City
Sam Gosling, University of Texas, Austin: Department of Psychology
Seth Grimes, Alta Plana
Matthew Hurst, Microsoft Live Labs
Paul Janowitz, Sentient Services
Shawn Kung, Aster Data
Roddy Lindsay, Facebook
Jamie Pennebaker, University of Texas, Austin Department of Psychology
Martha Russell, Media X at Stanford University
Miles Sims, Small World Labs
Marc Smith, Telligent
Daniel Tunkelang, Endeca

Download the pdf summary here.

lazy communication: chronicles of our social graphs

I've been thinking about social graphs... and how egocentric they are.

There's a classroom experiment where students are asked to write the letter E on their forehead. Some people orient the E towards themselves; others think more empathetically and draw an E that's readily perceived by others. When facing themselves, it's an example of an egocentric bias.

Point is, being egocentric isn't necessarily about being selfish or about social desirability, it's a tendency to see things in a limited way, from your perspective only. Like kids who can't yet put themselves in others' shoes.

I think we're egocentric in our communication patterns. Even though we're not charged for long-distance-- calls or emails-- we tend to talk to people really nearby. Think about the ever-enduring silos in organizations. HBS researchers studied >100M emails and >60M calendar entries from +30k employees at a complex corporation in 2006:

"Our analysis indicates that two people who are in the same SBU, function, and office interact about 1,000 times more frequently than two people at the company who are in different business units, functions, and offices, but are otherwise similar."

We communicate with people who are within an arm's reach. This reminds me of the classic 1950's study of friendship in MIT housing leading to the adage (?) that proximity is the biggest predictor of friendship. At year's end, people reported being better friends with people who lived next door to them; people who lived near staircases reported being better friends with people on the second floor!

Are we communicatively lazy? How immersed are we in our full social graphs? Seems likely that we communicate in pretty isolated networks, not dispersed across our entire graph.

Consider Umair Haque's statement on how to be a 21st Century capitalist:

"Yesterday's businesses were built on cash, factories, and IP - financial, physical, and intellectual capital. Next-generation businesses are built, instead, on human, social, natural, and cultural capital - to name just a few."

If tomorrow's business is really going to be built on social capital, we need to figure out how to span our networks.

learning, organizing, anticipating: better

I joked to my colleagues the other day that I wanted a personal assistant who would not only accurately intuit what I wanted for lunch, but proceed to feed it to me so I could seamlessly work (with two hands) and eat. Pathetic, yes, but after reading about CALO today in the NY Times, I’m optimistic things might pan out.

CALO is the “Cognitive Assistant that Learns and Organizes. After spending some time on the website, I gather that it will automate many of the ‘mundane’ activities in our worklives (e.g. set up meetings with the right people at the right times, and best yet, prepare you with relevant information), based on interactions, activities, and instructions.

Think: Intelligent Amazingness-- it seemingly learns your preferences and priorities by your desktop activities, online behavior, who you physically come into contact with, and perhaps even how you interact with these individuals. From the website:

“The goal of the project is to create software systems that can reason, learn from experience, be told what to do, explain what they are doing, reflect on their experience, and respond robustly to surprise.”

Although it seems unrelated, in a later post, I want to talk about this in relation to Facebook Connect and Friend Connect. I shouldn’t even make this link because the CALO concept is wholly unrelated, but I think it’s important to call out the differences in predicting future behavior based on our daily behaviors, ‘bumper stickers,’ and our connections.

I’ll just say, I’m a huge proponent of figuring out who people are, and anticipating future behavior by what they do in their daily lives-- in fact, it was the topic of my dissertation. I’ve been anti- self-report measures for several years and several reasons.

So this is an exciting development - a new model of intelligent software learning about your worklife and how you navigate it- bound to play a critical role in the evolution of the web and enterprise, even if just inferring what I want for lunch.

metrics in the hands of users

You might have been wondering what I meant when I suggested California Closets revamp my FB account-- my strange need to classify my social graph for optimal predictive ability (e.g. product and music selections). What would new ways to organize and search my FB account really look like? Do I really want a metric next to each friend telling me how similar our tastes are or how good of friends we *really* are?

As I was thinking about this, I was reminded of Xobni and its impact on the lives of two of the more addicted emailers I know. I remembered a former colleague telling me she found out I was her top emailee, but lowest emailer; she quickly stopped emailing me so much. Not only was the app cool, but useful.

Another friend told me Xobni clued them in to their "real" workgroup-- not the one the org chart dictated, but the colleagues who actually facilitate getting work done. Earlier today, someone mentioned an Adium app that analyzes IM usage stats to give you a sense of contacts' likelihood to respond, given your interaction history. Naturally, armed with this type of info, you can see how metrics can directly impact behavior.

So I started thinking about metrics in the hands of users (full disclosure: a discussion that evolved in talking to others). People always think of metrics as managerial tools, but really they should be in the hands of users to bring about the awareness and subsequent change in behavior that managers are most likely interested in in the first place.

the more you give...

As soon as TechCrunch announced the availability of Facebook Connect, I felt pressure to organize my life.

What I mean is, I think our success with Facebook Connect, as individuals, will be dependent on how much information we offer and how well we organize it. Basically, the more you give, the more you get: the more you weed out noise by managing your FB account, the more accurate subsequent communications with you will be. I'd like to have someone like California Closets come in and manage my FB account.

I'm not so concerned with a potential backlash based on people's discomfort sharing so much information. Instead, I'm more concerned people will become disgruntled by not having managed their social graph.

Will people be more discerning about connections from now on? Could FB give us (data-based) tips on identity management? Or is that a psychologist's role?

boundaries, blending and portability

Was watching a colleague login to a wiki the other day, trying various combinations of 3-4 usernames and passwords-- a situation you're probably pretty familiar with. We have multiple, yet limited, online identities, each with their own digital breadcrumbs, associations, and goals.

Even the earliest theorists of self (e.g. James, Meade, Cooley) saw this coming -- our online memberships automatically embed us in different networks; it's only natural to shift your identity based on context. Blogger, mother, NY-er, Texan, etc...

It made me think about our tacit strategies to manage our online identities and attitudes toward portability. Danah Boyd, of course, talks about related issues frequently (and early on).

My Links are fairly distinct from my Friends and who I Follow -- not 100% -- but I don't really use segmentation strategies within networks to adjust who sees what/ what I see.

With single digital identities and proliferating technology top of mind, the effect of maintaining distinctions across networks vs. merging, and porting identities are weighing down on me. A pretty extensive literature in social psych shows that when primed with a given identity, we adopt a certain awareness that guides our attention and evaluations, and influences our subsequent behaviors. This is natural... I'm curious about the opposite effect-- blending boundaries into a melting pot identity and the behavior that would result from a potentially merged self.

liking and living, collaboratively

There are three main reasons why I loved Clive Thompson's piece in the Sunday NY Times Magazine.

1. It questions fundamental differences in how we can define people and predict future behavior: self-reports and preferences vs. behavior.
   
   
2. It highlights the paradox of how social seemingly individualistic things are: The social nature of decision-making; not to mention, the very collaborative style of the Netflix' competition.
   
   
3. It touches on the limitations of computers in understanding nuanced human decision-making.

My favorite things!