12/10/2012

Sentiment weighting: In our experience, qualitative text analysis is far more effective—and ‘mixed methods’ —when not only are codes/themes applied to tag and subsequently count meaningful content or words/phrases (unless you are a certain kind of linguist, of course!), but when the ‘value’ of each application is further is tracked and understood. This is where code weighting by sentiment comes in.

So what does it mean to weight a tag or a code by sentiment in qualitative text analysis?

Sentiment weighting is used to add a value scale—and, hence, another dimension of qualitative text analysis—to each code/theme application to account for intensity.

For example, let’s say you are doing a market research study on hotels in Las Vegas. You want to speak to a certain target market about what matters to them when they are choosing a Vegas hotel. Let’s say you code for the key concepts of luxury, sophistication, and service.

In one interview, a person may mention all three concepts, but the sentiment with which they speak of each concept might not be the same. They may feel very strongly that luxury is key, service is nice to have, and sophistication is completely overrated. Now, we can weight each code application on a 1-5 importance scale with 1 being ‘Very Unimportant’ and 5 being ‘Very Important.’ For this person’s interview, we could list luxury as a 5, service as a 3 and sophistication as a 1.

When you analyze these data across all participants via sentiment weighting, you can see a richer, more precise picture than content coding alone—and bring an entirely new perspective to answering your research questions.

So, what is the best way to weight your codes? Every project is different, of course, but generally we advise people to keep it simple. Bucketing your weighting system on a 1-5 scale, as we did in the example above, allows for more efficiency when it comes to analyzing your data. This type of weighting loses much of its efficacy when the scale has too wide a range. A 1-10 scale might be helpful if you need greater nuance, but on a 1-100 scale the variation across the range will be so wide that you may miss the key insights weighting can bring to your qualitative text analysis. Not to mention the difficulty of staying on the same page when working collaboratively with a team.

An important note on qualitative text analysis and teamwork… At Dedoose, we feel it is important that you can collaborate effectively with your team. That means helping you establish inter-rater reliability is a key priority for us.

Adding a weighting scale can provide greater insight to your qualitative text analysis only if your team members are able to use the weight (and the code application, for that matter) systems correctly and consistently across the board. That’s why we created the Training Center. Using Cohen’s Kappa Coefficient Dedoose allows you to evaluate your team members’ application of codes. Using Pearson's Correlation Coefficient, you can evaluate how different members of your team weight particular code applications.