Once Upon a Beta: My NSSA Experience

I had the privilege of being one of the beta testers for NextStage Sentiment Analysis (NSSA) over recent weeks, and this is a total rewrite of my original report of my experience. Doing sentiment analysis on your own writing can be quite a revelation – it can definitely put you in your place. First of all, let me tell you a couple short things about me. I have worked in web analytics for about ten years. I view NextStage’s work as a kind of brave new world (that has such people in it!) that – even with careful and patient guidance – feels just beyond the edge of what I can comprehend. I will tell you honestly that, unlike (probably) other beta testers, I have not experienced a need for sentiment analysis in my job. What I am saying is that my testing approach and my desire to be involved came simply out of analytical curiosity. I didn’t come with a pile of case studies, or anything like that. I am going to do my best here to tell you about my experience. I invite you to ask me any questions that you think I may be able to help with, and I understand if you’d feel more comfortable ignoring this kind of layman-style review.
I am going to start by telling you why this is a total rewrite, even though you’ve probably guessed. I ran sentiment analysis on my first draft and it flagged high for some things I felt were fairly negative, and indicated I would be doing any readers a disservice. I will start with what was not surprising, and explain it to you. I flagged very low for confidence (the scale originally conceived as the BS meter). This is not surprising – as I described above, I am no expert on sentiment analysis (by a LONG shot) and I don’t think I spent enough time being clear about that in my original draft. And despite Joseph Carrabis’s efforts, I always feel a bit of an interloper here in this world of neuroscientific analysis. However, I grasp what my scientific value to the beta effort is: analyst. So I hereby am attempting to provide you with analysis, which should go better. I am sure my confidence will still be low, because that’s my nature, but it shouldn’t be quite so abysmal.
What surprised (and cowed) me was that I also scored very highly for “Retribution” and “Troll”. I see why I scored high on Troll. My tone in the first write was flippant, in a failed effort to project a bit of confidence. I made a few jokes inside, mostly of the variety that would only be funny to me, referencing Dr Seuss and throwing in statements informing you that you could ignore that, etc. Retribution really knocked me over though, and I’ve spent a few days pondering it (without reaching out for help, just as a test). I’ve decided that it was two factors: because of the research I did for my review and my references back to web analytics (which were largely flippant and probably didn’t help my Troll score either). But let me talk about that separately. I hope you’ll stick with me here!
If you’ve read the previous postings on sentiment analysis (I put a list of the ones I found most useful at the end of this posting), you know that in NextStage’s view you need multiple dimensions of data to determine sentiment truly. In preparation for writing this review article, I read a few articles on sentiment analysis. I didn’t do this in a particularly scientific way, I just sort of “turned up” my sensitivity to the term in my regular reading. I took this approach because I wanted to get a feel for what my peers (web analysts) thought sentiment analysis was. Or where it was failing, what was needed, etc. Every article I read talked about scoring a statement (usually a Tweet) as positive, negative, or neutral. Sometimes neutral was omitted. Because I did this “research” after doing my beta testing, you can imagine I had some preconceived notions about why this was inadequate. I likened these people to those that use web analytics only for Hits! (pretty cruel, really) Now, you (or heck, even Joseph) may argue that that can’t cause the retribution flag. That was the best I could come up with after pondering.
Which brings me to a very key point: I had the benefit of Joseph’s help through the beta process. The NSSA results (in current form – I understand plans are to work on this) are really not information that the average analyst can walk up and interpret. I don’t tell you this to trouble you; I am just telling you that the data is nuanced. My first beta interpretation was wrong in so many ways that it could easily fill another post. And, even though I’ve described reports and values individually, they are actually interdependent. For instance, not only was my confidence low, but I scored low on trust and affinity. That’s because I have a very limited idea of who follows Joseph – I know you’re not all web analysts. There’s a good chance some of you think web analytics is total bunk pseudo-analysis! I do not expect that number to improve on this review. I am struggling to give you a clear picture of how the various values interweave, but they truly do, which is a key point. Just like you cannot rely on a single isolated metric or KPI in web analytics, you cannot rely on a single metric in sentiment analysis.
So, to summarize before this gets so long as to be unreadable: the analysis is eerily accurate and eye-opening to say the least. If you write or read, you will probably find yourself in need or at least in want of this tool at some point in your life. I find no evidence out there that there is a comparable tool at your disposal, so when you find yourself needing to know author sentiment (including your own sentiment) you will come back to NSSA.

Thank you for reading,
Twitter: @jdaysy
Skype: cmjenday

P.S. As promised, the four postings that I leaned heavily on during the beta:

  • http://www.bizmediascience.com/2009/06/sentiment_analysis_anyone_part.html
  • http://www.bizmediascience.com/2009/06/canoeing_with_stephane_sentime.html
  • http://triquatrotritecale.hungrypeasant.com/?p=25
  • http://triquatrotritecale.hungrypeasant.com/?p=40

Taking the Harder Path

For reasons ununderstandable to me, there’s much overt and covert discussion about whether online analytics is hard or not, or even if it should be.

It seems an odd discussion to me. I regularly receive analytics reports on my various blogs and such. Interesting to look at and nothing I recognize as actionable. Guessable, yes, obviously actionable, no. I also receive NextStageish reports on my various blogs and such. Those let me know what my readers are thinking and responding internally. Much more valuable (to me) than “clicks on a page” because now I know why they clicked or didn’t.

For example, today I was sent Fascinating and frightening, but real magic! a customer review of Reading Virtual Minds V1: Science and History on Amazon.com. The review took me by surprise:

“What started as an exploration into how I might provide a better user experience to website visitors quickly turned into a journey of self exploration. Joseph Carrabis is able to explain complex subjects in simple and easy to understand text. This book solidified so many connections for me that on more than one occasion I thought to myself “WOW, so that’s why people do that!”
“As you get to the end of the book it becomes clear that ET in the wrong hands could be a bad thing. Yet, it’s exciting to think of all the positive and good uses for this sophisticated technology. The book really sparked my imagination and I can’t wait for volume II!”

I thanked Ms. Flatt via Twitter. Her comments are, to me, sweeter than wine, finer than gold, bluer than the most midnight sky and truer than my heart with my lovely Susan beside me.

Ms. Flatt’s review is an example of someone getting it. No study, no work, nothing is going to be as fulfilling as the work, study, play, whatever, you do that helps you discover yourself, helps you learn how to be a better being, a better you.

But the person getting it needs to be ready, willing and able to do the work involved in getting it.

And although I’ve been talking, teaching and presenting on such topics for over 20 years now, there is still no greater thrill for me than knowing someone has peeled that onion back just a little bit, has peered into the face of their own internal gods, has learned who, what, when, where, why and how they are and has started down the path of getting it.

Because to truly be of service to another you must first learn to be the other. One way to do that is to learn more about yourself, the peeling of the onion, and in most cases be prepared to cry.

My opinion, that.

The work is often challenging, the path often difficult, the rewards always wonderful and worth it.

My opinion, that, as well.

So although I don’t recognize online analytics as “hard”, I do recognize harder paths, such as the one of self-discovery.

I’m about to take such a harder path, this time volunteering for Web Analytics Without Borders (WAWB). This is the harder path (in my opinion) because rolling up one’s proverbial sleeves and offering to help is inherently more challenging than complaining about a problem, pointing out the errors others are making in working towards solutions and not getting involved oneself. The resources outside of oneself when volunteering are minimal if they exist at all therefore the resources must come from within, which means one must discover them, hence it is a path of self-discovery and inherently harder than otherwise.

So you, Dear Reader…are you willing to challenge yourself to make life less challenging for others? Contact Stephane Hamel or Adam Laughlin at WAWB.

And thanks.