Open Letter to (one) Data Scientist

*Photo by MervC

Dear Dear Friend, 

As you take on the world of analytics and data science make me one promise... well maybe a few.

1) Don't quantify that which cannot be quantified. Specifically qualitative data. The motivation will be to try to turn human behavior into actionable numbers. You will argue that at a macro level, given many transaction/iterations of a particular behavior, you will be able to draw some sort of grand conclusion based on people's propensity to take particular actions given a controlled set of stimuli. I know this temptation and I have and still fall victim, but these generalizations can be exclusionary and if they are institutionalized (i.e. an organizations adds them to the standard list of KPIs) can eventually become obsolete and wrong and can even end up doing damage to the population in question. I don't have a solution but having worked in analytics before I understand the urge and I thought it was always important to be aware of the weakness of my assumptions and I always tried to downplay the significance of my conclusion to allow for error that I was unable to see/measure/understand.

2) Scientists love building models but they are always wrong. The plethora of variables that influence a biological or natural system are broad an varied. Trying to model these is impossible. The inclination is to limit variables to only the important ones. By doing this an unrealistic "world view" is created. And because you are excited about answers and understanding things more completely it easy to want to broadcast the results and try to apply them to other situations. I get it. But this is a mistake. Your conclusion work in a very limited scenario.

3) People will misuse you ideas. You may understand the limitations of your conclusions but others won't. They invariably want science to give an answer and they will apply your model to scenarios that aren't remotely related.

4) Participant observation has to be part of the question. Anthropologists use this a lot. As an observer they are actually changing what is actually being observed. Instead of controlling for their presence they completely embrace the manner in which they change the observed outcomes and contextualize their presence. It is not uncommon for the opening line of an ethnography to read, "As a white protestant upperclass woman..."

5) When at all possible mingle with hoi polloi. Your numbers and observations are about people. You can build models about their behavior or their click through rate, etc. Your numbers couldn't possibly capture everything that is going on. So roll up your sleeves and mingle with your users as often as you can manage. At the end of the day this is all about relationships, behaviors, and people. As much as you may not want to admit it (I don't know if you do or don't) analytics is a marketing effort that tries to get data that is better than what the respondent can give. Combine your hard numbers with qualitative data to get a better picture. Its also more fun this way.

6) Someone once said that if you want to create a better user experience give your product to a toddler. If they can figure navigation out then your product is intuitive to use. The purpose also has to be really straight forward and obvious enough that a toddler gets it. This is good. You have more going on on your site than a toddler can understand? Maybe things should be scaled back. This is reductionist (as cautioned above) but it helps guide your approach, I think.

7) Have fun. The numbers are hat they are. Metrics are what they are. Behavior maybe what you think it is. This effort is all about getting better. There will be very few big wins especially in a big data environment. Its more about small victories.

54.5 MPG is Not About Gas Prices

Photo by: TL Davis

So, if all automobiles in the US improve fuel efficiency to 54.5 mpg and, let's assume that people aren't cutting back on transportation at the current fuel prices, then in the future quantity demanded for fuel will be less because of fuel efficiency and assuming supply stays the same, then fuel prices will increase.

Because there are economists at the Gas companies I would assume that the decrease in quantity demanded will result in an increase of price that results in P*Q=Profits that are similar to profits now.

So, 1) the future price of gas will not pay for innovation and cost of more efficient engines because prices will increase and 2) gas prices seems like a poor argument to justify improved fuel efficiency.

Decreased negative externalities is justification enough for improving automobile fuel efficiency but fuel prices aren't.

Thoughts? The hive is smarter than I am. Surely you can resolve this.

Loudness Wars

This is an image from wikipedia's entry on loudness wars and this is a perfect example of what is happening in the recording industry. Why does this matter? Its not because it really is all that loud, you can turn down the volume of your stereo. Its because all the good part of the music lives on the fringes. Example: image #1 there is plenty of space for "sparkle" to exist in this song. image #3, there is absolutely no space for "sparkle" to live. This occurs because in digital sound there is no going over a maximum level, period.

Loudness Wars suck because the song looses its sparkle. Sparkle is what makes you feel like the musician is in the room with you. It gives a song life. So what can you do about it if everyone is producing louder and louder songs? Well, nothing at the moment. But that will change soon with some exciting products. This also goes back to yesterday's post, What is Sound?

Loudness wars matter and sound matters because if everyone is listening to highly compressed recordings, the dynamic range is lost (i.e. sparkle). Its like eating pure sugar without flavoring. The crazy thing is that this has been going on so long that we don't even know what we are missing.

Why do you care about sound?

What is Sound?

Sound is perfect. We and our instruments are imperfect. We damage or poorly recreate sound waves and they slowly lose something as they move through the air. So what is our role as listeners? Do we try to perfect our ability to recreate or process sound or do we just give up and make sound what we want it to be? It will never be perfect anyway.

This is a philosophical question about truth and authenticity the answer to which plays out in the relationship we have with our surroundings. And this is important because it helps us understand our preferences.

Do you care about sound? Do you care about it being perfect? Or do you care that it just not suck?

This is an important question. I will tell you where I stand, but you have to tell me where you stand first. Don't miss out.

Milk

The past few weeks I've heard over and over again, from a variety of people - young and old - that the idea of milk is very gross. I've heared long diatribes against 2% and how horrible it is for you. Somebody referred to it as just a creamy mess. I drank milk as a kid but mostly the public school chocolate milk. My dad loved to make banana-chocolate shakes with lots of milk when we were kids, but I remember loosing interest in the white stuff really early on. These days it makes my stomach hurt.

So, have I stumbled upon a small group of outliers or has milk fallen on hard times? The Milk commercials are good, but maybe they are just grasping at straws. Do you hate milk as bad as these people? Am I missing something?

Teams and Metacognition

I've been side tracked by facebook and twitter for the last year. I've posted short updates on my life and my opinions which have mainly been directed at my peers. (And I play there because I like being able top comment on their posts.) But I find that I am censoring myself. The truth is that I have been trying to have a conversation on FB that is probably most suited for a blog. Here I am not limited to 147 characters and I am not worried about straining relationships because of my perspective. So, I hesitate to say, you may be hearing more from me here and I may begin to explore my personal struggle with my new surroundings.

You see I have moved to a different part of the country and my social network has expanded to include people that are much different then back on the east coast. This is compounded by the fact that I now live in a boarder state and I find myself confused with the way people have come to their conclusions. You see, I'd like to say that I approach the issues of the border with logic and reason but I completely lack the experience in the region. Don't get me wrong, as a Latino immigration has been ever-present in my life but I have a starkly different perspective than the locals.

Immigration is, directly or indirectly, a daily topic here in the SW. The issue surrounds the people in this community differently than in any other part of the country and I find there is a common rhetoric that is predominant in my new set of friends. It is a privileged and homogeneous group and they tend to be unaware of how nice they've had it. There is less diversity amongst my new friends and they all tell the same story. To be honest, I haven't come to terms with the difference and I really struggle with the fact that my perspectives are so different. Maybe I feel intellectually alone...Gasp. This isn't meant to be a self reflection, but there are a few stories in this place that keep coming up again and again. Because I spend most of my time at work, its not a surprise that it happens there.

I work for a state entity. It is a government job but it is also Higher Ed. So when coming to work here, I expected a particular level of intellectual rigor. Either I haven't found it or the intellectual process has taken place around a topic that I wasn't prepared or willing to engage. All the political drama aside, I am part of a young diverse team. Most of us are under represented minorities or from other parts of the world. We all have very different approaches and our story has been mainly about building. Our organization has been going through a lot of change and we keep hearing about building this team and getting to know each other more. The truth is, the efforts have been half hearted and mediocre at best. Either way, the issue of team building has been approached in terms of learning each other's styles and preferences so that we can communicate better. This is where metcognition comes into play.

Metacognition is awareness of one's cognitive processes, as well as an ability to develop a plan for achieving a goal and evaluating one's effectiveness of reaching that goal. (http://wik.ed.uiuc.edu/index.php/Metacognitive_knowledge) This seems to be a really important strategy in pedagogy. Conceptually it makes sense too. It is important to teach children to develop an understanding their own learning process so that they are able to formulate plans and evaluate their own progress. I hypothesize that this is what is missing on my team. We are all asked to share our personal styles but it's suggested that we do so so that we can learn how to communicate with others. In reality we should be asked to understand how we learn as individuals so that we can learn how to adapt to meet the team's needs more effectively.

So, to the purpose of my post. Is it important to engage in a metacognitive analysis when building teams? I can tell you all day long that I am visual learner and somehow that makes it ok for me to continue to ask for visually presented information. But is the true role of metcognitive analysis to force team members to self assess in order to find ways to change or "flex" to different styles? How effective has this been in your experience? Your thoughts please.

The "V"

Mark my words:

Housing sales increased in the month of December. Housing Prices are up. Jobs were cut this week, but employers commonly lag in recognizing when the economy is turning positive. I'll also mention that I have gut feeling that this is almost over. Does this mean that the initial bailout worked to free up the credit markets and therefore bank lending? What about Obama's proposed stimulus plan? If the economy is begining to swing positive and Obama manages to get his stimulus plan then that makes the positive slope steaper, right? Your thoughts are always welcome.