A friend asked me how to approach data analysis and how to decide on the data presentation.
From my experiences and reading, Dan Roam in his book, The Back of the Napkin had explained the process very well. Below is the excerpt from his book with my two-cents' worth thrown in.
Data analysis is both an art and a science. It is increasingly important due to the advent of Internet. Today, we are flooded with information. The sad thing though is that, we are inundated with details at the expense of the big picture. While in the past data was power, today making sense of data is more powerful. In essence, we need to be able to separate the wheat from the chaff.
How to Look Better - 4 Rules to Live By
1. Collect everything we can look at, the more the merrier (at least at first)
2. Have a place where we can lay down everything and ready look at it, side by side
3. Always define a basic coordinate system to give us clear orientation and position
4. Find ways to cut ruthlessly from everything our eyes bring in - we need to practise "visual triage".
Remember: When data is packed in individual tiles and records, it is impossible to look at the big picture - but getting everything out in the open makes otherwise invisible connections visible.
General Rules of the Thumb
1. It's the data that matters, let it show.
Many people find numbers boring, so we jazz up our charts with visual bells and whistles hoping to make pictures more interesting
That's only the style. In my opinion, substance is the most important ingredient. Let us face it, insightful data is exciting! If what we show resonates with our audience (either it shows exactly what they hope for or it scares the daylights out of them), they won't fall asleep.
2. Always show the fewest possible pictures to make a point.
Less is more. Pick the simplest model to make your point. I prefer charts to tables as the former provides hooks to catch our visual memory. In the case of tables, if we cannot remember the precise numbers, we would not have a larger context to fall back on. On the other hand, with pre-cognitive quantity charts, it enables our eyes to read immediately, compare and viscerally recall long after we have forgotten the numbers.
However if the differences among the slices are critical and yet too small to be visually detectable, one is better off with the non-pictorial table.
Friday, November 25, 2011
Wednesday, November 23, 2011
Part 1, Where good ideas come from, Steven Johnson
I started reading "Where Good Ideas Come From" by Steven Johnson a few nights ago. I was captivated. This is a fantastic book.
In his book, Steven Johnson takes us on a fascinating tour starting with Charles Darwin's long, slow hunch back in the 1800s, the "liquid networks" of London's coffee houses to today's high-velocity web.
A totally enchanting read so far, where I learnt about the linkage between the rich living organisms in coral reef with densely populated cities. While people often credit their ideaVs to individual "Eureka" moments, Steven Johnson shows that it is through exchange of ideas that innovation is born.
He has cleverly linked many diverse fields such as chemistry - carbon, its four valence bonds, and its high propensity to form new combinations with other atoms, the origins of all living things, the neurons in our brain and drawing comparison with the linkages in the World Wide Web. Read: the linkages among neurons in our brain is much, much denser than that in the World Wide Web. Homo sapiens (that's you and me) are simply amazing.
Environment is important, Johnson argued, YouTube would be successful had it not for the internet platform and technological advancement that enabled users to upload their videos on the web and share with others seamlessly. This is what he termed as "adjacent possibles".
I am now only about one-fifth into the book. So you bet there will be more sharing as I read on.
In his book, Steven Johnson takes us on a fascinating tour starting with Charles Darwin's long, slow hunch back in the 1800s, the "liquid networks" of London's coffee houses to today's high-velocity web.
A totally enchanting read so far, where I learnt about the linkage between the rich living organisms in coral reef with densely populated cities. While people often credit their ideaVs to individual "Eureka" moments, Steven Johnson shows that it is through exchange of ideas that innovation is born.
He has cleverly linked many diverse fields such as chemistry - carbon, its four valence bonds, and its high propensity to form new combinations with other atoms, the origins of all living things, the neurons in our brain and drawing comparison with the linkages in the World Wide Web. Read: the linkages among neurons in our brain is much, much denser than that in the World Wide Web. Homo sapiens (that's you and me) are simply amazing.
Environment is important, Johnson argued, YouTube would be successful had it not for the internet platform and technological advancement that enabled users to upload their videos on the web and share with others seamlessly. This is what he termed as "adjacent possibles".
I am now only about one-fifth into the book. So you bet there will be more sharing as I read on.
Wednesday, November 9, 2011
Eightfold Path
Below is a brief summary of what I have read thus far from the book on Eightfold path on policy analysis.
The eighfold path is a thinking guide for one to go through the intricacies of policy analysis.
Step 1: Define the Problem
Appropriate problem definition is important. It is useful to use deficit-and-excess approach, such as too many, too few. E.g. 1. There are "too many" patients in the hospital. 2. There are "too many" crimes in the country.
Other terms such as "growing too fast/ slow" are also useful to frame the issue which is not a problem at the current moment but could become so in the future.
One should be mindful not to prescribe a solution implicitly in the problem definition. E.g. the earlier statement "There are too many patients in the hospital" if rephrase into "There are too few beds in the hospital", the implicit solution will be to have more beds in the hospital. This will limit the scope of one's thinking.
Step 2: Assemble the Evidence
Data are facts, or representation of facts.
Information is data with meaning. Information helps us to make sense of the world, by allowing us to categorise information into different groups.
Evidence is information that affects the existing belief.
Before one embarks on policy analysis, it would be a tremendous plus to think on the type of evidences that one needs before one leaps.
Step 3: Construct the Alternatives
Often, we need a base case - status quo and assuming current trends persist.
Another thing to take note is that alternatives may not be mutually exclusive. Sometimes, they may co-exist.
Step 4: Select the Criteria
Resources are limited. So as policy analysts, we need to be mindful of the evaluative criteria. E.g. when we talk about taxation, there are at least 2 key criteria - the rich pays more than the poor; the tax rate cannot be so high that pushes the rich out of the country.
Remember evaluative criteria are not used to judge the alternatives, or at least not directly. They are to be applied to the projected outcomes. It is easy to get confused about this point because of a commonsense way of speaking: "Alternative A looks to be the best - therefore let's proceed with it." But this way of speaking ignores a very important step. The complete formulation is "Alternative A will very probably lead to Outcome O_A which we judge to be the best of the possible outcomes; therefore, we judge Alternative A to be the best."
Applying criteria to the evaluation of outcomes and not alternatives makes it possible to remember that we might like Outcome O_A a great deal. However, if we lacked sufficient confidence that A would actually lead to O_A, we may decide not to choose Alternative A after all. With that judgement on the table, it would be possible to look for other alternatives with a greater likelihood of producing O_A.
Step 5: Project the Outcomes
This step tells us that it is important to compare "outcome" of the alternative and not the alternative.
Assuming for the moment that benefits are uncertain while costs are not, ask yourself 2 questions: (1) Given what I know for sure about the costs of this alternative, what is the minimum help we need to get from Condition X to ensure adequately offsetting benefits? and (2) How reasonable is it to believe that Condition X will actually produce that minimum?
Implementation scenarios should be written in the future perfect tense. This encourages concreteness, which is a helpful stimulus to the imagination.
Step 6: Confront the Trade-offs
Economics tells us that trade-offs occur at the margin. In policy analysis, we ask this question "If we spend an extra X dollars for an extra unit of Service Y, we can get an extra Z units of good outcome."
This puts the decision maker in the position to answer the question "Does society (or you) value Z more or less than X?" and then to follow the obvious implication of the answer.
Step 7: Decide
Put yourself in the shoes of the policy maker and ask yourself which policy alternatives would you choose.
This is also known as the twenty-dollar bill test which is a joke about economist - there are 2 friends walking on the street. They saw a twenty-dollar bill on the floor. The economist resisted his urge to pick up the bill as he asked himself when no one has done it before him.
Step 8: Tell the Story
This is on public communication and we can apply the Grandma Bessie test. In other words, we need to be able to explain the policy to the man on the street.
As it is a thinking guide, in the final product, we do not write our thinking process. Rather the essence of it, just like in the case of report writing. Just because one has gone through the thinking process, it does not mean that one has to beat the drum to tell the reader about it. The reader will be more appreciative if one could distill the important key takeaways from the paper.
The eighfold path is a thinking guide for one to go through the intricacies of policy analysis.
Step 1: Define the Problem
Appropriate problem definition is important. It is useful to use deficit-and-excess approach, such as too many, too few. E.g. 1. There are "too many" patients in the hospital. 2. There are "too many" crimes in the country.
Other terms such as "growing too fast/ slow" are also useful to frame the issue which is not a problem at the current moment but could become so in the future.
One should be mindful not to prescribe a solution implicitly in the problem definition. E.g. the earlier statement "There are too many patients in the hospital" if rephrase into "There are too few beds in the hospital", the implicit solution will be to have more beds in the hospital. This will limit the scope of one's thinking.
Step 2: Assemble the Evidence
Data are facts, or representation of facts.
Information is data with meaning. Information helps us to make sense of the world, by allowing us to categorise information into different groups.
Evidence is information that affects the existing belief.
Before one embarks on policy analysis, it would be a tremendous plus to think on the type of evidences that one needs before one leaps.
Step 3: Construct the Alternatives
Often, we need a base case - status quo and assuming current trends persist.
Another thing to take note is that alternatives may not be mutually exclusive. Sometimes, they may co-exist.
Step 4: Select the Criteria
Resources are limited. So as policy analysts, we need to be mindful of the evaluative criteria. E.g. when we talk about taxation, there are at least 2 key criteria - the rich pays more than the poor; the tax rate cannot be so high that pushes the rich out of the country.
Remember evaluative criteria are not used to judge the alternatives, or at least not directly. They are to be applied to the projected outcomes. It is easy to get confused about this point because of a commonsense way of speaking: "Alternative A looks to be the best - therefore let's proceed with it." But this way of speaking ignores a very important step. The complete formulation is "Alternative A will very probably lead to Outcome O_A which we judge to be the best of the possible outcomes; therefore, we judge Alternative A to be the best."
Applying criteria to the evaluation of outcomes and not alternatives makes it possible to remember that we might like Outcome O_A a great deal. However, if we lacked sufficient confidence that A would actually lead to O_A, we may decide not to choose Alternative A after all. With that judgement on the table, it would be possible to look for other alternatives with a greater likelihood of producing O_A.
Step 5: Project the Outcomes
This step tells us that it is important to compare "outcome" of the alternative and not the alternative.
Assuming for the moment that benefits are uncertain while costs are not, ask yourself 2 questions: (1) Given what I know for sure about the costs of this alternative, what is the minimum help we need to get from Condition X to ensure adequately offsetting benefits? and (2) How reasonable is it to believe that Condition X will actually produce that minimum?
Implementation scenarios should be written in the future perfect tense. This encourages concreteness, which is a helpful stimulus to the imagination.
Step 6: Confront the Trade-offs
Economics tells us that trade-offs occur at the margin. In policy analysis, we ask this question "If we spend an extra X dollars for an extra unit of Service Y, we can get an extra Z units of good outcome."
This puts the decision maker in the position to answer the question "Does society (or you) value Z more or less than X?" and then to follow the obvious implication of the answer.
Step 7: Decide
Put yourself in the shoes of the policy maker and ask yourself which policy alternatives would you choose.
This is also known as the twenty-dollar bill test which is a joke about economist - there are 2 friends walking on the street. They saw a twenty-dollar bill on the floor. The economist resisted his urge to pick up the bill as he asked himself when no one has done it before him.
Step 8: Tell the Story
This is on public communication and we can apply the Grandma Bessie test. In other words, we need to be able to explain the policy to the man on the street.
As it is a thinking guide, in the final product, we do not write our thinking process. Rather the essence of it, just like in the case of report writing. Just because one has gone through the thinking process, it does not mean that one has to beat the drum to tell the reader about it. The reader will be more appreciative if one could distill the important key takeaways from the paper.
Subscribe to:
Posts (Atom)