Monday, November 25, 2013

Availability Heuristic (TFS part 7)

I am reading Thinking, Fast and Slow, by Daniel Kahneman. In this series I will summarize key parts of the book and supply some comments and reflections on the material.

Part II: Heuristics and Biases
Chapters 12-13


These chapters are all about the availability heuristic: the process of judging the frequency of something by the ease with which instances of the thing come to mind. This is an example of substitution: people substitute the easy question "how do I feel about it?" for the hard question "what do I think about it?"

Examples: the perceived number of divorces among celebrities versus the population at large, frequency of infidelity among politicians versus the population at large, the perceived safety of flying after the news reports a plane crash, people purchasing more insurance AFTER an accident or disaster, and the estimates of the main causes and probability of death being warped by media coverage.

Media coverage could lead to an availability cascade, a "self-sustaining chain of events, which may start from media reports of a relatively minor event and lead up to public panic and large-scale government action" (e.g. Love Canal, Alar scare of 1989, acts of terror).

How do you correct for this bias? Be aware of your own biases; focus on content, not feelings; be on the lookout for bias ("maintain high vigilance").

Some more interesting studies highlighted in these two chapters:

One study showed that the "awareness of your own biases can contribute to peace in marriages, and probably other joint projects." The study asked each member of a couple how much they contributed to keeping the place tidy in percentage terms. The total was larger than 100%. But then the observation that the total was greater than 100% was often enough to make people aware of their own bias and diffuse arguments.

Another study showed that people in power are much more likely to fall victim to this bias. In addition, "merely remembering a time when they had power increases trust in their own intuition."

Kahneman issues one word of caution: the error can go the other direction, too. A study by neuroscientist Antonio Damasio showed that "people who do not display the appropriate emotions before they decide, sometimes because of brain damage, also have an impaired ability to make good decisions. An inability to be guided by a 'healthy fear' of bad consequences is a disastrous flaw."

My Thoughts:

Kahneman's choice of examples of availability cascades is very interesting. Acts of terror can definitely lead to a cascade, and I believe the government has gone too far because of one. That is why I Opt Out. For more modern examples of availability cascades, I would have included the perceived importance of the federal response to Huricane Katrina and the perceived safety of schools in light of recent shootings (At least some media outlets are aware of the latter. USA Today reports: Schools are actually safer now by almost every measure than they were 20 years ago). The inclusion of Love Canal as a cascade is very jarring for me. I was taught this was one of the most disastrous environmental events in US history.

There is a big exposition in these chapters about experts and citizens which I think is clunky and not really very good. But there is one quotation which is good: "Every policy question involves assumptions about human nature, in particular the choices that people may make and the consequences of their choices for themselves and for society." If there was an award for the social science that does the best job of explaining its assumptions about human nature when recommending policy, economics would win the Gold Medal hands down. While being transparent allows all sorts of justified and unjustified mocking of the field, stating assumptions is much better than letting the assumptions go unsaid, which is often the case elsewhere. I get the feeling Kahneman is making hidden assumptions about human nature that he is not being explicit about when he makes policy recommendations in these chapters, but even so, psychology would probably take the silver medal -- but only when it acts like economics.

One last thing: Even though it's nice to think that our country is run by experts (it's their job to spend all day researching and debating on whether a law is good or not), our representatives are people, too. They can exhibit these biases and fail in representing us. One of my biggest pet peeves is when a politician either makes a policy or changes a policy stance FOR THE WHOLE COUNTRY based on ONE PERSONAL EXAMPLE that comes easily to mind. One of the more egregious recent examples of this was when Rob Portman changed his stance on gay marriage because his son came out as gay. All of a sudden, it was okay because of one personal experience.  No cost and benefit analysis; no talk of freedoms or rights or the role of the government in legislating morality. He even acknowledged he never really thought about the issue before. And based on the interview, he clearly he STILL hasn't thought about the issue since. Hopefully, Portman isn't put in charge of making any critical decisions for the country. This is not they way to make policy on ANYTHING (to be fair, Portman only talks about economic issues -- social issues are completely off his radar -- so he may make actual decisions on economic issues; but I wouldn't bet the house on it).

Food for Thought:

1) Kahneman closes these chapters with this observation: "Rational or not, fear is painful and debilitating, and policy makers must endeavor to protect the public from fear, not only from real dangers." Do you agree? Why or why not?

2) Given that our perceptions of the frequency of events comes mainly from memory and the media and that biased presentations in the media distort our perceptions of important frequencies in reality, is there a role for a "Fairness Act" to control the content of the media? What are the pluses and minus of unfettered freedom of the press?

Friday, November 22, 2013

Anchors Away! (TFS part 6)

I am reading Thinking, Fast and Slow, by Daniel Kahneman. In this series I will summarize key parts of the book and supply some comments and reflections on the material.

Part II: Heuristics and Biases
Chapter 11: Anchoring


Anchoring occurs when people consider a particular value for an unknown quantity.

In a study, judges were asked to sentence a shoplifter after rolling (weighted) dice. "On average, those who had rolled a 9 said they would sentence [the shoplifter] to 8 months; those who rolled a 3 said they would sentence her to 5 months; the anchoring effect was 50%."

Why do we anchor? It is partly a System 1 bias of priming. It is partly a System 2 adjustment process. When answering we start at the anchor then adjust in the correct direction until we are "unsure." This leads to reporting the end of a confidence interval and systematic bias in the direction of the anchor.

How do you correct for anchoring in negotiations? Focus your attention and search memory for arguments against the anchor; focus on the minimal offer you think the opponent would accept. This more actively engages System 2 and removes the anchoring bias.

"The main moral of priming research is that our thoughts and our behavior are influenced, much more than we know or want, by the environment of the moment."

My Thoughts:

This is the first chapter where I got the feeling that every solution being presented to correct for System 1 biases can be summarized as "Pay more attention! Engage System 2!"   Which is fine; I agree; but there is only so much attention that can be paid (budget constraint!), and he doesn't talk much about ways to engage that System 2 thinking process.  There are lots of ways people engage System 2. For example, having rubrics, checklists, or standards in place that help you focus on the decision process, being consistent, and conforming to an established precedent.

If a really important decision is coming up, acknowledge you are attention constrained and focus on that problem and your psychological biases related to that problem rather than on whether you are being tricked by the way the supermarket stocks their shelves that day or something.

Rules also can help combat anchoring (and of course other biases), too. One of the reasons we have mandatory sentences is so that there is less discretion and more consistency in judges' sentences.

But rules don't always help; they can introduce another anchor, so there is a tradeoff here. Kahneman asks us to consider the effect of capping damages in personal injury cases to $1 million. This could anchor small cases, pulling up awards that otherwise should be much smaller. It could also prevent a correct award in the rare case more than a million dollars is deserved.

Food for Thought:

1) How free are we to make our own decisions? How much does the environment affect our decision process?

2) What is unbiased opinion?

Wednesday, November 20, 2013

The Law of Small Numbers? (TFS part 5)

I am reading Thinking, Fast and Slow, by Daniel Kahneman. In this series I will summarize key parts of the book and supply some comments and reflections on the material.

Part II: Heuristics and Biases
Chapter 10: The Law of Small Numbers

Summary: People are bad intuitive statisticians.

Detailed Summary:
  • We incorrectly want to apply the Law of Large Numbers to small numbers as well ("the Law of Small Numbers")
  • We have exaggerated faith in small samples. 
  • We pay more attention to message content than information about its reliability (e.g. sample size, variance).
  • We want to see patterns where there is only randomness.
  • There is no "hot hand" in basketball. 
  • Researchers err when choosing sample size for experiments. "[P]sychologists commonly chose samples so small that they exposed themselves to a 50% risk of failing to confirm their true hypothesis!"

My Thoughts on Chapter 10

First off, a quick quibble: "[P]sychologists commonly chose samples so small that they exposed themselves to a 50% risk of failing to confirm their true hypothesis!" should read "...a 50% risk of failing to reject a false null hypothesis!" 

I also have a quibble with how Kahneman explains our tendency to see patterns where there is only randomness. He claims "causal explanations of chance events are inevitably wrong." This is true, but misleading given the point he is trying to make. If you know the process behind the event is random, then yes, a causal explanation is wrong. But what if you don't know the mechanism? What if you are unsure what is going on? I think what he wants to say is that people are good at finding patterns and bad at accepting the absence of patterns ("randomness"), but I'm not sure. Also, when Kahneman mentions "randomness" and "chance" he almost always is assuming independence of events. Sometimes this assumption matters a lot and he seems to assume it in some places that are questionable. 

Sometimes the assumption of independence of events when events are in fact NOT independent can be disastrous. Case in point: the sub-prime mortgage crisis. All these fancy financial instruments were priced correctly assuming default on mortgages were relatively unrelated. In fact, they were highly correlated. Tim Harford has a fantastic basic description of the crisis and how bad assuming statistical independence can be when it's not true. Read it here. You will never think about eggs or mortgages in quite the same way again.

Given that statistical mismodeling can be so disastrous, is it such a bad thing that people search for patterns and take extra precautions after something unexpected or extremely unlikely happens? Even if it turns out that, often, it's just a fluke?

One last thing -- (and I will come back to this experiment several times in discussing these chapters) people may actually be pretty good intuitive statisticians! Results of experiments that ask difficult questions seem very sensitive to how questions are asked in a laboratory setting -- a lesson I thought we learned when we talked about happiness

Food for Thought:

1) At the casino, you are assured the roulette wheel is fair (there is an equal chance of red and black and a small chance of green). You observe the first spin lands on red. Then you see the second spin land on red. Then the third. You keep standing there, and you keep observing spins land on red. How many spins do you need to observe before you become more sure than not the wheel is NOT fair? How would your bets change over time? How do your answers depend on the reputation of the casino? on who told you the roulette wheel was fair?

If you were the owner of the casino, at what point would you close the roulette table and have the mechanism of the wheel inspected?

2) You are assured by a friend a coin is fair. Your friend tosses heads. Then heads again. And again, and again. How may heads in a row do you need to see before you are more sure than not that the coin is not fair or your friend is not tossing the coin "randomly"? How does this depend on how much you trust your friend?

(A quick anecdote: In high school AP Statistics, our teacher asked everyone in the class to flip a coin 10 times and record the flips. More than half the class "surprisingly" recorded all tails or all heads on their flips. Several students claimed we had defeated statistics. We redid the exercise the with requirement that we had to flip the coin high in the air and let it fall on the ground several feet away from us. The teacher monitored the class more closely. Everything came out normal that time.)

3) A hurricane of record strength devastates the coast. People who didn't have insurance now buy insurance. Should they? Why didn't they buy insurance before? Are they irrational? How does your answer depend on your beliefs about the effects of global warming (in particular, your beliefs that over time the average strength of hurricanes will increase)?

Wednesday, November 13, 2013

How Happy Are You? Five. (TFS part 4)

I am reading Thinking, Fast and Slow, by Daniel Kahneman. In this series I will summarize key parts of the book and supply some comments and reflections on the material.

Part I: Two Systems
Chapters 8-9


System 1 is continuously monitoring the world, making assessments, and answering easy questions. System 2 can answer hard questions, but if a satisfactory answer is not found quickly enough, System 1 will try to substitute a "related" easy question to which it already knows the answer.

Detailed Summary:

These two chapters are dense! They are a catalog of studies about what System 1 is good at and what it is not. A lot of these studies are new to me, so my reading slowed down quite a bit here. Let's go through some of those results describing what some of the easy and hard questions are:

System 1 is good at determining whether someone is a friend or a foe. It is also good at determining two traits about people: their dominance and their trustworthiness. How dominant and trustworthy a political candidate looks influences how people vote (even more than "likability").

System 1 is good with averages, but poor at sums. This leads to a "prototype" bias where people become blind to quantities. For example:
In one of many experiments that were prompted by the litigation about the notorious Exxon Valdez oil spill, participants were asked about their willingness to pay for nets to cover oil ponds in which migratory birds often drown. Different groups of participants stated their willingness to pay to save 2000, 20,000, or 200,000 birds.... The average contributions of the three groups were $80, $78, and $88, respectively.... What the participants reacted to, in all three groups, was a prototype -- the awful image of a helpless bird drowning, its feathers soaked in thick oil. The almost complete neglect of quantity in such emotional contexts has been confirmed many times. 

System 1 is good at matching intensities across diverse dimensions. For example, it is good at answering questions like: "If Sam were as tall as he is intelligent, how tall would he be?"

People suffer from the affect heuristic, in which people let their likes, dislikes, and emotional feelings determine one's beliefs about the world and which arguments one finds persuasive.

And one more:

 "How happy are you?" is a difficult question to answer. Kahneman presents the following as "one of the best examples of substitution." The study asked two questions: "How happy are you?" and "How many dates have you gone on in the last month?" When the questions were asked in that order there was zero correlation between the answers. When the order was reversed ("dates?" then "happy?") the correlation was astronomically high. In the second group, people substituted the first question for the second because "dates?" is an easy question to answer that contributes to happiness that is fresh in their mind while "happy?" is a difficult question. Similar results hold when asking about finances or relationships with parents instead of dates.

My Thoughts:

I have been skeptical of the happiness literature that takes surveys about happiness too seriously ever since I saw Justin Wolfers present this paper. "On a scale of 1-7, how happy are you?" just seems like an impossible question to answer, let alone make systematic sense of many peoples' answers to it over time. Happiness depends on a lot of things, and it's unclear whether the feeling is absolute or relative, so what do those numbers mean anyway? My happiness is five. Is that on an absolute scale? Or relative to what my opportunities are right now? Or relative to my neighbor? Or is it an answer to some other question? I only needed introspection to figure out that "happy?" is a difficult question to answer, but I am glad there are actual studies that establish this fact.

This seems like a good time to emphasize the following: utility is not happiness (not even happiness perfectly measured). It is a mistake to think about utility as only squeals of glee. Happiness is one of many emotions. Utility is a more general measure of how well off you are, and a utility function is simply a representation of your preferences over bundles of goods or choices to be made. When you make a decision or consume a bundle of goods you can be better off but less happy. (Higher utility, but lower happiness.) Decisions that lead to lower happiness are not a mistake if they make you better off.

And that's the end! ... of Part I of V. Oh, my. There is a lot to this book! I am really getting my money's worth!

Food for Thought:

1) On a scale of 1-7, how happy are you?

2) Have you ever gone to a movie knowing it didn't have a happy ending? Where you better off because of it? Where you happier because of it?

3) What books, novels, or stories have you read that made you better off but less happy?

4) Have you ever sacrificed your own happiness for others? Where you better off because of that sacrifice? Was society better off?

Monday, November 11, 2013

Don't Grade the Smart Person's Test First! (TFS part 3)

I am reading Thinking, Fast and Slow, by Daniel Kahneman. In this series I will summarize key parts of the book and supply some comments and reflections on the material.

Part I: Two Systems
Chapters 4-7

Priming is important.

Q: How do you make people believe in falsehoods?
A: Frequent repetition, because familiarity is not easily distinguished from truth.

Q: How do you write persuasive messages?
A: Make them legible, use simple language (this increases the perceptions of credibility and intelligence!), make them memorable (e.g. verse, mnemonics), and use easy-to-pronounce sources.

The more often something happens, the more normal it seems.

Confirmation bias, the halo effect, overconfidence, discounting missing evidence (the "what you see is all there is" bias), framing effects, and base rate neglect are all important psychological tendencies people should be aware exist.

My Thoughts:

These systematic biases are very important and prominent in many peoples' behavior. They are the bread and butter of intro psych and becoming aware of them (and learning how to correct for them) was the most significant thing I got out of taking psychology as an undergrad. 

The best anecdote in this section is Kahneman's discovery that he wasn't grading his students' exams fairly. He originally graded his exams in the conventional way -- one student's exam at a time, in order. Here's the excerpt:
I began to suspect that my grading exhibited a halo effect, and that the first question I scored had a disproportionate effect on the overall grade. The mechanism was simple: if I had given a high score to the first essay, I gave the student the benefit of the doubt whenever I encountered a vague or ambiguous statement later on. This seemed reasonable. Surely a student who had done so well on the first essay would not make a foolish mistake on the second one! But there was a serious problem with my way of doing things. If a student had written two essays, one strong and one weak, I would end up with different final grades depending on which essay I read first. I had told the students that the two essays had equal weight, but that was not true: the first one had a much greater impact on the final grade than the second....
So, he tried grading the exams one question at a time rather than one student at a time.
Soon after switching to the new method, I made a disconcerting observation: my confidence in my grading was now much lower than it had been. The reason was that I frequently experienced a discomfort that was new to me. When I was disappointed with a student's second essay and went to the back page of the booklet to enter a poor grade, I occasionally discovered that I had given a top grade to the same student's first essay. I also noticed that I was tempted to reduce the discrepancy by changing the grade that I had not yet written down, and found it hard to follow the simple rule of never yielding to temptation. My grades for the essays of a single student often varied over a considerable range. The lack of coherence left me uncertain and frustrated....
The procedure I adopted to tame the halo effect conforms to a general principle: decorrelate error! ... To derive the most useful information from multiple sources of evidence, you should always try to make these sources independent of each other. 
I am a huge proponent of grading by a rubric in addition to grading one question at a time as much as possible in order to avoid exactly this issue. Grading with a rubric increases consistency and fairness and decorrelates error. Sometimes teachers grade the smart person's test first and using it as a key rather than making their own key with rubric. I disapprove of this (as tempting as it is to do it) -- it saves time, but as Kahneman points out, there is probably a big halo effect here.

Why don't more lecturers grade with a rubric? It takes much longer to grade this way and it is much more tiring because you are actually evaluating all the questions and everyone's responses equally. It also makes assigning the final grades more difficult because the scores are not as obviously "separated" into nice groups. Many lecturers just assign grades based on where the obvious breaks in the scores are without realizing they have created those breaks themselves from biased grading of perceived smart students' tests and perceived dumb students' tests. The halo effect.

Creating the rubric also shows you when discrepancies arise in real time. As I grade a question, sometimes I adjust how "wrong" I think a particular answer is. Then I have to go back and adjust all the previous exams with that answer to bring the scores in line with my new judgement. If I wasn't recording this, I might not catch these changes, unfairly punishing some students simply because of the order in which the exams were graded.

Grading from a rubric also almost completely eliminated from my classes a time honored tradition at the UofC: point mongering. Being able to state exactly why an answer is right or to what degree it is wrong is huge. Students know you are doing your job and respect you and accept the grade they earn. It also makes it much easier to identify (for the students and the teacher) actual grading errors.

Food for Thought:

1) Should the government take advantage of psychological biases and systematic errors people make when creating policy? To what extent should people be educated about them and then left to their own devices, and to what extent should the government create rules (reducing costly decision making) and "nudge" them?  The government is leaning toward more nudges.

2) The Obama campaign utilized academic social scientists more than ever. How are the biases highlighted in these chapters exploited in political campaigns?

3) How much information does it take for you to reverse your first impression of someone? When was the last time  you realized your first impression of someone was not correct? 

Friday, November 8, 2013

Restrictive Church Rules (TFS part 2)

I am reading Thinking, Fast and Slow, by Daniel Kahneman. In this series I will summarize key parts of the book and supply some comments and reflections on the material.

Part I: Two Systems
Chapter 3: The Lazy Controller

Summary: Paying attention is costly. Experiments supporting this show that exerting willpower is costly, self-control is tiring, there is ego depletion, and increased cognitive control is correlated with increased intelligence.

My Thoughts: This line of research is interesting and compelling. It attempts to explain why we sometimes make bad decisions in a very economic way: focus and attention are scarce resources that have to be allocated! 

What about those church rules? Maybe you remember you Sunday School teacher or your parents giving the following advice: don't put yourself in a position to be tempted (the "don't do anything fun" rule). Some denominations go as far as banning lots of activities that are not sinful in and of themselves, but can easily lead to sinful behavior if engaged in "too much" (e.g. drinking, or even dating). People (esp. kids/teens) often laugh -- what, you think we have no self control and don't know what's right and wrong? Well, not necessarily, but your will power gets depleted the more often you are in novel, challenging situations. And then you will do things you regret. It may not even be regret later -- it could be regret as you make the bad choices, and you know on net you are worse off, but you keep going anyway because you have no willpower left to stop.

The church advice is interesting because it acknowledges physical limitations and the psychology of the situation (breakdowns in rationality, some would say; simply acknowledging an often overlooked budget constraint, says me). But it also takes advantage of rational thinking: You (or at least your elders) KNOW if you put yourself in a bad situation your will power could get depleted leading you to make decisions that make you worse off. So don't put yourself in those positions in the first place! It's costly (you don't go to the fun party), but on net you are better off because of it (you didn't crash your parents car after succumbing to peer pressure and binge drinking -- a big plus). The rule is taught, thought about, and learned (maybe even legislated!) in a setting emotionally far removed from the situations in which it needs to be implemented. That's a rule that comes from straight-up, cold-blooded, forward-looking, rational thinking.

And here's a bonus -- once you learn the rule, it's much less costly to implement since you don't have to reason it out every time. But it becomes a heuristic that might lead you astray in other situations -- a System 1 bias.... Around and around we go.

Some smart people cite these ridiculous, "arbitrary" rules as reasons why they leave a particular church (sample: people I have known). But, the last result Kahneman presents in this chapter (control is correlated with intelligence) sheds a little light on why that's a trend (besides the obvious answer that it's not a trend and I am biased in my perception of the trend -- see the next post). Rules taught are tuned for "most people" or people "on average." In this case, it's for people with average will power and control. If smarter people have more will power to allocate, then these rules will be much more restrictive, and less good for them, than average people. It will be optimal for them to break more of these rules in practice and exert more self-control. It's also optimal for them to NOT encourage others who have less self control to break the rule.

Food for Thought:
1. "I don't buy bags of potato chips even though I know everything is fine in moderation. I can't eat just one. In fact, I know I will eat the whole bag in one sitting."

2. Alice: "I'm happy the FDA is about to ban trans-fats. Now I won't be tempted to eat so many doughnuts."
Bob: "I love trans-fatty donuts! They aren't as oily and taste better than other doughnuts. Sometimes I want to eat unhealthily (even though I know I may not live quite as long); let me eat the enjoyable, unhealthy doughnut I want!"

3. Who would be better off in a society where church rules are legally enforced? What if you could costlessly move to a society or city where your preferred church rules are enforced; would you?

4. How many difficult decisions can you make in a day before becoming too tired to make any more?

5. "If it's immoral then it should be illegal."

6. "If it's illegal then it's immoral."

Thinking, Fast and Slow (TFS part 1)

I am reading Thinking, Fast and Slow, by Daniel Kahneman. In this series I will summarize key parts of the book and supply some comments and reflections on the material.

Part I: Two Systems
Introduction, Chapters 1-2

Summary: Kahneman is a psychologist who dabbles in areas of interest to economists (he won the Nobel Prize in Economics a few years back). This book is mostly a summary of the current (as of 2011) state of research in the topics that interest him. Most of it isn't his own research, but is related to it.

Kahneman likes maintaining the following useful fiction: You are two agents, System 1 and System 2. System 1 is the automatic system which relies on intuition and heuristics to make quick, automatic, low cost decisions. System 2 is the effortful system, which is slow and requires conscious, mental effort to reason to a decision. Paying attention really is costly, so shortcuts (using System 1 first) are often necessary to navigate the world. Much of what interests Kahneman are systematic biases in decisions System 1 makes to our detriment (bonus if it's not what an economic actor would do in basic econ models) and what we do when these two systems are in conflict. Kahneman makes it absolutely clear that System 1 and System 2 are useful fictions -- not little people battling in our head -- but makes the statements of what research shows a lot easier to digest by allowing the use of simple, active language.

My thoughts: Economists could do better at explaining why "utility maximizing" is a useful fiction in the same vein. That's not really how individuals make decisions, but it is a useful fiction that makes explaining things a lot simpler. It avoids having "as if", aggregation, or "on average" statements everywhere and lets us focus on what's really important.

To me the biggest difference between economics and psychology is the level of focus. Psychology is really interested in how YOU, PERSONALLY make decisions. How the actual individual "works." And at it's most basic level, Thinking, Fast and Slow is a self-help book for the academically minded. Economists care much more about market level, group behavior, and allocations of scarce stuff (who gets what and why). In order to talk about these things coherently, economists need to over-simplify the individual's decision process and motives because it's one level down from what we actually care about. Of course, there is no hard line drawn, and there is much overlap between the fields; usually as a result of either economists or psychologists taking themselves too seriously. Hence the existence of fields like behavioral economics.

Saturday, November 2, 2013

How Science Goes Wrong

The Economist did a nice cover piece last week on How Science Goes Wrong. It is worth reading in it's entirety. Here's the leader and here's the cover story.

A major problem in science is the lack of incentives to replicate others' research. This is the thesis of the piece.

A problem that they don't mention directly, but alludes to, is that bad studies keep getting cited as truth even though they've been retracted, debunked, or otherwise discredited. EVEN IN THEIR OWN FIELD. Angrist and Krueger (1991) I AM LOOKING AT YOU. You would think that once someone showed their results could be qualitatively replicated using random data the paper (and technique of using quarter of birth as a valid instrument in the setting) would be relegated to the dustbin. You would be wrong. David, I feel your pain.

There are numerous examples underlying this thesis cited in the Economist, which should be concerning to anyone who values scientific research and its (subsidized) place in the modern economy.
A rule of thumb among biotechnology venture-capitalists is that half of published research cannot be replicated. Even that may be optimistic. Last year researchers at one biotech firm, Amgen, found they could reproduce just six of 53 “landmark” studies in cancer research. Earlier, a group at Bayer, a drug company, managed to repeat just a quarter of 67 similarly important papers. A leading computer scientist frets that three-quarters of papers in his subfield are bunk. In 2000-10 roughly 80,000 patients took part in clinical trials based on research that was later retracted because of mistakes or improprieties.
... nine separate experiments had not managed to reproduce the results of a famous study from 1998 purporting to show that thinking about a professor before taking an intelligence test leads to a higher score than imagining a football hooligan. 
Why the downturn in verification and replication?
The obligation to "publish or perish" has come to rule over academic life.... Careerism also encourages exaggeration and the cherry-picking of results.
The cover story does a good job of going through the statistics of how most published research (even when done right and in good faith) could be wrong. And it's not all in good faith. Because replications and reviews are not valued highly, and moderate or negative results are not usually interesting. It's important not only to do good work, but capture the interest of your fellow scientists when publishing. This leads to incentives to distort or exaggerate your work. If it's controversial, even better -- you'll get cited. Truth matters less than sparking debate. Journals publish successes, not failures.

And those success are rarely checked as carefully as they need to be by peer review once papers make it to the journal stage. Editors and reviewers need to spend time writing their own papers, not reading someone else's! Usually, the paper reviews get dumped to grad students as busy work, further reducing the quality of the review.
When a prominent medical journal ran research past other experts in the field, it found that most of the reviewers failed to spot mistakes it had deliberately inserted into papers, even after being told they were being tested.
... John Bohannon, a biologist at Harvard, recently submitted a pseudonymous paper on the effects of a chemical derived from lichen on cancer cells to 304 journals describing themselves as using peer review. An unusual move; but it was an unusual paper, concocted wholesale and stuffed with clangers in study design, analysis and interpretation of results. Receiving this dog’s dinner from a fictitious researcher at a made up university, 157 of the journals accepted it for publication.
 How to Fix the Problem?

One thing the Economist notes, which I fully agree with, is that applied scientists need to understand statistics better. Also, scientists should actually follow the scientific method. Unfortunately, something like this usually happens instead -- if you're lucky! Here's more of their recommendation:
Journals should allocate space for “uninteresting” work, and grant-givers should set aside money to pay for it. Peer review should be tightened—or perhaps dispensed with altogether, in favour of post-publication evaluation in the form of appended comments. That system has worked well in recent years in physics and mathematics. Lastly, policymakers should ensure that institutions using public money also respect the rules.

Why Does this all Matter, Anyway?
The governments of the OECD, a club of mostly rich countries, spent $59 billion on biomedical research in 2012, nearly double the figure in 2000. One of the justifications for this is that basic-science results provided by governments form the basis for private drug-development work. If companies cannot rely on academic research, that reasoning breaks down. When an official at America’s National Institutes of Health (NIH) reckons, despairingly, that researchers would find it hard to reproduce at least three-quarters of all published biomedical findings, the public part of the process seems to have failed. [emphasis added.]
And that's the problem in a nutshell. Basic research is funded publicly because it is a public good -- so every company ever doesn't have to reinvent the wheel -- that's wasteful. But if that basic research is mostly wrong, and every company that wants to use the result has to independently verify it, why are we funding basic research publicly, anyway?

Science has to be better at establishing actual Truth. A threat of retracting funding should be the stick that whips science into shape.