Sunday, August 22, 2010

Failure is an orphan

An eye opener! That's the least to say about this inspiring article for Prof. Stan Szpakowicz in Computational Linguistics journal.

According to the article, one project out of a hundred produces results that justify the investment. However, we tend to count ourselves among the 1%. That's because we need to show we outperformed others in order to publish more in order to have a thriving career in research. That's a compelling reason for us NOT to invest more time on something that produced negative results.

But WAIT. "Suppose you have set up an experiment carefully and in good faith, but still it comes up short. That’s not a positive outcome. Maybe your intuition has let you down. Maybe this cannot work. Wait, maybe you can prove that it cannot work?" THAT would be a useful outcome. But you need to get it published.

The problem boils down to peer reviews which aggressively reject failures. "A forthright admission of the inferiority of one’s results—despite the integrity or novelty
of the work—is a kiss of death: no publication. There must be improvement ... conformance to reviewers’ expectations is an asset. Indeed, we write so they are likely to accept". But we're talking about ourselves. WE are the reviewers. If we -the authors- insist on striving for success and run away from every negative result, then we -the reviewers- will check papers for signs of success.

An experiment carefully thought out, a systematic procedure, an honest evaluation—these are the ingredients of good science. It is not mandatory for the results to be positive, though it certainly lifts one up if they are. In areas where empirical methods dominate (e.g. computational linguistics) people try things which fail at the experimental stage. This may be due to lack of rigor, but often there are deeper, unexpected, and intriguing reasons. We can learn a lot if we analyze scientifically why an intuitive and plausible experiment did not work. Then again, to know what leads to dead ends in research surely can warn others off paths which take us nowhere. Simply put, a negative result can be a useful lesson.

Q: Philosophy aside, I want to publish a serious, worthwhile negative result I've obtained. Where to go today?

Here's a non-comprehensive list in different disciplines


  1. really amazing topic..... :)

    *but could u explain to me more what does negative results in research mean?

  2. @Muhammed.. Suppose you had an idea to improve the quality of web search (i.e. get search results which are more relevant to user's query). When you implemented, experimented it, and evaluated the results, you discovered that your idea made the results worse. This is what negative results mean.

  3. That is the different story you have created for poor people. Bad writing has bad reviews and this professor has named it as orphan. Failure cannot last longer son never lose your confidence.

  4. Nice post, As if the all inclusive community will be uneducated, by then they would not have the ability to find for the new things as they don't have the direction and this site about supporting things on the planet.

  5. Entirely different concept and different way of presenting. It clearly defined the actual situation how is person must deal, but we never follow such things. After reading this article definitely somethings will change

  6. I do not like the word failure while people use it to mock others but it is true that this failure could be his strength once. explore more to know about the quality of writing.

  7. I must say that failure is not as saddest part of life as someone make it. Only failure will teach you about how to succeed. Failure will tempt you to make yourself perfect and do hard work to become a successful person. I'm recently working on literature review writing service learn more and enjoy your visit