Sunday, May 17, 2009

Great Essays

I browse the Internet a lot. I browse at office (when I should actually be working). I go home in the evening, have my dinner, and browse till late in the night. My browsing is not limited to a particular topic in any way. I browse on topics that interest me either at that point of time, or that have always interested me.

During such browsing periods, I sometimes come across articles that have me going, "Wow..!!" or, "Amazing...!! What a brilliant piece of writing!!" After my previous post, I was thinking on what to post next, when I realized I could have a post that linked to these essays - and also explain to readers why these essays were good enough to feature here.

As I listed the essays in my mind, I realized for an essay to be called a "great essay", I would have to set a bar. This is the bar I have set:

The article or essay must have
1) changed my thinking in some fundamental way (or),
2) must have opened up my mind to a different thought process, (or)
3) must have brought out what was in my mind all the while, but whose existence I had not 'realized', or was afraid to acknowledge.

With that bar set, I present before you these essays - essays that I term, "Great". A note here before we proceed: some of the essays are software development essays, so if you are not in that field or if you are not interested in software development, then you may not understand what makes those essays Great Essays.

Here we go!!


Usability for programmers (link)

Joel Spolsky's essay comes up first, simply because it was the first Great Essay that I read. This essay is software development related, but users of software may also relate to it; so I would actually recommend any software user to read it. Yes, at 9 pages, its a pretty long essay , but its worth reading. I say this essay is worth it even for non-software guys because the essay mainly deals with how to make an application easy to use. This means that the essay shows you examples of bad UI design that confuses users - something that non-software guys would readily empathize with. That is the reason why I recommend this essay to everybody who uses and creates software.

At the time I read this essay, I was working in a particular team in my previous company, and we used to share interesting links among ourselves. This team included the VP of the company I was working for. I was initially reluctant to share this link, as I knew my team-mates would shout me down for sending a 9-page essay, but after much dithering, I took the risk and sent it anyway. Next morning, when I checked my mail, I found a mail from the VP, that went like this:

"This is brilliant!! Must read for all!!!"

All day long, the VP looked at me and smiled with the realization that a bit had flipped over in his brain. I don't think he ever smiled at any of us like he did that day.

This essay was the one that introduced me to Joel Spolsky and I have learnt a lot from his essays ever since. From his website, I have got to know the Business on Software forum (where I got to know various aspects of business, as well as some different views of India and what people in other lands think of India) as well as StackOverflow.com (where I have come across some great questions, answers and links). It has been a great association with Joel so far.




Milgram experiment (link)
I generally read a lot of articles on Wikipedia. However, I am not sure how I stumbled upon this one. Guess it must have been while researching the various topics mentioned in Michael Crichton's novel, Sphere. In the novel, the main character performed experiments on the lines done by Asch and Milgram. Wanting to know who Asch and Milgram were and what was the significance of their experiments in the first place, I browsed Wikipedia and stumbled upon this one.

The article basically talks about an experiment conducted in 1961. We all know the horrors of the Holocaust during WW II. Yale University psychologist Stanley Milgram wanted to answer the question of whether people who commit such horrific, disturbing acts as those performed during the Holocaust were simply "following orders from above". Milgram devised this experiment to know whether ordinary people would commit acts against their conscience when prodded on to do so by a person in authority.

As I read through the article, I was nodding my head at every point, and when I finished reading it, I realized the impact of the experiment - I could very well be one of the 'teachers'!! Read the article to find out what I mean...!!

Staying on the topic of psychology experiments, do not miss the conformity experiments by Solomon Asch.



You and your research (link)

I have linked to the original article, but I actually read the article on Paul Graham's website.

This is actually a speech given by Richard Hamming at Bell Communications Research. Hamming was a mathematician involved in building telecommunications and computer systems. Hamming was brought in to Bell Labs so that he could program computers that would calculate the solutions to the equations that scientists working on the atomic bomb brought in.

Somehow Hamming hated this - he felt this was menial work, compared to the actual work of developing the bomb. He wanted to know why he was put in to do this work, when others were obviously working on the more exciting parts. He started to analyze what were the differences between him and the others; when someone achieved something great, he asked that person what made him do it, why, and wondered why others did not do the same. This led to some profound observations that are listed in this awesome essay.

Basically, the speech is about what one should do if one wants to do great research. By now if you are a person not associated with research in any way, you are probably thinking, "I am not associated with any research fields, so this essay is not for me". Actually, this isn't the case as Hamming himself says:

I will talk mainly about science because that is what I have studied. But so far as I know, and I've been told by others, much of what I say applies to many fields. Outstanding work is characterized very much the same way in most fields, but I will confine myself to science.


Yes, you might not understand few things in the essay if you are not part of the science & research fields or do not keep track of those fields, but I would still recommend this essay to you because its filled with great information and quotes on how to become great in your particular field - most of what Hamming talks about surely apply to your field and to you. Here's one example: most people believe you need luck to acheive greatness. There is also the belief that you need "lots of brains". Hamming takes the bull by the horns here:


...I find that the major objection is that people think great science is done by luck. It's all a matter of luck. Well, consider Einstein. Note how many different things he did that were good. Was it all luck? Wasn't it a little too repetitive? Consider Shannon. He didn't do just information theory. Several years before, he did some other good things and some which are still locked up in the security of cryptography. He did many good things.

You see again and again, that it is more than one thing from a good person. Once in a while a person does only one thing in his whole life, and we'll talk about that later, but a lot of times there is repetition. I claim that luck will not cover everything. And I will cite Pasteur who said, ``Luck favors the prepared mind.'' And I think that says it the way I believe it. There is indeed an element of luck, and no, there isn't. The prepared mind sooner or later finds something important and does it. So yes, it is luck. The particular thing you do is luck, but that you do something is not.

For example, when I came to Bell Labs, I shared an office for a while with Shannon. At the same time he was doing information theory, I was doing coding theory. It is suspicious that the two of us did it at the same place and at the same time - it was in the atmosphere. And you can say, "Yes, it was luck." On the other hand you can say, "But why of all the people in Bell Labs then were those the two who did it?" Yes, it is partly luck, and partly it is the prepared mind; but 'partly' is the other thing I'm going to talk about. So, although I'll come back several more times to luck, I want to dispose of this matter of luck as being the sole criterion whether you do great work or not. I claim you have some, but not total, control over it. And I will quote, finally, Newton on the matter. Newton said, "If others would think as hard as I did, then they would get similar results."


There it is - Luck favours the prepared mind and If others would think as hard as I did, then they would get similar results!! Who could have put it better than that? Yes, there is a certain amount of luck, but if you do your work diligently, always striving to improve yourself, your knowledge and the quality of your work, luck ceases to be a significant factor very soon.

There are other great quotes here. While on the topic of drive, Hamming says...


One day about three or four years after I joined, I discovered that John Tukey was slightly younger than I was. John was a genius and I clearly was not. Well I went storming into Bode's office and said, "How can anybody my age know as much as John Tukey does?" He leaned back in his chair, put his hands behind his head, grinned slightly, and said, "You would be surprised Hamming, how much you would know if you worked as hard as he did that many years." I simply slunk out of the office!

What Bode was saying was this: "Knowledge and productivity are like compound interest." Given two people of approximately the same ability and one person who works ten percent more than the other, the latter will more than twice outproduce the former. The more you know, the more you learn; the more you learn, the more you can do; the more you can do, the more the opportunity - it is very much like compound interest. I don't want to give you a rate, but it is a very high rate.


There are other great nuggets in this speech, which I shall leave it to you, the reader to enjoy. The gist of the essay is that you must ask yourself three questions to be great:

1) What are the important problems of your field?,
2) What important problems are you working on?, and
3) Why not?

Sadly, I sent this article to a friend of mine. This speech is a pretty long speech, and my friend immediately dismissed it, saying it was too long. In the process, she missed a great chance to improve her life by standing on Hamming's shoulders. Do not make that mistake - go on, read it!!



Good and bad procrastination (link)

I do procrastinate a lot and always felt guilty about it. All my life, I have been reprimanded by my parents for this. I thought I was the problem and tried a lot to change myself - I slightly did improve once I bought my cell phone and started storing my tasks in it. But the guilt remained, I guess. However, once I read the article's title, I was shocked - can there be any 'good' procrastination? But it turns out there is, as Paul Graham explains wonderfully.

Most people who write about procrastination write about how to cure it. But this is, strictly speaking, impossible. There are an infinite number of things you could be doing. No matter what you work on, you're not working on everything else. So the question is not how to avoid procrastination, but how to procrastinate well.

There are three variants of procrastination, depending on what you do instead of working on something: you could work on (a) nothing, (b) something less important, or (c) something more important.

That last type, I'd argue, is good procrastination.


Paul Graham says that procrastination isn't really bad - yes, you are putting off tasks, but as long as you are putting off unimportant tasks to work on more important ones, who cares? And that's the big difference here - when you procrastinate, there are two things happening: a) you are working on something, while b) the one you are supposed to work on lies unattended. What Paul Graham says is to ensure that the task you left unattended is less important than the task you are doing. When you procrastinate, ask yourself whether what you are doing now will give you much more benefits than the one you were supposed to do. If the answer is yes, then it is good procrastination - something you stand to benefit from, and hence don't have to feel guilty about.

Paul Graham also differentiates between errands and tasks. Any person has a never-ending stream of to-dos on his plate, but each one should learn to differentiate which of these to-dos are errands and which are tasks. (Here 'errands' are stuff that don't provide much benefits in the long term, while 'tasks' are stuff that provide benefits in the long term). Errands are small stuff, and hence its OK if we skip doing them. But how do we determine which is small stuff and which isn't? Here too Paul provides the answer...

What's "small stuff?" Roughly, work that has zero chance of being mentioned in your obituary. It's hard to say at the time what will turn out to be your best work (will it be your magnum opus on Sumerian temple architecture, or the detective thriller you wrote under a pseudonym?), but there's a whole class of tasks you can safely rule out: shaving, doing your laundry, cleaning the house, writing thank-you notes—anything that might be called an errand.

Good procrastination is avoiding errands to do real work.


Aha... tell me, which of the items in your to-do list right now would be mentioned in your obituary? Imagine your funeral - are people going to stand around and say, "He was such a good man - he used to shave regularly!!" No, they aren't - they are rather gonna list your achievements in life. For that, you should have done some real work - something they can say proudly at your funeral.

At this point, you are probably saying, "But not all errands can be ignored!! If I don't shave for two or three months, nobody around me is gonna like me!!" True, there are some errands that just cannot be ignored - these errands must be taken care of, but it may be ok to delay them by a day or two. Of course, there are other errands that can be safely skipped.

Note that there are other errands that simply cannot be skipped - for example, Paul describes an "absent-minded professor" who is so involved in solving a problem that he forgets to shave, eat or even look where he is going. Whenever, I think of such professors or scientists, who are so engaged in a problem, I also think of ignored wives. (I don't know why - hope I don't come across as sexist). In this case, an ideal scenario would be for the scientist to spend time with his wife, while still working dedicatedly at his research. Such kinds of errands require a balancing act, and one that comes with experience and understanding of the work you do and the persons you interact with. In my opinion, it would be better if we treated this as a task, rather than as an errand, since this does have ramifications in the long term.

Reading this article gave me a relief - it told me that there was nothing really wrong with me - procrastination wasn't really bad, as long as you learnt to use it to your advantage.




The Doomslayer (link)

I came upon this essay after reading Michael Crichton's novel, State of Fear. State of Fear basically says that in many cases, public concerns such as global warming are not really major concerns - rather they are fears generated by governments/institutions interested in keeping people under control and making them behave within certain limits. Thus, Crichton goes on to say, global warming could possibly be yet another fear and hence doesn't really deserve the attention that it currently gets.

This is disturbing - and I went on to the Internet to verify some facts stated in the novel and that's how I came across this article. This article deals with the same lesson that State of Fear tries to put forward - that certain issues are not really issues worthy of attention; they are rather, unfounded fears.

The article talks about how Julian Simon and Paul Ehrlich came to lay a bet between themselves. Julian Simon was a professor of business administration while Paul Ehrlich was an entemologist. In the 1970s, Ehrlich became popular by his work, The Population Bomb, that warned people that "famines of unbelievable proportions" would strike by 1975, "hundreds of millions of people" would starve to death in the 1970s and 80s, and that the world was "entering a genuine age of scarcity". The basis for these predictions was that the growth in the population is not matched by the growth in natural resources, which would evidently lead to an increase in prices, and making these resources inaccessible to most of the population.

Simon did not believe so, and very soon, the two laid a public bet - Simon asked Ehrlich to select any five raw materials, and select any date more than a year away. Simon believed that the prices on the future date would be less than the prices on the day of the bet, thus proving that raw materials would be more accessible and would not be affected by the burgeoning population.

Ehrlich took up the bet - he selected chromium, copper, nickel, tin and tungsten, and on paper bought $200 worth of each metal using the prices of Sep 10, 1980 as the index. The future date selected was Sep 10, 1990, a ten year period. During that period, the prices fell, with the result being that on Sep 10, 1990, Ehrlich sent Simon a cheque for the difference in price.

How could that be? As people use more and more resources, natural resources should
come down, right? After all, people can produce more of their kind, but natural resources cannot reproduce, so at one point, we should be hitting a limit, shouldn't we? In fact, the article asks this very same question...


After all, people are fruitful and they multiply but the stores of raw materials in the earth's crust certainly don't, so how can it be possible that, as the world's population doubles, the price of raw materials is cut in half?


And there is an explanation, as the article itself states...


It makes no sense. Yet it has happened. So there must be an explanation.

And there is: resources, for the most part, don't grow on trees. People produce them, they create them, whether it be food, factories, machines, new technologies, or stockpiles of mined, refined, and purified raw materials.

"Resources come out of people's minds more than out of the ground or air," says Simon. "Minds matter economically as much as or more than hands or mouths. Human beings create more than they use, on average. It had to be so, or we would be an extinct species."

The defect of the Malthusian models, superficially plausible but invariably wrong, is that they leave the human mind out of the equation. "These models simply do not comprehend key elements of people - the imaginative and creative."


On the Wikipedia article for this bet, a quote from Simon says that he concedes that increased usage of such resources does inevitably lead to increased prices - but they are only in the short term, since in the long term, people generally find out solutions to the problem.


More people, and increased income, cause resources to become more scarce in the short run. Heightened scarcity causes prices to rise. The higher prices present opportunity, and prompt inventors and entrepreneurs to search for solutions. Many fail in the search, at cost to themselves. But in a free society, solutions are eventually found. And in the long run the new developments leave us better off than if the problems had not arisen. That is, prices eventually become lower than before the increased scarcity occurred.


In case you are thinking that I have changed my views on global warming and other such concerns, no I haven't. But one thing that was new to me was the fact that I had never considered that problems such as the one put forward by Ehrlich were solvable - to me, once population began to increase, it was like it was the end of the road. I guess I am not the only one like this. It never crossed my brain that man's ingenuity would solve these problems.




IT Survivors - Staying alive in a software job (link)

In a particular company in which I worked, the project in which I was in rapidly turned from a sedate one into one where we were spending long hours at office. I used to come into office at 10am, only to leave at 2am or 3am the next day. The worst part was that the deadline for the project was initially stated as January, but later on extended onto February, then it became March, then May and so on... and during each of these months, we worked from 10am to 3am. I felt frustrated at the long hours, but the work was good, since this was my first experience in a software services co. as well as a big company, and I was being exposed to new methods and styles of working.

But family life suffered heavily, with my Mom one day picking up a newspaper article headlined, "Separate your work life from your home life", and pointing it at me, screamed, "This is for you!!" As such, I was hoping for a quick end to the project. One day, an opportunity to leave for another project presented itself, and I left to join the new project. But my mind didn't rest - it kept working on why there was a need for the horrid nature of my work hours during that period - and again, turned to the Internet for answers. I searched for answers as to why people put in long hours in software projects and I very soon started searching for why people manage software the way they did in my team, and one fine day, stumbled upon this article that aptly catches the reasons from a very high level.

First off, if you have read the breathless news reports that appeared when I was in school (and possibly still appear) about the IT field, you would have got the impression that nothing about the Indian IT industry can ever go wrong. In fact, if you tell anyone outside the IT industry that you work there, you can see that they are thinking: "Lucky guy! Wish I were in his shoes!!" (In fact, a medical professional who had come to our office for a presentation said, "You are IT guys. IT guys are intelligent people - you understand things fast..." (May not be his exact words, but they were to that effect). His assumption was that since we were IT guys, we would all have high IQs, we would grasp things immediately, and we were generally a cut above the members of the general population like him. This is the hype/respect around IT companies in India.)

To all you readers, if you have ever thought so, you will have to read this article. Note that all IT companies are like this - but if you have worked for sufficient time in the industry, then you know what Harshad Oak, the author, is talking about.

Because of the hype about IT companies, and also because of their contributions to India, many software developers generally do not talk about the conditions in such companies. Harshad hits the nail on the head when he says:

However I was avoiding writing this particular piece as it seems like an unpatriotic thing to do, to tell the world how bad the working conditions in software companies in India have become.


Yes, it does seem unpatriotic - just that I did not realize that patriotism has also leaked into this.

The reason why no one talks about is because the pay in such companies is generally higher than the pay anyone of the previous generation would have earned, when they retired. Hence the young guys assume that because they are being paid "unreasonably", there is nothing wrong in expecting unreasonable amount of work hours.

The software professional Indian is today making more money in a month than what his parents might have made in an year. Very often a 21 year old newbie software developer makes more money than his/her 55 year old father working in an old world business. Most of these youngsters are well aware of this gap and so work under an impression that they are being paid an unreasonable amount of money. They naturally equate unreasonable money with unreasonable amount of work.


Sadly, it does appear to be the way in which most software cos in India operate. This makes it hard to look around and switch cos - a reason why everybody seems to have accepted this as a way of life and started switching around for money. Software cos have also observed this, and seem to have started equating employee welfare with more salary. Harshad again hits here:

Top bosses of companies like Infosys, TCS, Wipro, etc. need to send the message loud and clear to their company and to other companies listening at national IT events that employee welfare is really their top concern and having good working culture and conditions is a priority. Employee welfare here does not mean giving the employee the salary he/she dreams of.


In A Field Guide to Developers, Joel Spolsky mentions that
If you start to hear complaints about salaries where you never heard them before, that’s usually a sign that people aren’t really loving their job. If potential new hires just won’t back down on their demands for outlandish salaries, you’re probably dealing with a case of people who are thinking, “Well, if it’s going to have to suck to go to work, at least I should be getting paid well.”


Sadly, in my belief, Indian IT companies have crossed that point long ago.



OK so...

Those were some essays that I felt deserved mention. You may or may not accept my list. In your opinion, there could be essays that are far better than the ones listed here - in such cases, do leave a link in the comments section and also a short explanation of why you consider these essays to be Great Essays. It would be nice if you could also list which of the above 3 criteria your essay satisfies to be called a Great Essay. Also, in case you do agree/do not agree with some points in my post, do comment!! In the meanwhile, I will leave you with links to three other essays that can also be considered Great Essays.

Why startups condense in America
Another essay from Paul Graham. He lists the reasons why Silicon Valley type environments are currently available only in the US and what it would take to replicate them elsewhere.

Why nerds are unpopular
An awesome essay that tries to answer the question of why geeky students are picked upon in school. Paul Graham goes into history to analyse the reasons why this occurs. Awesome!!

Toilets in Japan
Toilet - when you hear this word, you probably think, "What's there to speak about a toilet?" I thought the same too - I thought toilets were done; there was nothing that could be done to improve them, if anyone actually thought about improving them. Also, I had the belief that there could not be much technological advances in toilets - after all, its a toilet, what could there be in it? But this Wikipedia article opened my mind to the amazing toilets in Japan.

Sigh... why aren't such toilets available in India?