An ode to rebuttals, writing in general, and to things unsaid

There are some things that need to be said,
But the words escape me,
The thoughts are there in my head
But the verses escape me

... to be continued

Talaash

This verse is actually in Hindi, transliterated

Kis zameen ko dhoondhta hai tu
Kis aasmaan ki talaash hai
Sab to hai, bas yahin, tere saamne
Aankhein phere rehta hai kyun

Thanks to Google's transliterate tool, here's it written in Hindi:

किस  ज़मीन  को  ढूंढता  है  तू
किस  आसमान  की  तलाश  है
सब  तो  है , बस  यहीं , तेरे  सामने
आँखें  फेरे  रहता  है  क्यूँ

Emergence of chaos

It's been a long long time since I wrote one of these, but reading James Gleick's book Chaos got me thinking about all that is going on these days and how much you can apply it to them, albeit in an abstract manner, since I obviously lack the mathematical skills to model it accurately.

Let me start off by introducing this one equation that started a lot of it, the Logistic Difference Equation. It may sound complicated, but all it says is that x_next = alpha * x * (1-x), where x is anything you're modeling. What many scientists found was that choosing different values of alpha can lead to wildly varying behaviors, behaviors which are actually observed in diverse fields such as weather, populations, economics etc. I'll give one small example of populations, that if alpha is small, populations remain constant, or decline, but if alpha rises above a certain threshold, it can rise explosively. Now this is a very simplistic model, but it shows what "non-linearities", i.e. functions that are not a straight line, are capable of.

Why it applies in the current context is because of the attempt to control a number of highly undesirable developments happening these days. The economy is going to hell, education is starting to suffer in a number of notable countries (I know that more than most :P). As a result or otherwise, politics is becoming much more extreme. Since the USA is too easy an example, I'll take my home country, India. Currently, a notable gentleman by the name of Anna Hazare is leading a crusade to wipe out corruption from India, starting with the government. While some may applaud, and others may scoff, my interest for the time being is to see if his approach is viable or not. For a thorough critique of his methods, try reading Greatbong's blog post about it. So Anna's idea for rooting out corruption is to create a body above all others, whose members are not elected, but nominated from the notable members of society. This body, headed by a "Lokpal" would bring forth charges against officials and try them. Now, most engineers would notice the lack of feedback in the system, meaning the system would function wonderfully if we assume the members of this board to be unaffected ghosts bereft of all feeling and earthly connections, i.e. those who cannot be swayed by us common, corrupt folk. However, if an error enters the system, e.g. a corrupt person becoming a Lokpal, can have drastic consequences, since it's very hard for the system to filter out this error. Herein lies the nonlinearity and the scope for drastic and dictatorial repercussions. Like the logistic difference equation, specific values of alpha make this an amazing system, some bad ones have the possibility of making this a mistake of Nazi proportions...

The same idea applies to economics, since this idea owes much of its origins to economics. Markets, the US economy and consequently the dollar are suffering like an enemy of Chuck Norris/Rajnikant. People are quite upset about this brouhaha and want something to be done, what this something should be varies considerably from person to person. The core of the problem is, and still remains, that the US economy does not make enough money to finance the expenditures of its people. Like it or not, that's the plain simple truth, don't believe me, look at the labels on anything sold in the USA and see what percentage is actually made there. Made in the USA is a very rare label to see these days. The one thing that's held the US up is the dollar, since being the world's reserve currency, no one wants to see it depreciate, in spite of rampant injections of fresh dollars. This is one of the reasons that USA has been able to recover atleast partially from the housing bust while other like Greece have not, as borrowing is a lot easier when you can print the payments without seeming loss of value. To those who might not understand this, just imagine a supply-demand system, where the supply of any commodity becomes almost unlimited, what would happen to its price. Truth be told, smaller scale vibrations like this have happened in the world economy in the past, it's just that things could be brought back under control much easier than today, cause the world's economy was simply in much better shape. This time it's so bad, because the USA is stretching the limits of its borrowing power (again, like it or not, it's the truth, see China's reactions, they're the biggest debt provider to the USA), and its people are not yet ready to accept that their spending is beyond their means. Similar situations unfortunately exist in a lot of the developed world. So this time the nonlinearity could be brought to heel, or could spiral out of control.

Why I keep harping about this nonlinearity is cause I work on simulations, and know that like an experimental simulation code, the point where your simulation becomes unstable and explodes is usually preceded by small oscillations in either direction, i.e. the situation becomes better, then worse, then better, then even worse and so on. These oscillations increase in amplitude and at a critical point, take over the system and every bit of expected behavior is overturned. Unfortunately for us, very little is understood about such systems, yet all of nature, physics, and even human behavior is completely non-linear. We have managed so far by approximating these, since our "alpha's" have been small, but it seems like we're quickly getting to the point where these oscillations could escalate into chaos. After all, doomsday prophecies don't always have to mean that the world ends literally...

The dance of politics, faith, and EXPTIME problems

It's been a while since I last posted, cause I've been keeping busy in other things. Now that I have a deadline coming up, my natural inclinations to shirk work are rising again. This post begins with a rather frightening thing I've noticed about news coverage in USA and the amount of debating that goes on about lots of issues. It seems like everyone has an opinion about everything, and are hardly interested in any other points of view. Though I don't claim to know the subtleties of any of these issues, the lack of interest in an actual debate annoys me no end, even though I can see the reasoning and vested interests in avoiding actual debate.

Finding correct answers to anything but the most trivial questions is a hard problem. I can give a quite few reasons for this, causality, complexity of the problem (watch my new TV favorite NUMBERS for a very hand-wavy explanation of what I mean here :) ). Tracing things back to computer science, my bread and butter, "hard" problems are usually considered in a class of NP problems. These are problems to which solutions cannot be computed by any known method in time proportional to a polynomial function of the input size. In layman's terms, this is a class of problems to which computing an answer is hard, using regular computers in any sane amount of time. This is why your online transactions are safe, cause you can't factor numbers into their prime factors easily. But the interesting part in this class is that the solution can be verified in polynomial time, i.e. easily. For example, if you know a possible factorization of a number, say 20 as 2x2x5, you can check it simply by multiplying them. Now CS has only reached these problems till now.

Coming back to political debates and arguments, you'll see that even though you may find a solution by some definition, actually proving it to be a correct solution is not possible easily. In fact whether a solution exists is not know apriori. To me at least, and for life as it is, these problems are much more interesting. After a little searching, I found out that these problems are called EXPTIME algorithms. Obviously, this claim of mine is not exactly true, but what I'm trying to say is that solutions to these problems can be found and verified in exponential time, a.k.a. very long time in layman terms, but still possible. For the time being lets assume my definition of this class of problems is accurately represented by EXPTIME, but it's more likely the EXPSPACE class (look at the wiki pages for these if you're interested). My point is simply that these are the most encompassing of all problem classes I could find.

Now, even with NP problems, algorithms only attempt to produce an approximately correct solution, since an exact algorithm can only be computed once P=NP is proven. Till then, the only known algorithm for these is pure brute force. This can be understood by trying to get out of a maze by going through every possible path. Since, in reality there are means of finding out if you're going horribly wrong (like towards a waiting minotaur), you can prune your search "tree", which is the basis of most AI methods like the ones used by Chess playing computers like Deep Blue. The thing is that for finding solutions to generalized problems, knowing when a solution is correct is really hard too, so we must rely on some intuition or develop some approximation to model the problem. The real world analogue to this is faith, i.e. when something is so complex that understanding it goes beyond one's limit of knowledge, one starts to make assumptions and trust the assertions made by someone. The gap in science and religion stems from this very fact, since science keeps encroaching on religion and magic. I quite liked the explanation given in a trailer to the upcoming Thor movie, which said that his world was one where science and magic merged, as everything could be proven by science, so everything was magical yet scientific. Coming back, faith is a sanity check employed by most problem solving machines in nature, due to the inability of any mind to model anything in its entirety.

For the CS inclined and otherwise, it's important to look at this in terms of a tree. Any brute force search is visualized as going from the root of a tree to a specific leaf which is our correct answer. We must choose a branch at every decision making step. Faith is like turning a blind eye to certain branches, saying those are off limits and our desired leaf can't possibly be on one of them. The scientific method, and to an extent an actual debate, is quite the opposite, it must traverse every possible branch unless proven to be wrong. A lot of this also works from a bottom-up philosophy, by saying we want to reach this leaf, so lets make assumptions along the way down to the root to get us here. The alternate is a top-down approach, where you want to find the leaf by traversing branches. While faith will claim assumptions as it assumes the leaf to be paramount, logic tries to find the "right" leaf by finding reasons to choose each branch to be correct.

If any of that made any sense to you, it's easy to see that faith and logic can rarely agree. Thus it is so very important to understand why we're trusting something or someone. Blind faith is like going through a maze with a blindfold on, you may think you can get out with your amazing senses, but it's likely you'll hit the minotaur, and your glorious life will come to an abrupt end. Even so, a large populace will never adopt a purely logical approach because certain effects cannot be modeled by pure logic, without using probability and statistics, which as we all know, everyone hates cause you can use them to prove pretty much anything. True nirvana can only be attained by realizing that there is no "correct", there's only entropy maximization. So unless you want to expend a lot of effort to fight randomness, the only option is to mostly accept things that don't make sense, and fight your small battles against it, hoping that like a butterfly flapping its wings, your feeble pokes at randomness can actually curtail and guide it.

Borobudur

If you haven't heard of it before, I wouldn't blame you. Do a google search on it, and you'll find out more. Borobudur is an ancient relic, a building that pictorially depicts the life of Buddha, and his journey to enlightenment. The building is multi-storeyed, and as you go around all the floors on your way to the top, you see Buddha progressing on his journey to enlightenment, when you reach the top, you've seen his story, and symbolically atleast, attained enlightenment. The interesting feature of the building is that you cannot see any floors above you, but you can always see the floors below you, an expression of the causality in life, and a reminder to always remember where you're coming from. Ever since I learnt about the place, I've wanted to see it, and hopefully someday I will.

The second building I find fascinating isn't something you can see in one location at all. Technically, it doesn't exist at all, out of a computer i.e. If you've played civilization, you'll know the building I'm talking about. The intro video to the game revolves around this building as it's constructed and shows a fast forwarded progression through the ages as the building evolves over time. However, we always only see the facet of the building that represents the current age.

Those of you who've read my posts before know this is about to get hairy, so here goes. I was thinking about maturity, personality, and how we grow mentally over time. Language lets us put binary tags on people, like whether they're mature or not, and the like. However, something I realised recently, is that our maturity really is scenario dependent, i.e. the same person could be very mature about one thing, and yet wholly immature with regards to another. My reference to the virtual building is a pointer to that, that we are like a building under construction, and each facet to it proceeds rather independently of the other. They can pull each other up or down, but that's about it. The independence alludes to the fact that each part can be built very differently, and can have varying levels of stability. People can react very differently to different things, their behaviour is somewhat consistent when the transition between situations is smooth, but when shocked, the variation is quite stark. Even more importantly, the stability, let's us try different things. If we are stable and confident about a part of our self, we're much more relaxed in trying out different things, cause we're well grounded in that and know we can cope with whatever new thing gets thrown at us. You'll always find that the people most adamant about a stance, are the ones who're too dependent on it, or too scared as to what will happen if they're proven wrong. These are the ones who'll fight tooth and nail for it, while the more relaxed, more mature ones will stand by. Not to be confused, the ones who have no stance at all, will do so as well, but merely out of the fact that they have no maturity on it at all. It is similar to window cleaners cleaning using a swing. You need a certain amount of strength to hang one, and so not all buildings can take it. As a kid, I was always taught to not fight but listen, cause when you listen you can understand the other point, and to an extent, it helps you ground your own ideas in more sound arguments. As they say, only fools rush in.

I may have mentioned the part about Borobudur before, that it's easy to see how obvious the right answer was later, the gift of hindsight, as they say. But more important is the patience part, waiting and progressing when the end is not nigh. Agreed life doesn't always, if ever progress linearly, but it's a useful trait nonetheless to have, one I believe is the hardest to acquire, not that I have, yet :)

This post is a realization of the fact that no building is ever truly finished, and that outside veneer doesn't say anything about what's inside, and that even though it may feel like we've reached the top and stuck our hand out as far as it may go, there're more floors above and still further you can go on that ledge (figuratively of course, please not literally), and the fact that the ground seems so far below, is a good indication that you've progressed a lot, but there're still other floors to go to, and other parts of the building to complete, lest you forget about them. That's all for the building puns for tonight folks!

De(con)struction and (De)bugging

Over the past few days, weeks and months, I've been involved in the arduous process of debugging my code to make it work. As anyone before me who's done the same can attest, it is a process that can test the very limits of your sanity. Interestingly, it does offer some interesting insights, which I noticed today. One of the most important parts of debugging, atleast to me, is to try and understand where and which of our assumptions breaks resulting in the bug. Which is interesting as you have to try and think why you did everything you did. The part about retracing your steps and trying to reason your decisions gave me an idea of one place where this idea needs to be used more.

Recently more than ever, we've heard phrases like "like god intended", or "like the forefathers/founders intended" tossed around for every thing from gay marriages to immigration. These are some of the most irritating phrases one can hear, cause to be so bold as to claim that you know exactly what went through someone else's mind is naive at best, completely stupid at the worst. There's an old anecdote which goes "Only fools are certain", which rings true at this juncture. The biggest problem is that most people never stop to examine their assumptions and check if what they're thinking isn't based on something spurious. Deconstructing the series of choice one made to reach a conclusion, and the seeing the underlying assumptions is something most people are incapable of. The best example of it is a sportsman trying to recover his "form", since he didn't know the reason why he was good in the first place, he can't figure out how to get back at it. The irony is that for most such people, the reason they were so good was that they weren't paying attention and letting their instincts guide them, instead of coaches. That is an inherent risk in trying to deconstruct a chain of reasoning, that one might simply get lost in the intricacies that we didn't pay attention to, but were snuck under the rug, under the guise of an assumption. Most people are unaware of the number of assumptions we work with everyday, which is a good thing, cause it's our mind's way of coping with the excessive amounts of data it faces. The point when it starts to become counterproductive is when we ignore the checks in place which are supposed to tell us when we're using an assumption, and when it's based on fact. A simple example, if someone comes up and says that by reading one book by an author, he can completely understand the author's mind, we'd know that person's insane. Even better, if one were to say this after reading one of the helper notes like cliffnotes to such a book, you can rest assured the person's quite a loony (or maybe a genius, but the a loony nonetheless). Substitute the person making the claim with any religious authority, and the book by any religious text, and you start to get the picture of why blindly trust clergy with interpreting the will of the god(s) (assuming you believe there is one or more to begin with) is nothing short of lunacy. The same logic holds good for people advocating against gay marriages and the lot using such arguments of faith and religion, cause extrapolating what's written in a text when the author isn't present to clarify the context, is placing too much faith in your understanding or that person or god.

People bashing aside, most texts present this weakness because they're intentionally vague to cover as many bases as possible. Now if a religious text were to try and explain why something should or should not be done, it would be clearer and more easily understood, not to mention the hordes of mathematicians and the like who would admire the completeness of the doctrine. Ok, maybe expecting that from religion is a bit of a stretch, but certainly expecting this from the laws of the land isn't. We have such complicated laws that we need a whole court system to interpret these laws. Now, different judges still have different interpretations of each law, and interpretation often clouds the actual purpose of a law. It would make sense for lawmakers to include a reasoning and context for which a law is being based, so as to circumvent any loopholes that may present themselves to be exploited. Though this would present extra work for lawmakers, as well as the lack of loopholes which could be exploited for extra income and gratification, it is something in the interest of the people who are expected to enforce as well as follow these rules. Even more so, it would help to demystify laws to the layman and remove this sense of loss of clarity when trying to comprehend the reason for submitting forms in triplicate. To be honest, it is what is expected of major software undertakings, so that different people can collaborate on a single project. Then why do laws never give reasons or explanations on why they are as they are. After all, if professionals are expected to do this, is it too much to expect our government to be a little professional?

Alas the apple fell

Recently there's been a lot of talk about the new iPhone, and it's signal troubles. One small thing that got swept under the carpet, was the unfortunate effects of the new iOS4 for users of the iPhone 3G, like me. Apple, as usual, never really gives an "option" on whether to upgrade an OS or not, nor does it ever issue an advisory on whether the new OS can render your phone a relic of the past. But, before we delve any deeper, lets start from the beginning.

I've always been a Windows and Linux guy, ever since I played my first games on MS-DOS. I do have respect for Apple's design, but not as much for the systems they contruct (hardware wise they seem a little behind the times to me). So when I bought the then state-of-the-art iPhone 3G, I surprised myself too. Quite frankly, I hadn't planned on it, but that beautiful phone caught my attention as soon as I saw it, it was love at first sight! Logic failed me, and I ended up shelling out a hefty amount (I'm still a grad student) to get this shiny new plaything. Granted, I never regret having made that decision, I stand by it. But 3 months later, I realised Apple couldn't care less, and rendered me and my shiny new toy obsolete. A couple of months of depression later, I realised it probably wasn't as bad as I first thought, Apple's new plaything was only a minor improvement, and not something that would scar me forever as I initially thought. Yet, that reget of not having waited a few months before taking the leap lingered. Time passed on and we got to the new year with more hope.

As rumors of the new model reached a crescendo, so did little leaks of a new iOS, and of support for multi-threading at last! My brain told me not to increase my expectations, my friends advised me against it, yet the fool that I was, I walked right into it! A few weeks later came the shattering news of the fact that my phone would not support the hallowed feature. Alas, Apple broke my heart again. Add to it the inscrutable thinking that Apple projected, made me wonder about the sanity of their thinktank. The silence and hostile actions against all who dared to doubt its vision confounded me. Yet, with the news that it had overtaken Microsoft as the biggest computer firm by share price, Apple's popularity seemed to have reached a zenith. Not losing hope, I rejected overtures from contenders like the Droid, ah those sluts! How could they ever match the beatific and virginal lure of Apple! They tried to convince me that Apple did not reciprocate my feelings of admiration and love, and that it would never, yet I did not give up.

But alas, the new iOS and iPhone 4 broke the camel's back. They supported something I had long despised, something that made Microsoft one of the most hated Goliaths of the 90s, dictatorial decisions and a lack of openness. They cancelled the long promised support for flash, blocked all development tools other than their own, the new OS took 10 attempts to install, and finally took about 6 hours to install! And even then it has made my phone like an old man's wheelchair on the freeway. Atleast when MS released a new OS, they had the decency of telling you which computers it would make obsolete, so you would not install it on those. Apple has yet to learn that it seems. The new phone has none of the etheral beauty of the versions past, and seems like a botched half attempt to do too many things. Quite frankly, the aura that it once had has diminished, and the other options seem more and more attractive, seemingly not just for me, but for others as well. If all stays the same, Apple might return to it's position of something that has much promise, but never quite knows how to deliver. It's behavior over the past few months reminds me of this line from LOTR, "The door is shut, it was built by those who are dead, and the dead keep it, the door is shut!" (mostly the door is shut part, nothing dead yet)

And so I know I must return to my comfortable world of Windows and Linux, and put this ill-fated experiment behind me. Android seems to be on the rise, and well, Google isn't that evil yet. Every day Microsoft too seems more and more attractive. Yet I shall forever remember my little escapade, before the apple fell....

(Ps: if this seems a little insane to you, you're not reading carefully enough! :) )