Wednesday 16 May 2012

Software Challenge

When looking at the history of anything it is very tempting to focus on the competitive aspects only, e.g. the great wars, the clash of civilizations, etc. Unfortunately, all too often this is presented as the essence of history, and although that may be a story that is easy to understand or remember (or memorize), it may also miss the underlying fundamental changes that are happening all the time, in times of war and peace. Software history is no exception, as this picture attests: 

 

But are there any other stories to tell besides the epic battle between the Microsoft empire and the open-source barbarians? It is hard to believe that all significant advances in computing come from hardware alone. Can you find any data measuring the improvement in software over time? This question is intentionally vague and because of that I'll have to accept any metric you can find. Examples could include measuring software quality (lack of bugs or defects) or productivity: are those who write the software become more productive over time either individually or as a group?

What is the impact of new operating systems, programming languages, scripting languages, etc. on productivity or quality? In other words, is there any evidence that they enable software developers to produce more or better code? Has anyone figured out a way to quantify this? Send me any document that has numbers in it in the form of tables or figures (published or unpublished).

Another point of view is from algorithmic efficiency or computational complexity. Can we trace the improvement of algorithms from the early days of computing to the present? And how does that compare to the improvement in hardware, e.g. would you rather have an old (and very slow) computer running a modern (fast) algorithm or the latest (and fastest) machine running a primitive algorithm?

Of course, there are many different algorithms for all kinds of tasks, having different tradeoffs. But to keep things simple, what can we see if we just focus on one attribute, say, speed (which is also called "time complexity"). For example, consider sorting algorithms (arranging a bunch of numbers in increasing order). This website has some demos illustrating that the speed difference can be quite substantial! Could we plot their performance (speed) versus the time they were invented? If yes, can we see a trend?

Of course, sorting algorithms may not be the most interesting ones. There are many other kinds of algorithms in computer science, machine learning, data mining, pattern recognition, or artificial intelligence. The coolest thing is that one doesn't have to understand all the technical details in order to research this, only a few key concepts and the notation. If you are interested in these questions, I'll gladly bring you up to speed on that! :) 

No comments:

Post a Comment