Comment by RodgerTheGreat
14 years ago
Progress for computing, absolutely. Progress for software engineering? Questionable. Are the clever solutions applied in legacy spellcheckers obsolete simply because sufficient brute force is now available on everyday machines to solve everyday problems?
The problem with a lot of clever solutions is that they often obfuscate the code. This makes it harder for someone to figure out exactly what and how you are doing something. The clever solutions were necessary back then because cycles, memory, and storage were at a premium. Today they are mostly premature or unnecessary optimizations.
As an example, a few months ago a young developer I worked with implemented a set of attributes on a table as a bit field. He had just read an article about then and did it because it sounded cool. It made running reports against the data and doing imports via plain SQL impossible without writing a bunch of extra functions. His clever solution that saved about $0.0001 worth of disk ended up costing several hours of developer time because he didn't just a join table and a foreign key.
Yes. Engineering is constraint management. You're constantly trading off one thing for another: speed, flexibility, maintenance, cost, durability, etc. When one thing fails to be a relevant constraint, it's ok to focus on the other things that are of more immediate concern.
There are always questions on the edge of computational complexity. In 30 years: "Indexing all of the individual web pages in the world used to be a tough challenge. Network speeds were in the hundreds of megabits, and..."
I would be surprised if that analogy holds up. I have heard the Google guys state that the growth of the web easily outpaces advances in computing and bandwidth. The English language, not so much...
You're missing the point. "Growth of the language" wasn't the problem for spell-check, it was the size of the language combined with limited space. Computational breakthroughs solved that problem. The parent to my post implied that solving the 'space' problem in a different way (i.e. by having more space) somehow harmed software engineering, which it hasn't. Ergo, the suggestion that the web is outpacing computing and bandwidth all the more supports my point, since that problem will be persistent.
> In 30 years: "Indexing all of the individual web pages in the world used to be a tough challenge."
Eh, that needs qualification to make sense. Just crawling and trivially tokenizing the web counts as "indexing all the individual web pages" but that's never been such a monstrous task.
Having results fresh to-the-day/hour/minute is a better example of something that will probably be looked at is child's play even though that's a very new, very computationally expensive development in search.