And if it requires merging together several different mutually exclusive paradigms, such as OO and FP, isn’t this just getting messy for the sake of having too many little moving bits?

In the end, are we looking for intricate little bits to fiddle with or are we actually trying to take something with a massive amount of complexity and reduce it into something that is simple, maintainable and solves (some of) our user’s problems?

So, if after fifty years of progress, the technologies aren’t actually getting more simple or obvious, and are in fact getting way way harder to grok, isn’t that a strong indicator that there is some type of serious problem with them? Do we get more from our modern technologies, or is it that there is just more code and data available to play with?

Paul.

C’mon, you know I’m not smarter than you :-) It’s just a matter of getting used to some paradigm/methodology/etc. For example, Python and Ruby make functional programming easy to integrate in a object-oriented language. Once you have gotten used to use functions as first class citizens, you feel that Java imposes you too many limits.

That’s why I like C++ and Python so much:

- traditional procedural language (check)
- object oriented features (check)
- functional programming (check). In C++, if you want to pass around functions, you’ll need to write a class that overloads the () operator (usually called a functor). It’s not as straightforward as in Python, but it’s something you cannot do in Java.

Since kids have grown up using high level tools, it should be easier for them to think in abstract terms, that’s the key IMHO.

Agreed - though I’d say to think in the new paradigms, as I don’t know that they’re necessarily more abstract. In the case of OO, I’d say that it’s more natural and more concrete, not more abstract. But that’s just a matter of words - I agree that that’s key.

You don’t look at problems the way I do. You quickly see approaches and solutions that I find very natural once you’ve pointed them out to me. I’m hoping to change that, but maybe you just are smarter :-)

I didn’t think all this out in great depth, and some of you have raised good exceptions that I agree with. As I said to Tim Bray in Twitter, I could have easily argued the other side. But I do believe history shows that that is ultimately the dinosaur’s side of the debate.

BTW, I probably wouldn’t have written the article at all if I hadn’t found such a cool image on Flickr to go with it.

The VFS example is a good one. Maybe Twitter is a good example for discussion. They used very high-level encapsulation (RoR) and it took them a long way, a really long way - much further than 99% of startups get. And yes, they now have to re-engineer a lot of stuff (throw out some encapsulations, and adopt some others that are more reliable). But you might argue that if they hadn’t built their first site in days they might not exist at all.

Amazon web services and Google’s App Engine are another good example. Scaling is really hard. But certain aspects, including some pretty significant ones, are now encapsulated to the point where any programmer who can put together a few HTTP requests can build massive systems that scale (at least in some important ways). I see things like AWS as being huge steps forward. They may not get it all right immediately, but it’s very early days yet for cloud computing.

I think the point of the article is not that Esteve doesn’t know about what’s underneath, but that growing up with more powerful tools (OO, patterns, etc.) changes the way you see the world. Take for example the Linux VFS layer, it’s a fine example of OO applied to a project that has to deal with lots of low level things, there’s inheritance, polymorphism, etc. Someone who hasn’t been exposed to OO might have implemented it differently, but since it’s the product of a generation that grew up with objects, Al Viro (the guy who designed the current VFS layer) thought it was natural to design it that way.

Although our primary language at Fluidinfo is Python, we have had to deal with things in C and C++, so I don’t think that a programmer who only knows high-level languages can be a good one.

Since kids have grown up using high level tools, it should be easier for them to think in abstract terms, that’s the key IMHO.

It’s funny to write about oneself in third person.

I’m not claiming that progress in encapsulation is always (or even often) linear. Neal gave an example above (in beer brewing) in which we’ve gone back to re-examine how things were encapsulated. Also, I guess from your comments that I was thinking (or at least speaking) at a more general level than IDEs and programming languages etc. We do make mistakes, I agree. But the overall trend is towards more being encapsulated, towards good encapsulations gaining mindshare, etc. Very few people will ever need to write their own random number generator, or hashing code, or bitblt code - or you name it - there are hundreds of examples of that kind.

I also don’t like IDEs :-) Emacs is enough for me in that regard…

We have, in a very real sense gone a long way down a specific technology path, heavily investing in a style of building tools. But are they just more complex because they are, or are they a huge mess? Horribly colored code in a flaky IDE, running on a platform that acts irrationally may look prettier than the old text-based screens, but honestly what is it doing now, that it wasn’t twenty years ago? Sure, the data is massive, but the functionality?

As we stretch the limits of this current technology we are quickly getting to a level beyond which it is too unstable to manage. To get past that threshold we’ll have to go back and refactor our foundations. At that point, either Esteve goes back and re-learns all that was known or he goes forward and just re-invents the whole same mess again (for the 4th time?)

Just a guess.

Paul.
http://theprogrammersparadox.blogspot.com

I don’t know where your blog is, I don’t think. I’m surprised you know where mine is!

I’d like to read that article. We’ve run into some really interesting things just thinking about how to convert one function (an iterator that periodically goes to the network). It’s fun but also hard, as you say. I’m goin gto post about it to the Twisted list soon. Or maybe I should just blog about it…

Are you going to EuroPytho?

Terry

There’s value in encapsulating knowledge and making it available to others. You wrote a book and you have a blog where you bundled up decades of extremely hard won experience. It benefits you to do so, and it benefits the rest of us to learn from you the easy way. And so it goes…

I am a big fan of scripting languages, and I work with them for a living, but I think a programmer needs to understand things at a lower level as part of his education. The more languages and paradigms you know the better, but I think you’d need to know some C for example, perhaps not at the level of a professional C programmer, but what C provides would be understood. I am not sure programmers need to understand assembly, memory segments, hardware, etc. The more the better, but for programming I feel that low level is perhaps not that necessary.

On the other hand it goes without saying that we need young C programmers, new interpreters are written, new libraries are developed, people have to evolve operating systems, maintain interpreters, write bindings, etc. In general high-performance software will be needed and a guy with only high-level concepts can’t write it.

Now we’ll see if any of that helps me solve the captcha :-).

I still think the programming world has a need for high-level application programmers and low-level system programmers for a very very simple reason: while digital clocks replace analog clocks, we don’t have high-performance processors that are able to think in high-level abstractions. Python still needs C because there’s no Python CPU, the same way C must first be compiled in machine code by a good compiler because there is no CPU able to read C and make sense of it.

The maxim goes: write the app and then optimize it — do not optimize prematurely. Which is really something that should read: write the app in a way that is easy to read and follow for an external party and overall makes sense, and then you dive down and spend time with the implementation of your concepts to see if it lives down to your speed requirements. High-level languages that offer some functionality “crystallized” in a library are a barrier to this, as after you have made the app as good as you can, the library then becomes the bottleneck, and if it’s still an unacceptable bottleneck you have to get down a level and start messing around with the library’s internals, if you can (long live FOSS!).

Which, in two words, means: there’s a very fine place for both kinds of programmers in this brave new world. Neither makes the other obsolete.

However, I think where Graham is correct is in the hyper-niche that is the startup world. It encompasses a vastly smaller segment of the worlds jobs and mind-share, but is dominated by doing “neat things” with “old tech”. As evidenced by Steve Yegge’s ranting about fundamentals, it is relevant if you’re trying to do something special, especially performance related.

I was lamenting with a friend the other day that, really, very few of my peers know what their computer is doing, even the ones that know how to code, at a basic level. It’s not important for them to build a business/website/widget. They’re not inventing the New New Thing, but they’re successful and happy. Really, this is only a lament because *I* learned all that “basic” stuff that doesn’t really matter on a day-to-day basis. Or didn’t until I got a new job that focuses on performance.

I think the balance of knowledge (or lack of knowledge) is a great and wonderful thing. Computers are slowly replacing many of the old tools that are no longer required to make us successful. And the brain-damaged fringe that really like the details can work at the startups, or become hobbyists :) But, really, there’s better things to do with brain cycles than remember old, worthless stuff like Lisp (j/k).

Also there are some fundamentals that must be learned, just like any craft, e.g. electricians, architects, etc. We don’t make mathematicians use slide rules, but they still have to know the basic theory as they type formulas into a computer.

Hi Neal - those are good comments, and I agree. I guess you might say that in a slightly different way the whole food industry got packaging somewhat wrong (esp fast food) and that we’re now seeing the effects and working to re-package.

A difference from the computational world is that, with certain notable exceptions, the people doing the packaging and using the packages are mainly pulling in the same direction. Sure, committees can screw anything up, but their hearts are usually in the right place. And certainly that’s true of open source projects, and a competitive market pushes, in theory, towards better packaging. But when people are mainly trying to make a buck or are focused on the short term (next quarter’s numbers, etc) then I guess mistakes and corrections in packaging should be expected.

I’ll think about your comments in the context of computational systems more though. Certainly, I think, we’ve progressed in how we make APIs. The computational world is, at least in theory, more easily quantified. If you’ve got a better random number generator or compiler, you can either prove it or demonstrate it pretty effectively. So I think advancements there do get baked in. The fly in that ointment, as with lots of optimization, as you know, is that there are many trade-offs and nothing is really the best. So there’s lots of wiggle room.

Anyway, thanks for the comments. Shouldn’t you be writing that dissertation instead of inciting Esteve to riot? :-) Good luck.

Anyway, I’ll take Neal’s tip :-)

I think this is just a basic condition of mankind. My father and I would have similar discussions about how to get a job done or if knowing how to change out the transmission in your car is important.

I’d add to this idea that there is a periodic resetting of this encapsulation where the old methods are re-examined and portions re-embraced.

Example: Beer brewing. From 1960 or so on, beer became an industrial homogenized product through massive brewing innovations and distribution. Then about 1990 all these homebrewers start founding breweries and go back to the old methods of producing full flavored beer… and go digging around in Belgian monasteries for inspiration. Now we have a wide variety of beer styles available again.

I think with software the difference is the cycle time is about 4 years max. If a technology isn’t recycled or parted-out within 4 years it will slowly rot away.

Esteve: don’t ask for a raise, ask for more equity! At 25, you’ll just waste the money, while you’ll chase that equity till the job is done.

I agree with what you’ve written. That’s why I say that it’s only rarely that the lower level skills are needed. Some things really need them, but lots don’t. When we release our stuff the world will be one step closer to not needing that Boost, TBB work that you did. Then one day you can be a dinosaur too! Can’t wait :-)