On Scheme

Thoughts on Scheme and teaching Scheme

The Next Big Programming Idiom

Posted by Peter on May 6, 2006

Before I begin I would like to state, for the record, that I am not trying to incite a language flame war (at the moment). I’m not asking here which language will be the next popular language, but what programming style will be popular. Will functional programming make a comeback? Will OOP refuse to die? Or will it be something we haven’t seen before?

One thing I am sure it won’t be is natural language programming. Attempting to program using complete sentences is the opposite of what you want a programming language to be, namely concise and powerful. Have you ever heard someone tell you that the code is the documentation? This is because for the most complicated algorithms pseudo-code is a shorter and a more accurate description than words could ever be. As programmers we want our languages to be concise and precise, and natural language is neither.

The rise of a given programming style, historically, can be correlated with what kinds of problems were popular. For example when systems programming was the hot topic C (and the imperative style) became dominant. Functional programming had its heyday with AI researchers, because passing functions around makes algorithms in AI research much simpler. Most recently OOP has become popular with the rise of GUIs, because objects and messages are a natural fit for window controls.

So to predict what will be next we should look at the kinds of problems that programmers deal with most, and I think it is obvious that the “hot topic” in programming is distributed applications, specifically web services, AJAX applications, and collaboration software. From this list it seems likely that the next popular programming idiom will need to have concurrency, state, security, and errors as its strongest points. Concurrency means that the idiom needs a transparent way to deal with threads and synchronization. If you ever need to explicitly create a thread or lock a mutex then the idiom has not solved this problem. State means that the idiom must support the ability to maintain unique data for any number of clients in a natural way, and possibly the ability to unwind state when necessary (much like a continuation). Security means that stupid programmer mistakes, like passing the user name from a text field directly to an SQL query aren’t possible. Tainting is currently a way to “solve” this kind of problem, but I can’t say that it is a very good solution. Another way might be to do away with the ability to pass strings directly to library functions. Finally, errors mean that there must be a transparent way to deal with the unexpected, either by correcting the error in place or exiting from the part of the program that raised the exception. Once again, having an explicit try-catch construct is a sign that errors are a problem that hasn’t been solved as part of the idiom.

Web apps have been around for a while now, so you may be wondering why the next big thing hasn’t arrived already. I think the delay is because in this generation of problems we have several strengths required of new idiom that are at odds with each other, while in previous generations there was the real problems were closely related to each other, and thus easier to solve all at once. What I mean by all at once is that it can’t be simply be four solutions glued together, it needs to be one solution that solves all four problems, much like how OOP solved data encapsulation and message passing at the same time. And of course they must be transparent solutions, meaning that the best solutions with respect to these problems are also the easiest solutions, and require the least amount of programmer effort possible.

OOP enthusiasts will of course tell you how well OOP deals with these problems (try-catch, thread objects, synchronization objects, yadda yadda). These solutions are not transparent at all, and more tellingly the fact that you are expected to use a framework in many OO languages to create large scale web apps (java, ruby, ect) shows that the solutions provided by this idiom aren’t really up to solving all the problems. However, because its replacement hasn’t arrived on the scene yet programmers are stuck with OOP, resulting in buggy and insecure applications. Yes, you can make an OOP project thread safe, resistant to errors, and secure, but it is a radically different kind of task than throwing together a GUI, so why should programmers be using the same techniques? Unfortunately functional programming isn’t the solution either. Although functional programming handles concurrency well (assuming there are no side-effects), it suffers when you try to implement an error handling system, or synchronization when you absolutely must have side-effects.

A solution to the threading and state problems might be event based programming. In event based program the application is structured around a number of events, each of which is called asynchronously. Also each event is called in a “context” (local state). With respect to this context some events are exclusive, meaning that only one can be invoked at a time, while others can be running simultaneously with each other. Event based programming solves the state and concurrency problems, but not those of synchronization or error handling, meaning that it will not be the next idiom, since all of these problems must be solved in one fell swoop. I mention it simply to show that the idioms we currently work with aren’t the end of all language development.

The only conclusion I can provide unfortunately is that whatever the next idiom is that I haven’t seen it yet. If you think that you have feel free to leave a message here.


18 Responses to “The Next Big Programming Idiom”

  1. phil said

    “it seems likely that the next popular programming idiom will need to have concurrency, state, security, and errors as its strongest points. Concurrency means that the idiom needs a transparent way to deal with threads and synchronization.”

    We really don’t have good abstractions for concurrency as we’re still stuck in the Von Neumann mindset. ‘Threads’ are not the best abstraction – maybe actors are better, maybe processes. With the advent of multi-core processors (with the Cell being the current ‘extreme’ example with something like 8 cores on a chip) we in the software world definitely need to come up with some better ways to describe and think about concurrency. Perhaps we need to go more towards a dataflow model. It wouldn’t hurt to take a look at hardware description languages like VHDL and Verilog: they’re declarative and dataflow. Not that you’d want to start programming in an HDL, but there are ideas to borrow from the HDL domain.

  2. OO is not only for the GUI, it helps in real-world modeling which leads to better automation.

    IMHO, the shift is required in the architecture and design approach rather than the programming. The functional requirements usually overshadow the security and ‘secondary’ requirements and are sometimes ignored. They should be given higher priority. Frameworks are not a bad way of development, choice of wrong frameworks is.

    The next programming idiom should support more dynamism, users being able to pick ‘things’, modify and move ahead. Perhaps MUD can lead us to the next programming idiom.

  3. Peter said

    I don’t remember where I read this, but it stuck with me: sometimes you want a computer to be a computer. Trying to model everything with objects is a handicap, an artificial restriction on what the computer CAN do. It’s like not allowing yourself to use macros. Sure you don’t NEED them, but they make life much easier. Go read some critiques of OOP if you don’t believe me.

  4. Joel said

    Trying to model everything with objects is a handicap, an artificial restriction on what the computer CAN do.

    Good call Peter!
    I was trying to do something with a binary ‘structured field’ type file the other day in .NET and I found myself wondering why it seemed so hard – then I realised the reason was because the .NET languages are not C – the OO stuff really just felt in my way. I made it work, but it wasn’t as natural as using some old school ANSI C to get the job done.

  5. Conrad said

    I like this article- quite insightful…

    I always think that somehow the next programming paradigm would involve the old idea of “programming by example”… So you would show the computer examples of what you’d like done and it writes the program that does it, following the laws of Occam’s razor- Sort of like Test-Driven-Development, except that you only write the tests.

    I can already see how there is some similarity between the functional tests I write in Ruby and the source they test- If you could “factor out” this similarity somehow, you would get an idea of what I’m thinking… Then, if the program is running and no example exists to disambiguate a situation, the program would halt and ask you to disambiguate (what do I do when two users are trying to update a record at the same time?, etc.)

    just thinking aloud…

  6. Peter said

    That’s an interesting idea, but could we make it percise enough so that the computer behaves predictably?

  7. Carl said

    All I know is, the next programmer to implement a language for web apps that doesn’t require string encodings to be explicitly declared is gonna get a knuckle sandwich.

    And the PHP guys aren’t safe either.

  8. I believe the best way to handle concurrency is to use transactional memory. This is currently a very hot topic both in academia (see herlihy, harris, peyton jones) and in industry (intel, sun, and microsoft research are all working on it). It will likely show up in Java 7 or 8, and it’s part of the new Fortress language being defined at Sun. Transactional memory makes it trivial to reason about concurrency in the presence of state, and it gives you a form of rollback as well.

    Phil suggested declarative and dataflow programming. I whole heartedly second this suggestion, and I happen to be actively working on two different approaches to this idea right now. The first is FrTime, which is a functional reactive programming language based on Scheme, designed by Greg Cooper. The second is StreamBase, which is a commercial implementation of a stream processing engine with support for continuous queries. Two very different approaches to solving somewhat related problems.

  9. bek said

    i try to find something at google.com and take it on your site…thanks

  10. Bob said

    functional programming plus exceptions is good enough for me.

  11. Andreas said

    I find it a bit extreme what you require of a new programming model. It sounds like you expect error handling to be completely handled by the language (be that actual language or framework) itself which I think is impossible. After all neither does OOP make the objects vanish.

    As to security the bug is not that you can pass any string into library functions but rather that there are functions that should not accept strings in the first place: Whenever that string is parsed, whether as SQL or as a Unix command. Such functions need a structured interface, not a string argument.

  12. Kreiger said

    I think the word is “paradigm”, not “idiom”. Otherwise good article.

  13. […] a post titled The Next Big Programming Idiom, Peter discussing this issue. I’m not asking here which language will be the next popular […]

  14. Daniel said

    I like what I’ve seen of Erlang for concurrency, and it seems to be cropping up a lot lately… message passing, low-cost + functional processes as the basic structural element. It would fit IMO with your idea of popular paradigms being determined by the problems we see now, since multiprocessing is getting big.

  15. Jeff said

    When you combine the ease and straightforward nature of the Ruby language with the JVM, giving it the capability of using all those Java libraries, with inherent concurrency that will be absolutely necessary in order to take advantage of the multi-core processor, it sure looks like JRuby could be the NBL.

  16. […] Programming paradigm or common library features as idiom […]

  17. […] Programming paradigm or common library features as idiom […]

  18. […] Programming paradigm or common library features as idiom […]

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: