Thursday, October 13, 2011

Why "Polyglot Programming" or "Do It Yourself Programming Languages" or "Language Oriented Programming" sucks?

Last year we saw the launch of a new Web programming language Dart - Structured Web Programming from Google. A very interesting approach to support web application development. Not so long after Go, Groovy, Ruby, Scala, << name your DSL here >>; we see Dart. Is it a good thing to have at least one programming language to solve one problem? The answer is, like we already know, it depends.

Some important backgrounds you should know about the multi programming language paradigm are following:

1. You can read Martin Fowler article about language oriented programming with language workbenches which enables you to write small programming languages easily. In this article I see everyone writing their small languages, everywhere. In this concept we see DSL (Domain Specific Language) as the future of our programming activities. Source: http://martinfowler.com/articles/languageWorkbench.html

2. Neal Ford talked about Polyglot Programming, combining multiple programming languages in application development. Later Mr. Fowler add this paradigm with Polyglot Persistence, using different type of databases within one application. Source: http://memeagora.blogspot.com/2006/12/polyglot-programming.html and http://martinfowler.com/bliki/PolyglotPersistence.html

Since 2006 I already discussed and collected some experiences in multi programming language paradigm:

1. I remember a “hot” discussion in year 2006 with Sebastian Meyen, chief in editor of JavaMagazin Germany, also the biggest organisator of Java Conference JAX. We agreed to see the future of programming in multi language paradigm concept. I also said that all those languages will be based on Java VM. I also told him that in one day SAP will move ABAP as a language which can be run within the Java VM, so just another language within the Java environment, no two different personalities any more. Today we see the beginning of this in the project called Caffeine ABAP. Source: https://cw.sdn.sap.com/cw/groups/caffeine

2. Also in year 2006 I had a project in which we also used different kind of languages and also creating our own DSL:

  • Java for the most implementation stuffs
  • UML for design of the business objects. We generate a lot of things using the concept of MDA (Model Driven Architecture)
  • Groovy for a lot of things, especially for writing unit tests
  • Based on ANTLR we create our own DSL for some aspects of the application
It was really exciting and we had some very good programmers in the project. The result was a very nice and flexible product, just as what we expected in the beginning of the project. Please read this article in German language for more information: http://www.sigs.de/publications/os/2009/02/dewanto_egger_OS_02_09.pdf

So after all those nice things about multi language paradigm I told you, why this sucks at the end? Here are some reasons from my point of view:

1. As typical in application development the problem comes first in the maintenance phase, after all the capable programmers leave the project. Did you, programming language creators ever try to teach a new programming language to a “normal”, 9 till 5 programmers? I’m not talking about 9 (am) till 9 (pm) programmers who love to learn every new languages. It is definitely tough to be proficient in one programming language. This is comparable with the languages we speak everyday. I’m not a native English speaker, so I’m quite sure that I made a lot of syntax and grammar errors in this article. It is possible to be able to speak three or four languages perfectly but this is an exception.

2. Did you ever try to maintain a big Struts web application with AJAX? Just try to add a functionality and you will end up with creating and editing a lot of files: Action and Form files, Struts XML configuration files, JavaScript files with JSON and also HTML or JSP files. Can you imagine to add Groovy, Scala, Dart additionally into that web app? The complexity of such a project is very high.

3. Creating a new programming language means that you also have to build the environment for it. Good IDE, good documentation, good community support, clear roadmap, backward compatibility are some points to be done. Groovy is a bad example of this. In the early version of this language the editor for Eclipse was really bad. After a while they improved the editor but they make a lot of basic changes in the language so your old groovy applications do not work anymore. You are punished to update to the new version. This never happens to Java. You still can compile Java 1.1 applications with Java 6 compiler.

4. Before you are creating your own DSL with e.g. ANTLR ask those language Gurus first, how hard it is to maintain a programming language for a long term. Before you discuss with them don’t ever create your own DSL. Especially if you are working for SME (Small and Medium sized Enterprise). With a small team and small budget you will never ever maintain your own language decently.

So in year 2012, six years after my support to Polyglot Programming, I hope to see following things happen:

1. One language for all aspects in one application is the best concept ever. I name this as “One for All Programming Language paradigm”. Just like we don’t use English for technical language, German as literate language and Indonesian as community language, to be able to communicate internationally with each other we just use English pragmatically for all aspects of our life. In Germany you need to speak German in all aspects to be able to communicate with others. My best solution sofar is Java + XML, that’s it, no more, no less. No mixing with Groovy, Dart, Ruby, Scala, << name your DSL here >> in one application. Especially if you are working as contractor, please don’t try to use all those languages just for a small Java web application. I don’t say that you should not use the other languages at all. The only thing which is important is not to mix those languages in one application. In SME you maybe also want to use just one programming language for all your applications.

2. Concept like GWT (Java to JavaScript compiler) or XMLC (XML compiler which compiles XML, HTML to Java classes) is great. You can work just in plain Java. Guice with all Java and no XML is also a great solution (I know that SpringFramework is also doing this with Annotations). Android is great because it uses Java as its application programming language.

As a conclusion I can only hope to see more such pure and plain Java solutions in year 2012!

19 comments:

Pierluigi Vernetto said...

I agree BUT...an application should be written in only 1 language, for sake of simplicity, agreed, but often accessory tasks like scripting the configuration of WebLogic domains need to be written in Python or Groovy... and honestly introducing the opportunity to learn a new "hot" language like Scala can attract on the project the best programmers.... surely the risk is that people lose focus and play with the technology...

anyway this is not a perfect world... it will never be...

laker2000 said...

I agree. We can only be masters of a small subset of languages. I'm already required to know Java, SQL, JavaScript, XML, HTML. My brain can only accommodate so much.

Christoph Adamek said...

I disagree. I haven't seen a project written in just one language for quite some time and never ever one based on Java. You always have additional languages included: HTML, JavaScript and all those ad hoc languages based on XML.

So idea to have one powerful language for everything is not new at all. In fact PL/1 tried to deliver this. But it obviously failed. The book on traps and pitfalls was three times the size of the language manual.

What you can do is to use a flexible language which allows for internal DSLs. You can adapt the language to your problem by in fact creating several variants of the base language. Classic example is Lisp but you can do it with Ruby and Scala, too. Certainly not with Java.

Sandro Heitz said...

Unfortunately: the truth.

Doovde said...

I'm sitting in a project with Java, python, jython, tcsh, bash, php, xml and expect.
It's impossible to debug in efficient manner, trouble shooting is like surgery while joggling.

I think although is needed to have several languages in the project, I hope the practical matters becomes better in the future.

lofi said...

@Christoph Adamek: Instead of creating and using a lot of programming languages and DSL you can use ONE programming language with many FRAMEWORKS. IMHO this is a better way... It is already hard to learn frameworks but still easier than learning another programming languages...

Cheers,
Lofi.

.killdream said...

First things first, Dart, Groovy, Ruby, Scala, etc. are not DSLs, they're general-purpose programming languages. General purpose programming languages are overtly complex, and you should avoid using many of these in a project.

When people say that you should be using DSLs for making your project easier to maintain and write, they're saying that for those things you're likely to do a lot, that'll maybe be executed by (below-)average programmers or that don't translate well into the host language without loads of boilerplate should be factored out in a small, declarative language that captures the essential primitives that express the meaning of the thing you're doing.

In essence, a well designed DSL allows non-programmers to write "code" in it with a few minutes of explanations of the basics. Good examples of it are:

- XML, Self-ML (writing document-structured stuff)
- Markdown, ReStructuredText (formatting text in ASCII)
- JSON, YAML (configuration)
- BNF, EBNF (syntax grammars)
- JavaDoc, NaturalDocs (embedded documentation)
- QML, Edje (declarative user interfaces)
- SQL (database queries)

So, what's common to all the things listed above? They're overtly minimal (probably with the exception of JavaDoc and ReST, but the basics of these two are quite straight forward).

- They all can easily be written by non-programmers after a few minutes of explanations.

- They all can be kept in one's mind without problem, since they're minimal.

- They don't need extensive documentation, IDEs, or any advanced stuff like that.

Now, what about DSLs for writing programs. Well, that's one of the basic premises of Racket, and one of the things Haskellites like to do to the extreme (which I believe comes from the obsession of Mathematicians to create specific notations to express their problems well).

Haskell allows you to write embedded DSLs. That is, you don't need to write parsers, you don't need to maintain a separate thing for your languages, everything has the same core semantics, but you can still be highly expressive. Granted, this is because Haskell is an astonishingly expressive language, which Java isn't ­— but there are alternatives for Java, as well).

One of Racket's core ideas (and in fact, Lisps in general) is that you always create a DSL for solving a specific subset of problems. Racket allows you to extend the syntax of the language from within the language, though, so you can create whole new syntaxes and languages, that conform with the same semantics, can still use the very same libraries, etc.

I think Racket and Haskell's approach to this is the way to go when you need actual programming capabilities, since they reduce the complexity and learning overhead a lot. But that's not to say you can't achieve similar results with a separated run-time, as long as you capture the semantics of the domain in a really minimal subset.

.killdream said...

Oh, btw, I recommend Paul Hudak's paper on Domain Specific Languages, as he explains better than I could only hope to why there's a need for DSLs in any reasonably large project.

http://haskell.cs.yale.edu/?post_type=publication&p=126

Jim Barrows said...

I somewhat agree. Part of the problem is Java itself. We need meta-programming information, and we use XML and annotations, neither of which is a programming language, and so absolutely horrible at their jobs. We need javascript and HTML for front end web programming etc.

The other part of the problem is that DSL's are for a domain of knowledge, not general programming. Using Javascript for the front end and Java on the backend doesn't meet this key criteria. Customizing a language (like Java, Ruby etc) so it reads more like how accountants talk for an accounting application does meet that description.
Martin Fowler's example of ordering coffee is a perfect example. Walking into a Starbuck's and ordering coffee is a whole new language, but, it's ultimately done in your native tongue.
All of this is a perfect example of the old Lisp addage "All languages attempt to emulate Lisp, badly", since it easily allows new DSL to be created without changing the base language. No other language I've encountered does this with the ease Lisp does.

Foudres said...

First like others said you seems to mess up DSL and general purpose language like scala that allow easy building of DSL.

You can make your application entirely in scala, or javascript or ruby if you want to.

The funny thing is among many general purpose languages, JAVA is the worse in term of DSL. Scala, python, javascript... All have better support for it.

So you are in kind of weird case where you need an external tool to create, build and test your DSL.

So why not just go directly to the next phase? Using only one language is a good idea, but why not use a more suited one? More expressive? More powerfull and that allow API to fell more like a DSL?

That the best of both worlds: you have a good intuitive API for you domain, the standard language tool still works, and you write your API directly in the language.

No need for a parser, an additionnal compilation phase and others things like that!

That's exactly scala, ruby, or even better lisp offer to you: a very flexible general purpose language that you can use to express any idea in an effective way.

soc said...
This comment has been removed by the author.
soc said...

"2. Did you ever try to maintain a big Struts web application with AJAX? Just try to add a functionality and you will end up with creating and editing a lot of files: Action and Form files, Struts XML configuration files, JavaScript files with JSON and also HTML or JSP files. Can you imagine to add Groovy, Scala, Dart additionally into that web app? The complexity of such a project is very high."

This doesn't make _any_ sense. You claim that "One language for all aspects" is a good thing and then pull an example which uses not only Java, but also XML, JavaScript, JSON, HTML, JSP, Annotations, Bytecode-Processing, Code-generation, etc, etc, etc.

Did you consider that some people use a better language to reduce this chaos caused by Java?

The equation is not "Java, XML, JavaScript, JSON, HTML, JSP, Annotations, Bytecode-Processing, Code-generation" PLUS "$OtherLanguage", but "$OtherLanguage" INSTEAD of "Java, XML, JavaScript, JSON, HTML, JSP, Annotations, Bytecode-Processing, Code-generation".

lofi said...

@Killdream, @Foudres: I’m aware of the difference between DSL and general purpose programming languages. In my context as a user of those languages I see no difference. You still, for example, need a good editor for each of those languages. I don’t want to work with a plain text editor to edit my DSL or my general purpose programming languages. Or do you still use plain text editor to write your SQL or XML or Java? All those DSL are useless or will be a maintenance nightmare if you don’t have any good editors for them.

About Haskell: I seldom met “normal” business app programmers who know Haskell or know how a functional programming language works. And if you take a look at the statistics, imperative programming languages have broader users than the functional one. For many of us it is still easier to read and write imperative programming languages.

I’m aware of products such as JetBrains Meta Programming System http://www.jetbrains.com/mps. Check out their intro for more info: http://www.jetbrains.com/mps/docs/tutorial.html … but still, it is complex just to create a simple calculator. They do it quite good with all those infrastructure tools (editors, etc.) but still why not just building a simple framework with good API?


@Jim Barrows: you said “Martin Fowler's example of ordering coffee is a perfect example. Walking into a Starbuck's and ordering coffee is a whole new language, but, it's ultimately done in your native tongue”.
IMHO: walking into a Starbuck's and ordering coffee is just adding a “framework” on the top of the language. I don’t see any added value by defining “frapuccino” as a new language. It’s just a simple framework on the top of the used language.


@soc: you said “This doesn't make _any_ sense. You claim that "One language for all aspects" is a good thing and then pull an example which uses not only Java, but also XML, JavaScript, JSON, HTML, JSP, Annotations, Bytecode-Processing, Code-generation, etc, etc, etc”.
Please read my sentence carefully. I said that using those stuffs makes everything much more complicated. I said, I just need Java, no more no less. Therefore I said I like the idea of GWT (Java to JavaScript compiler) and XMLC (XML to Java compiler), so you don’t need to work with JavaScript/JSON/JSP/HTML, instead you just use pure Java. Can you imagine how simple it is if you just can use Java for all those things? No JavaScript, JSP, HTML, etc.?

You said: “The equation is not "Java, XML, JavaScript, JSON, HTML, JSP, Annotations, Bytecode-Processing, Code-generation" PLUS "$OtherLanguage", but "$OtherLanguage" INSTEAD of "Java, XML, JavaScript, JSON, HTML, JSP, Annotations, Bytecode-Processing, Code-generation"”
Nope, the solution is just Java (+ XML), no more no less ;-)

soc said...

@lofi:
I fail to see how Java is an option here.

Simple example: Want to use a Database? Java provides no solution which could be considered "coming from this century".
Either a complete mess of SQL Strings being passed around or stuff like JDBC/JPA/JDO/... which involves huge amounts of magic and still fails to provide a usable, modern interface to data stores.

Java just doesn't work in production without XML, annotations, custom classloaders, bytecode manipulation, aspect-oriented programming and code generation.

Take a different real-world example: Same story.

lofi said...

@soc: Java is NOT YET an option here but as I said, with technologies like for example GWT you don't need to use JavaScript anymore to build interactive Webapps. So we are getting closer...

We will never ever get rid of SQL for relational database, or XML for data representation, but I'm already happy if we don't have to use JSP, HTML, JavaScript etc.

Andy Czerwonka said...

Why is it that carpenters can learn to use different tools to do the best job and programmers resist? For me, that's the fun part.

soc said...

@lofi: You are joking, right? Have a look at F# type providers. Welcome to the 21st century, btw.

Web Developing Services said...

Some kinds of difference only in C and C++ languages.but it is the basic for web development and application development.
Website Design Companies | Web Design Companies

steve sebastian said...

This is very essential information and every web programmers and software engineers should know this information from this blog.
Web Design Company India | Web Development Company India