Steve Cook:

Cameron has been blogging about new features in our product.. In a recent post he used the term Explicit Design. I’ve been reflecting on this, and I like it. In software development we really do need to capture design data that is not just the code, but should be saved and versioned just like the code. What do we call it? We could call it “models” but “model” and “model driven development” are subject to so much historical baggage and methodology and terminology arguments. “Model” just seems to imply baked-in code generation and round tripping, when there is so much more that you can do with it: planning, verifying, testing, refactoring. We need new vocabulary that represents our ability to capture versioned design data at a more abstract level than the code without simultaneously implying the history of CASE.

I have to agree: changing the name doesn’t solve the root of the problem, but perhaps we start thinking in a different way.

A recent post from Markus Voelter replying some articles from Martin Fowler has resulted in an excellent discussion about repositories for MDSD and textual vs graphical DSLs. I was thinking about joining the discussion in the original post, but as my answer is a bit long to be added as a comment, I’ll reply him here.

In his post, Markus raised a question that recurrently appears in my thoughts: where is it better to store models? as files in a VCS (like CVS/SVN) or as structured data in databases?

Unlike Markus, my first answer is usually: without any doubt, in databases. I suppose that this answer has much to do with my professional background, since in the past I have worked extensively with CASE tools. Yes, I believe I’m one of these people that M. Fowler describes in his post about MDSD:

The MDSD vision evolved from the development of graphical design notations and CASE tools. Proponents of these techniques saw graphical design notations as a way to raise the abstraction level above programming languages – thus improving development productivity. While these techniques and tools never caught on too far, the basic core ideas still live on and there is an ongoing community of people still developing them.

But also, it is because I’m used to working with very large repositories, where there are lots of relationships between models. If you work with lots of components but few relationships, you usually do not have problems working with local files (such as Eclipse does). VCSs are able to deal quite well large amount of files. But what happens when you have, for example, your entire transactional system modeled (> 30,000 components), with a very high index of reusability between models (lots of links), and each model can belong to a different owner (only the owner can modify it)? As Markus points out at the comments:

Also, text files tend to be a bit hard to scale. Often the minimum you need is some kind of “cross-indexer” via a database so you can efficiently cross-ref, search, etc. In a “real repository” that’s easier.

Consider the Xtext case. What do you do once you have hundreds of Xtext resources? Each linking into each other. How do you efficiently load, unload, search, find-refs, etc? You need some kind of (in memory or persistent) index.

Exactly. When you work with huge amount of data and links, you need to provide some kind of impact analysis or cross-reference functionalities, you must be able to do complex queries, you must version not only the components but also the relationships, and you must be able to link to other models without the need to download them locally. Yes, I know that there are some solutions out there that provide some of those facilities also for files, as text search engine libraries (Apache Lucene) and query languages (for example, XQuery for XML or SPARQL for RDF). But IMHO all of these solutions, although they work very well with few components, are not as powerful as what you can get *for free* using relational databases.

But I must also say that my opinion has changed somewhat over the years, due to my experience working with repositories. The approach of working with databases also has some problems. One of them is that in order to work with the tool you always must be connected to the database, and, although this situation sounds silly, this may limit the productivity of some developers. With VCS, you only need to be connected while you perform the checkout of the component, but after that, you can work locally with it. Another pain point is that in relational databases, you must to create a fixed schema (no matter if you use a metamodel, you always must create it), and that could be a mess when you need to modify the data structure, since RDBMS doesn’t provide schema versioning facilities. Fortunately, some new approaches has appeared in the market in the last years, as schema-free databases, that will help in this task. Another side effect is that if you want to preserve the integrity of the models and the relationships, you have to deal with locking mechanisms, so the scenario become worse, and usually, the system tends to be over-engineered. And finally, there are also some functionalities not provided by databases, as versioning, accountability (who, when and what) and in some cases traceability (why), so you must develop it by yourself (yay! we love to reinvent the wheel!). Almost all VCSs provide these facilities.

Let’s go back again to the original post. Markus talks about some conditions where repositories could fit well in this scenario:

My point is that a repository is not per se a bad thing, provided the following criteria: (1) you store all your relevant stuff in it (2) it provides versioning facilities (3) supports diff/merge on a meaningful abstraction level.

Ummm, I agree with almost everything, but I’ve some concerns:

1. Not sure, if he talks about storing all of the data that belongs to a model in the repository, then I agree. But if he talks about storing the model and the code together, then I disagree. There are some scenarios where this is not convenient. For example, when you want to be platform independent (and I’m not talking about all the MDA stuff). The various parser/generator/interpret could run and store the code/binary on several platforms, and not always the same platform where you store the model.
2. I agree, versioning is an essential facility.
3. Diff/Merge works well with textual DSL’s, with a concrete syntax. But, although this is a great feature, is it mandatory? I have worked a long time without this feature and I assure you that you can survive without it. And what happens with graphical DSLs?

Before concluding, I would also like to comment one of the latest projects where we applied MDSD. At this project, we decided to use a Oracle XML DB to store our models in XML (something like to what Eurocontrol-CFMU have done for their UML models), but we added also some metadata. By storing the XML directly in the database, we avoid the need to decompose the XML into a relational schema, and allow developers to download the XML and work locally without the need to be connected to the database. We could use also all the SQL query facilities, and for those situations where the performance could be a problem, then we use the metadata to store some relevant data and relationships. Oh, and this RDBMS provides us also with versioning facilities. At this moment, we don’t have enough data in the repository to tell you if this approach will be a success or not. Let’s see!

To sum up (or not!). I believe you should never reject the database approach (nor the VCS option). I can not give you a “Golden Rule”, but my advice is that if you are not going to have lots of relationships between models, then use the VCS approach. If not, then analyze first what a VCS approach could offer you, and if it doesn’t fit well with your requirements, then use the database approach. But please, be careful and don’t tend to design the metamodel too much complicated, or you could have lots of performance problems.

As the post is quite long, I will leave for another post my thoughts about textual vs graphical DSLs. In the meantime, what is your opinion? I would love to hear stories from other folks on what people are doing in their companies.

As I wrote in a previous post, one of the main problems I saw in Microsoft Visual Studio DSL Tools was the lack of support for UML. I’m not a big fan of UML, but I must recognize that a common modeling language could be helpful in some scenarios.

Now, it seems that Microsoft changed it’s view. Bill Gates announced at Tech·Ed 2008 for Developers that UML will be part of Visual Studio 10. Great news. But this announce does not means that Microsoft is moving away from DSL. As Cameron Skinner wrote in a post, Microsoft will be using an hybrid model, a combination of both approaches: UML at the “logical” layer and DSLs at the “physical” layer. Not as powerful as openArchitectureWare, but a great step forward.

Surely, by the time you read this you will have already read it. Last week, Microsoft released Visual Studio 2008 (codename Orcas).

But what has not been announced are the changes in DSL Tools for VS2008, mainly because there are not any new major features except some changes in the runtime, support for LINQ and lots of bug fixing. Anyway, if you are interested, Stuart Kent, Senior Program Manager with the Visual Studio Ecosystem team, summarizes them in a blog post, and most interesting, he also describes the roadmap for the next version of Visual Studio (codename Rosario).

I’m personally interested in DSL Tools mainly for two reasons. First, because last summer I finished my degree thesis about Model Driven Development, where I’ve been analyzing at length Domain-Specific Modeling and Model-Driven Architecture methodologies. Among other tools, I analyzed the DSL Tools that were part of the VS2005, and I found them one of the best and most advanced tools for designing domain specific graphical notations (as opposed to textual notations, which, depending on the domain problem, are better suited) and for generating code from models. The main problem I saw in this tool was the lack of support for UML and XMI. I’m not a big fan of UML, but I must recognize that, in some cases, a common modeling language helps a lot, specially to reduce the learning curve that it is inherent to all DSLs. I know that this statement sounds opposed to the main concept of DSL, so I will address you to the article “Unified or Domain-Specific Modeling languages?” (thanks Metacase for recovering this article), by S. Ambler, who explains this contradiction very well. Also, the lack of support to XMI limits the interoperability between tools, something I believe Microsoft is not worried at all. Despite of these problems (IMHO), I believe Microsoft has done a really bad job of publicizing this tool, which is one of the pillars of Microsoft’s Software Factories initiative.

By the way, if you are interested in MDD, I would recommend you openArchitectureWare, now part of the Eclipse GMT Project, which is using an hybrid model, since it uses both approaches (MDA and DSM), and it allows both graphical and textual notations. And it is FREE!!!. Just check the overview diagram, so you get an idea of which technologies supports.

The second reason why I’m interested in DSL Tools is because we have successfully applied some external and internal DSLs at work (see this post by M. Fowler to know the differences between external and internal DSL). For a long time, we have been using textual internal DSL’s (PL/I Macros) in order to improve the quality of the code and to abstract the developer of some complex parts of the system, and we are very satisfied with the results. And more recently, we have been using graphical external DSL’s to represent the models for the online transactions which runs on our main backend (a z/OS mainframe) and to generate code addressed specifically to an in-house runtime framework. Last year, when we submitted an RFP to renew our domain-specific tool set, we received several proposals from different vendors, and one of them came up with DSL Tools. We finally decided not to use this tool and, instead, use Eclipse and a modified version of the jBPM plugin. Anyway, as I told you before, I consider DSL Tools one of the most advanced tools for DSM, so, periodically, I try to learn which is the evolution of this tool. And just for your curiosity, here it is the DSL design for our main mainframe architecture using the DSL Tools Designer (yes, it’s a service orchestrator!):

And now to conclude, I am really interested in knowing if someone has a real experience using DSL’s. If this is your case, are you using VS DSL Tools? If not, which tools are you using?

BTW, if you are interested in the thesis document, just drop me a line using the contact form and I will send you it. Be aware that the document is written in Catalan language.

I have discovered another interesting online publication: The Journal of Software and Systems Modeling (SoSyM).

Software and System Modeling (SoSyM) is an English language quarterly international journal that focuses on theoretical and practical issues pertaining to the development and application of software and system modeling languages and techniques. The aim of the journal is to publish high-quality works that further understanding of the theoretical underpinnings of modeling languages and techniques, present rigorous analyses of modeling experiences, and present scalable modeling techniques and processes that facilitate rigorous and economical development of software.

The following are some of the topic areas that are of special interest:

• Methodological issues
• Model-based testing techniques
• Development of modeling standards
• Formal syntax and semantics of modeling languages such as the UML
• Rigorous model-based analysis
• Model composition and transformation
• Metamodeling techniques
• Measuring quality of models
• Modeling support for aspect-oriented development
• Ontological approaches to model engineering
• Generating test and code artifacts from models
• Model development tool environments
• Case studies and experience reports with significant modeling lessons learned
• Comparative analyses of modeling languages and techniques
• Scientific assessment of modeling practices

Check the list of articles at Springer, you will see well-known authors, as Ivar Jacobson, Alan W. Brown and Grady Booch among others.

Actually they have two Call for Papers for Theme Issues: Metamodeling and Model-Based Systems Engineering with SysML, UML, and OWL.

I get a copy of the last The Architecture Journal, by Microsoft, obtained by my manager at the Microsoft Strategic Architect Forum (SAF).

This issue is focused on “Software Factories” (I don’t like this name, it could be confusing) and it has nice articles that explains some experiences with DSL and how to fit into the software factories methodology. One of the articles is from Jack Greenfield, co-author of Software Factories: Assembling Applications with Patterns, Models, Frameworks, and Tools.

Just discovered that the lastest IBM Systems Journal issue is about Model-Driven Software Development. I’ll need some time to read it completely (more than 200 pages!).

You could find it at http://www.research.ibm.com/journal/sj45-3.html