But before proceeding, a disclaimer. What I am going to write about Jazz is just a personal opinion, may or may not be wise, may or may not have something to do with reality, but I want to make clear that this is an entirely personal opinion, and do not mean any endorsement from my current employer.

The Eclipse vision

To describe the Jazz platform, I think we should go back to the past, because in my opinion, Jazz is trying to evolve the vision/mission/wildest dream that Lee Nackman had in 1998: to create a single technology platform on which to build the various IBM’s application development tools. The objectives Lee had at that time were:

1. to solve one of the most customer complaints: instead of having tools with their own “personality”, customers demanded a common look and feel;
2. to be able to integrate different tools, especially from IBM, but also from external ISV in order to complement IBM’s product line;
3. all reducing the development costs, as at that time each IBM tooling group used its own specific platform.

With the help from the autonomous OTI subsidiary (acquired by IBM in 1996), and overcoming an enormous amount of skepticism within IBM, Lee and his team delivered a technology platform that became what today is known as the Eclipse platform. Looking at the success of this platform, especially in terms of IBM adoption across the different brands and tools, it seems that the main objectives were reached. Not to mention also that open sourcing the platform and several projects (as the JDT), they killed lots of competitors, shrinking the Java tools market, and created a great ecosystem around it.

The Eclipse vision revisited

But the knowledge and tool set that IBM acquired when they bought Rational Software in 2002, mixed with a retrospective analysis they did based on the experience they gained in the Eclipse development, helped them to figure out which were the new challenges for the software delivery process. I’ll try to summarize, IMHO, some of the improvements they realize:

• When IBM built Eclipse, their focus was on the developer productivity. But the software development process usually involves some more skills, specialists, roles, levels, … and they need to work together, they need to share information, and when all members of the team work in a geographically dispersed manner, the conflicts inevitably appears. So there is a need to involve all the team in all the phases of the software lifecycle regardless of their location and role, and instead of improving the productivity of the developer, we need tools to improve the productivity of the entire team, and directly or indirectly, the productivity of the whole organization.
• When talking about covering the overall development cycle, we usually find that we need several tools, and sometimes these tools are outside of the scope of Eclipse. And we also find that there are lots of barriers to share resources between these heterogeneous tools, as they use private vocabularies, formats and stores. So the integration between these external tools are usually built on bridges, and lots of times, highly cobbled (so they require updates with every interface change). There is a need to raise the level of integration. We need to be able to integrate and share cross-repository information using open interfaces and a loosely coupled approach.
• When thinking about non coding activities, we realize that not each role or tool needs a heavy desktop client. There are some situations where a web UI (or another type of client) is more suitable. Despite some incubators (e4 Bespin or Eclifox), Eclipse nowadays only supports its desktop client. It’s true that using Equinox and its underlying OSGi services, you can deploy Eclipse plugins into the server-side, but there isn’t any “standard” way to share user interfaces or a framework for the web UI. Which will be the problem? the same Lee discovered in 1998: tools with their own “personality”, tools without a common behavior.
• Process, process and process. How much we love them and how much we hate them also? Why is so hard to try to follow a process? Why the only Eclipse tools available (RMC or EPF) only try to author and then publish a static document? Why tools doesn’t live the process?
• Creating a new Eclipse environment with all of the required plugins, configuring the project, setting-up the build process and all the other little pieces that come into play to give code life could be a mess for a new team member. This kind of manual tasks are tedious and error-prone, and they are perceived by developers as a waste time. So it is not strange that tools like Maven had a great adoption.

So in my opinion, and as I told previously, the Jazz platform is the evolution of the original Eclipse vision, keeping in mind the above and some more other concerns, with an special focus on teams and collaboration. But its aim is not to replace Eclipse, they are distinct platforms with different goals, although Jazz seems to be the perfect complement to Eclipse. This new vision is well summarized at the About Jazz page:

Our goal is to provide a frictionless work environment that helps teams collaborate, innovate, and create great software. To that end, we are focusing on driving fundamental improvements in team collaboration, automation, and reporting across the software lifecycle.

The Jazz Platform

When trying to describe what compose the Jazz platform, albeit IBM have split the original Jazz project in several projects at the jazz.net site, I still have some problems trying to draw the line between the platform and the applications, to see which components are part of the Jazz platform and which are part of the different products based on it. So I will try to use the below picture, that I have borrowed from the IBM Rational guys, in order to clarify my ideas:

To enable a seamless and higher level of integration between tools, IBM has defined a reference architecture, API specifications, and a set of common services and tool building blocks, that together are called the Jazz Integration Architecture (JIA). At the center of this integration architecture we found the Jazz Team Server (that may consist of one or more physical servers that act together as a single logical server), which provides foundational services (RESTful web services) to enable groups of tools to work together. Let’s summarize each of these foundational services:

• Presentation: in a multi-tool integration scenario we usually found lots of linked resources that may not be familiar to a particular client tool or this tool may not be able to provide a user interface for theses resources. The services provided by the presentation foundational services enables a client tool to find and invoke a suitable user interface for any resource URL in order to present the relevant data. There are also two main components (I believe, but I am not sure, they belong to the presentation services) that allow tools developers to implement specific user interfaces: the web dashboards component, that provides the infrastructure and UI for creating and presenting dashboards in a web browser, and the web UI component, that provides a framework for rich web user interfaces (based on the Dojo toolkit).
• Process enactment: these are the services that allows to define and implement a wide range of processes. It is focused on agile processes, but it can also be used in highly-structured processes, as it provides the essential components of a development work flow, such as operations, roles, permissions, preconditions or follow-up actions. By default, it is packaged with several process, as Scrum, OpenUp or the Eclipse Way (PDF), and it has an editor to be able to modify the process configuration. Each time you create a project you must assign a process, but you can have several projects and each project can follow a different process. It governs all activities, artifacts, artifact relationships, and operations that are pursued within the context of the process area, and it works in a seamless and unobtrusive way, as it manifests itself through artifacts types, operations manipulating the artifacts, and artifact change events.
• Administration, users, projects, teams: For dealing with users, projects, security, and licenses, each server hosts a set of core administration services. For example, these services can provide a common user identity in order to support authentication (establishes user identity) and authorization (a particular operation can be performed) based on the team membership or role in a project.
• Collaboration: Collaboration between the team members of a project can be performed in real-time, but also asynchronously (especially important for teams working across time zones). It also occurs at different contexts: around tools, process, tasks or data elements. The collaboration services in the Jazz platform supports and enables some of these core functions, for example, instant messaging, sending email and SMS, maintaining subscriptions, etc. It is something like a mix of the Eclipse ECF and Corona projects (and I wonder why they did not use these projects).
• Storage, data warehouse and search: You may have noticed that I have deliberately grouped 3 core foundation services in only one. The reason is because the Jazz repository model is composed by three logical DB in one, working together in order to provide the above 3 services. I am going to use another “stolen” picture to describe it:
  
  
  • Instead of having a fixed schema (that make integration hard) or a very generic schema (that makes writing tools tough), the Jazz repository allows tools to store their data their way. So content resources are created in a particular representation by the client, and can only be retrieved in that representation. The server doesn’t know enough about the content to transform it into an alternate representation. The storage services provides a completely RESTful framework for CRUD operations on resources stored in the Jazz private DB.
  • For every resource stored in the private DB, there are a set of “indexed properties” that are stored automatically in another DB using RDF. The indexing process is able to extract RDF triples from some resource representations and able to extract text streams from some resource representations as well. This process extracts asynchronously each tool’s data into searchable indexes, consolidates them, and provides centralized query services for searching across the consolidated index. In this way, the search foundation services are able to provide common queries, both structured queries (based on SPARQL) and full text search (based on Apache Lucene).
  • And finally, we find the data warehouse DB, a periodically snapshot of all the information, used for public reporting. The data warehouse services relies on the Eclipse BIRT project for its reporting system.

Anyway, the Jazz platform is still in its early stages, and it is constantly evolving to meet additional challenges. What I have summarized previously is what it is know as the Jazz Platform 0.6, but a new version is expected to be delivered on June with a new name, the Jazz Foundation. So if you are interested in more deep details about the above or new services that are going to be delivered, I recommend that you go through the development team wiki (registration required).

The killer-application

There is also an interesting parallelism between how Eclipse and Jazz has been developed. In order to convince other IBM’s development tools product managers to adopt the Eclipse platform, Lee and his team decided to build a Java IDE. There were two reasons behind that decision: 1) to provide a real example (a killer-application) of a tool developed on the platform, proving in that way its benefits; 2) to help the Eclipse development team to better understand the needs of future consumers of the platform and to discover areas that required further development. This strategy was a success and the Eclipse platform and the Java IDE were quickly adopted inside IBM.

The Jazz project seems to use the same kind of strategy. They are developing a real tool using the Jazz platform. This killer-application is called Rational Team Concert and as far as I know it is also rapidly adopted inside IBM. I hope to write about this product in the near future.

Adoption

Looking at the jazz.net site the increasing number of IBM tools that are adopting the Jazz platform, I have no doubt that it will be another success in terms of IBM adoption. But …

• Will it be a success outside IBM as was the Eclipse platform? IBM has not contributed the Jazz platform to the open source world (in terms of a free software license), and nor it is licensing it in any way (as far as I know). The only way to get this platform is licensing some of the Jazz based products. I am sure they are going to attract some more new customers looking for a complete lifecycle solution, but I believe it will not be a great success as Eclipse was. Anyway, I think the current goal for the Jazz platform is different from the Eclipse platform goal.
• Are ISVs going to adopt this platform for their own products? There are the usual business partners that are complementing/extending the IBM’s Jazz based products with some new features, but it does not seem that they are going to adopt the Jazz platform for their own products. And it does not seem that IBM is trying to convince them to adopt the platform, as they did with Eclipse, or licensing it on an OEMs basis.
• Will this platform attract external developers? IBM is not encouraging them to contribute in terms of code (the rights of what you contribute are transfered to IBM). They are only encouraging people to influence the direction of products through direct, early, and continuous conversations at the jazz.net website. So it will be very strange to see any non-customer developer.

So in terms of external adoption/extension, it seems that IBM is focusing only in the interface as a way to integrate non-IBM tools, encouraging developers, customers and ISVs to participate in the development of the Open Services for Lifecycle Collaboration initiative, something like an open standards consortium.

Conclusion

Starting looking at the original Eclipse vision and how IBM revisited it after the Rational Software acquisition and the Eclipse success, I have attempted in this post to describe what it is the Jazz platform. I am sure some of you have realized that some of the improvements that I have described previously can be easily or are already implemented through Eclipse plugins, but I think these are the minor ones. There are three main conceptual differences between Eclipse and Jazz:

• a server centric approach instead of a local workbench in order to leverage the team concept;
• a persistent storage using a federated cross-linked repository to store resources;
• a seamless integration between tools using standard loosely coupled open interfaces and web protocols.

And in my opinion, these conceptual differences can only be implemented through the creation of a new platform. Instead of solving some particular problems in an isolated way, Jazz is trying to attack the essence of the software development process. Does it means that we must convert to a new religion, drop the Eclipse platform and adopt the Jazz platform? No, Jazz is not going to replace Eclipse. There will be a strong relationship between the Jazz and Eclipse environments, yet the
two are distinct and can run independently. Jazz is going to complement Eclipse in some particular scenarios:

Eclipse for Individuals, Jazz for Teams

We have also seen that IBM’s decision is not to open source this new platform, but to create a community around the jazz.net and OSLC websites. However, it will need to attract a broad and active participation from a wide external community in order to be a great success outside IBM, as it was Eclipse, something that I believe it is not a current IBM goal.

What will happen in the future? Sincerely, I don’t know, time will decide. From my particular point of view, this new vision match up what we have been doing for a long time and that has lead us to extend the Eclipse platform in order to create a custom collaborative tool set. So, as we are already an IBM Rational customer and we have licensed some of the Jazz products, I will be very happy if we can integrate, in an easy way, our custom tools with the IBM Rational tool set.

If you reached this point, please, participate in the conversation

Before concluding this long post, I would like to ask you:

• To the IBM/Rational guys: As I assume that I could make mistakes (history, goals, …) , if you want to add or point out something wrong, please write me, better as a comment in this post although I will also accept private emails, and I will correct it.
• To the non IBM/Rational guys:
  • How much of you have heard about Jazz? How much of you have experimented with it? Which is your (technological) opinion?
  • Must IBM open source the Jazz platform? Do you think it will be interesting and wide adopted? Why?

Additional information

PS: One of the latest tasks Lee Nackman did before his retirement at IBM, was to help spur the development of the Jazz platform.

PS: Most of the people actually involved in the development of the Jazz project were part of the Eclipse platform development team, so it is not strange to see that they are applying the same strategy, but also adopting the best practices they learned during the Eclipse development process.

PS: There is an excellent case study on IBM’s strategy and process for creating Eclipse at the Harvard Business School. It’s a worth read.

PS: Another interesting read is a paper on Collaborative Development Environments (PDF) by Grady Booch and Alan W. Brown, which seems to be the “spark” that started the new vision.

The first one is a really cool feature introduced in the Eclipse Communication Framework project that enables distributed teams to reap the benefits of pair programming. Based on a Google Summer of Code proposal, Mustafa Isik developed Real-Time Shared Editing, dubbed Cola (collaborate), a mechanism that allows two developers to work collaboratively in real-time to edit source code and/or documents. He has put together a short screencast showing the usage of this technology. Check it out! Digging further in this amazing feature, Mustafa pointed me to a Google Tech Talk he gave at EclipseDay at the Googleplex where he explained how this plugin resolves in real time any change conflict. The video is worth a visit. And if you want to add this feature to other editors (by default it has has been added to the JDT Java Source Code editor and Eclipse’s Default Text Editor), Scott Lewis has wrote some easy instructions … simply by adding a little bit of markup to plugin.xml.

The second one is the Usage Data Collector, a piece of technology that will generate statistics on how the various components of the Eclipse workbench (loaded bundles, commands and actions, perspective changes, view usage, …) are being used by developers. The Eclipse Foundation intent is to use this data to help committers and organizations better understand how developers are using Eclipse, in order to improve the overall user experience. Privacy must not be a problem, as this feature is opt-in (there is an option on the “Usage Data Collector” preferences page labeled “Enable Capture”) and it is completely anonymous. Although the data collected is not quite representative, you can see right now some statistics (I see lots of Cut-and-Paste Programming). I hope that these statistics will be public and the Eclipse Foundation will publish some reports regularly (I have not seen any notice about this). But besides the benefits that these statistics may have for the Eclipse Foundation, I believe they can also be attractive to some organizations which have developed internal plugins. And I say this from my own experience. One of the problems we had in the past was how to measure the use of the different plugins we developed, and also, which was the response time (we had several complains about the client performance). We finally had to create an infrastructure in order to collect and analyze these data. So, I see with interest the possibility of extending the official UDC API (both, listeners and monitors). Let’s see how it evolves in the future.

From the project website:

This visualization, called code_swarm, shows the history of commits in a software project. A commit happens when a developer makes changes to the code or documents and transfers them into the central project repository. Both developers and files are represented as moving elements. When a developer commits a file, it lights up and flies towards that developer. Files are colored according to their purpose, such as whether they are source code or a document. If files or developers have not been active for a while, they will fade away. A histogram at the bottom keeps a reminder of what has come before.

Check some of these open source projects visualizations: Eclipse, Python, PostgreSQL, and Apache. You’ll be amazed!

MIKE2.0 is a comprehensive methodology that can be applied across a number of different projects within the Information Management space, including those that apply to: Information Governance, Data Quality Improvement, Data Integration, Data Migration (once-off batch migration and parallel runs), Data Warehousing, Master Data Management and IT Transformation.

I didn’t had time to review the wiki contents (I’ll do in the next few days ), but what I really appreciate is the fact that they are releasing most of their content using a Creative Commons license and that they are using a collaborative model. Quoting David Weinberger:

If You Love Your Information, Set It Free.

They encourage every information management professional to join this open source community, to fully participate in this effort and to let them know what you think about the intellectual property they have shared. So if your job is related to the IM field, take a look and share your opinions with them.

Here are some answers that I’ve found interesting:

On Eclipse, the focus was on how to make an individual developer more productive. With Jazz, we are looking to teams with the goal of making teams and teams of teams more productive. To do so requires tool support across all phases of the software lifecycle.

The biggest challenge [projects face today] is to ship quality software on time in the face of changing requirements and often geographically distributed teams. This is hard. It requires talented people. This cannot be tooled away. A key lesson is that transparency in software development enables feedback loops, which improve predictability. With transparency also comes accountability, a key ingredient for a healthy project.

Jazz is process-aware. The first process we are working on integrating into Jazz is the “Eclipse way”, the process we breathe ourselves. However, Jazz is designed to be open for other processes and we expect that over time many other processes will become available for Jazz.

Other interesting aspects are that Jazz is extensible on both the client and the server. In particular, we are using the same Eclipse-based extension mechanism not only on the client, but also on the server. We are leveraging the work from the Eclipse Equinox project.

Simplicity through consistency. Collaboration in context. Agility through transparency… Jazz is about helping people work together to build software more effectively, while making the software development experience more fun!

Jazz is a joint project between IBM Rational and IBM Research to build a scalable, extensible team collaboration platform for seamlessly integrating tasks across the software life cycle.

While this technology is at an early stage, it is an exciting and important part of Rational’s future.

The website doesn’t contain much information, just a community page with some upcoming events. It seems that people who attend to the Jazz BoF session at EclipseCon 2007 will be eligible to join to a Jazz community pilot.

I will look forward to it, as more details are expected soon.