Last month I attended the QCon London 2008 conference, one of the best conferences about software development. I’ve been very busy after the conference and I didn’t had time to write my thoughts about some of the sessions I attended until now. In the meantime, InfoQ, one of the organizers, has published a worth read post with some views and perspectives of other attendees who blogged about this conference. Anyway, here they are my notes:

How Eclipse changed my views on Software Development – Erich Gamma

Erich started his keynote explaining the Eclipse evolution, from its inception in 2000 as a closed project, the reaction from the development team when IBM decided to release the code in 2002, the success of the transparency model, to the design of the Jazz Project in 2006 as a team collaboration platform to integrate all the best practices learned during the years of the Eclipse development.

He talked about how transparency and accountability added an incremental value to Eclipse, and how it’s related to the Open Commercial Development style, an hybrid development model that takes aspects of both the open and proprietary development models, something that IBM is applying to the Jazz Project and to Project Zero. He said that OCD is more than publishing the source code, it is an open, transparent process, from feature requests and planning through delivery. BTW, this model is very criticized outside IBM due to some misconceptions (developers works for IBM for free).

He described some best practices applied during the Eclipse development, summarized as “It is about being continuous: Continuous iterative and adaptive planning, continuous design/refactoring, continuous integration/testing, continuous delivering/demos, continuous feedback, continuous learning”. It’s what they call the “Eclipse Way”, some practices from all kinds of sources (Scrum, RUP, …) underpinned with values (for example, ship quality on time). But he also said there were some pain points, specially with some boring and painful tasks, and the lack of a integrated tool set.

Then, he explained the Jazz Project, which goal is to be a scalable, extensible team collaboration platform for seamlessly integrating tasks across the software lifecycle, and he finally did a demo about Rational Team Concert, the first product based on Jazz, and asked us to try it by ourselves at the Jazz.net site.

Amazon Services: Building blocks for true Internet applications – Jeff Barr

Jeff, Amazon’s senior web services evangelist, summarized in this session the different services offered by Amazon focused on Cloud Computing. He explained which are today challenges for a company that operates globally: data centers, bandwidth, operations and scaling. Then, he explained in detail the utility computing services available to users:

He also said that, unlike what many people think, most users who use their services are large companies. He gave the example that many of the Fortune 500 companies use Amazon’s infrastructure services to run their development environments.

Keeping 99.95% up time on 400+ key systems at Merrill – Iain Mortimer

Despite the session title, Iain, Chief Architect at Merill Lynch, talked about how Merrill monitored their systems (about 344 tier 0 & 1 globally disperse servers). Two interesting notes:

  • They implemented their own technology, as none of the vendors was able to guarantee a 99,95% SLA (eat their own dog food).
  • They monitor 9 billion messages a day.

Born to Cycle? An Agile Approach to Working – Linda Rising

Funny session, with lots of participation and a great discussion.

Basically, Linda explained that humans are not designed to be linear, instead we act by pulses: we move between energy consumption and the renewal of the energy consumed. Therefore, we must be able to manage our energy, not our time (see the article “Manage your energy, not your time“, Tony Schwartz, HBR, October 2007). If we can find a balance by establishing a regular rhythm of work and rest, then we will have a higher productivity in a more sustained way. She gave the example of setting four 90-minutes sprints (like the REM sleep cycles) a day, where, in each sprint, we have to concentrate on the work to be done without allowing interruptions, and then rest for 20 to 30 minutes. Anyway, everyone must find his own cycle.

She also explained that if you switch your attention from a primary task to a secondary one, then the time it takes increases 25%. The audience answered explaining that this was a nice statement, but hard to be done. What must we do with email, phone calls or IM interruptions? This question resulted to a funny discussion about a study stating that CNN or BBC tickers reduces IQ by 10%, and someone replied saying that smoking marijuana only reduces IQ by 5%. Session conclusion: is better to smoke marijuana than to watch CNN!

REST: A Pragmatic Introduction to the Web’s Architecture – Stefan Tilkov

Stefan did a great introduction to REST. First, he explained that there are 3 different definitions for SOA:

  • An approach to business/IT alignment: driven by business instead of technology, relying on strong governance and implemented using any technology.
  • A technical architecture: service oriented with clearly defined interfaces, and could be technology-independent.
  • Web Services: business data as XML messages implemented using WS-* stack.

Then, he explained 3 different definitions for REST:

  • An architectural style (the right one): what appears in the Roy Fielding’s doctoral dissertation.
  • The web used correctly (aka WOA or ROA): to use HTTP, URI and other Web standards “correctly”.
  • XML without SOAP: send plain XML via HTTP violating the Web as much as WS-* stack.

After this introduction, he explained the basic principles of REST with some nice examples. He finally stated that there isn’t any REST vs SOA war, it is REST for SOA. There are two visions for SOA, to use REST or to use WS-*. The difference is on the technical layer and on their roots:

WS-* roots = The Enterprise: “A gigantic, uncontrollable anarchy of heterogeneous systems with varying quality
that evolve independently and constantly get connected in new and unexpected ways.”

REST roots = The Internet : “A worldwide, publicly accessible series of interconnected computer networks that transmit data by packet switching using the standard Internet Protocol (IP).”

If you are interested in this presentation, Stefan has put the slides in his blog.

The Cathedral, the Bazaar and the Commissar: The Evolution of Innovation in Enterprise Java – Rod Johnson

In this session, Rod started citing some sources of innovation: creativity, experimentation, competition and economic motivations. Then, he detailed the J2EE evolution linking with the innovation factors before commented, with particular emphasis on how the standards established by the JCP resulted on the decline of J2EE.

He compared the JCP with two development models: the Cathedral model, software built by relatively few people, with centralized design; and the Bazaar model, many developers, especially linked to Open Source, who lay out their wares. He stated that neither model is a complete solution: the bazaar model encourages competition but may not produce innovation, and the cathedral model is more likely to produce innovation but doesn’t produce competition. He finally said that JCP acts like a politburo commissar (they know what’s best for the developers), creating excessive standards and ignoring existing technologies. Something that it’s actually evolving but that should change much more and much faster if they want to survive.

eBay’s Architectural Principles – Randy Shoup

Randy, eBay Distinguished Architect, talked about 4 architectural strategies they use at eBay:

  • Partition Everything:
    • Split every problem into manageable chunks. eBay uses 2 partitioning patterns: segment databases into functional areas and split databases horizontally.
    • No database transactions (lot of buzz from last year session by Dan Pritchett). Developers must careful order DB operations. And remember: consistency is not always required or possible.
    • No session state, so user session can move through multiple application pools. Transient state is maintained by URL, cookies or scratch databases.
  • Async Everywhere: use asynchronous processing as much as possible, applying message dispatch or periodic batch patterns.
  • Automate Everything: it is better to use automated systems to manual systems. Example: automated tool executes staged roll out, with built-in checkpoints and immediate rollback if necessary.
  • Remember Everything Fails: assume every operation will fail and every resource will be unavailable, so build all systems to be tolerant of failure. eBay enforces that every change must have a rollback plan, because they roll out their entire site every 2 weeks (16,000 application servers in 220 pools).

Functions + Messages + Concurrency = Erlang – Joe Armstrong

Great fun with Joe’s session, the inventor of the Erlang programming language. He satirized about the reasons that a 25 years old language like Erlang is scheduled in the “Programming Languages of tomorrow” track. He told us that Erlang was created by accident, due to the strong requirements from the telecomm industries (severe penalties if the system is unavailable for more than 4 minutes per year).

He explained that functional language is not the most important characteristic of Erlang. What really matters is concurrency and distribution oriented. He gave the example that Moore’s law is reaching its limit, and how after a 52% power increase in each new CPU architecture prior to 2002, now we are seeing increases of only 20%. He explained that computer architectures are evolving towards: multicore (without success, because applications are still running on a single CPU), cell computers (hard to program) and network on chip (NOC). He also discussed the significant decrease of new CPU’s power consumption: 850 KW for 1 Tflop in 1997 to 24W for 1 Tflop in 2007.

After that, he explained the Erlang’s excellences to run on multicore systems, essentially using the Actor Model paradigm, and the ability to be a fault tolerance language. Finally, he said that every year we will see how sequential programs will be increasingly slower, and for that reason, it is important to be prepared for concurrent-oriented languages.