davidbau.com

Representations Stand Alone

Thanks to Dion at TheServerSide, there has been some discussion on the distinction between Encapsulation versus Representation. Some good reading has been suggested. There have also been questions.

For example, if we distinguish between encapsulation and representation, where do "Java Value Objects" lie? Value objects have an encapsulating interface, but they clearly represent just data. They seem like the Platypus, neither fish nor fowl - or maybe both.

Representation is not Top-Down

One objection to the idea of using representation rather than encapsulation was well-stated by Kamleshkumar Patel:

If you were to start working with data directly, it would seem that you would be taking the approach of building a system from bottom-up instead of top-down. In my opinion designing a system from top-down would produce a better system in terms of sticking to proposed requirements and scope.

Patel represents the orthodox object-oriented view. And if you are building a single system, he is right: you don't want to begin by agreeing on your low-level data structures - you want begin with interfaces that come from top-level requirements. The reason? Because when you begin building a single system:

1. You have an idea of what your desired behavior is.
2. You do not know what your data structures are.

However, when building a network of systems, especially when intereacting with systems that already exist or do not yet exist, the situation is reversed:

1. You know you are stuck with specific kinds of data.
2. You have no idea what other systems do with that data.

Is the strategy of representation top-down? No - absolutely, it is not. But is it bottom-up? No! If you are doing bottom-up development and calling it "representation", you are just making a sophisticated-sounding excuse for avoiding top-down analysis!

Perhaps a better way of looking at representation is that it is not for "vertical" subdivision but rather it is a form of "horizontal" design. Representation is neither top-down nor bottom-up, but it is the right approach when splitting a system up into peers.

Java Value Objects

An example: Java value objects are data objects that are passed between components in Java. They can be used as a "representation" strategy. [Although Dhananjay Nene points out that they usually are not introduced for that purpose.] There is not much to the idea besides the fact that value objects do not do anything other than represent a bunch of data. Here is a value object interface for a car-repair order:

interface RepresentedCarOrder { int getCustomer(); // must be a valid 5-digit ID Date getDate(); // date of the request String getRequest(); // the request double getEstimate(); // esimated charge Employee getAssigned(); // another value object double getCharge(); // acutal charge }

When you are implementing your top-down design and get to the bottom turtle that does the actual work, value objects let you separate your behavior from your data. What do we mean? If we were not using value objects, we might define EncapsulatedCarOrder as a subclass of an abstract base class Order (see the last article), and require it to implement three basic Order methods:

class EncapsulatedCarOrder implements Order { public void show() { System.out.println(request); } public void execute() { Mailer.sendNote(customerID); } public void close() { Queuer.remove(this); } // and a constructor would be needed... private int customer; private Date date; private String request; private double estimate; private Employee assigned; private double charge; }

A real engineer not writing pseudocode like me of course could happily implement show() method to display a snazzy and useful form. This is all fine and good when the known set of behavior we must implement is perfectly fixed.

The Compatibility Problem

What is wrong with the above design? It seems like an excellent example of top-down practices.

The problem comes when the application expands horizontally. For example, suppose that late in the design process, a federal court tells us that we need to build an audit process for car orders that generates an alert whenever the actual cost of an order exceeded the estimate. If our car orders are all hidden behind Order abstractions, we are stuck with the following choice:

1. Break encapsulation to access the "estimate" data we need.
2. Change the Order abstraction to add the "audit" behavior we need.

Although in the age of refactoring IDEs the problem is getting easier, real choices like this are painful and inherently difficult. The theoretically correct solution is #2, but if we have already finished the system and released it to the world, there may be a lot of software relying on the unchanging Order interface, and it may not be feasible to change Order itself. So in practice, the only practical solution all too often is #1, and we write terrible code like this:

Order order = Queuer.dequeueOrder(); // Yucko yukco yuck! if (order instanceof EncapsulatedCarOrder && (EncapsulatedCarOrder)order.auditRequired()) Queuer.enqueueForAudit(order);

In other words, if encapsulation is overused, it is often necessary to break encapsulation to add new functionality.

Encapsulation is vitally useful. However, you need to be careful to avoid letting encapsulation get in the way of expanding functionality.

Representation Counterbalances Encapsulation

There is a way out, and the value object design provides a way of formalizing the solution. Rather than build the functionality into the same object that holds the data, functionality should be separated from data.

We should recognize that an abstract class such as "Order" does not represent a good permanent abstraction for data, but just the relationship between our data and our particular application. When we implement the Order interface, we should be careful to seperate the implementation of the context-specific Order interface from the objects we use for the more permanent underlying data. Schematically:

class CarOrderForm implements Order { private RepresentedCarOrder value; CarOrderFormHandler(RepresentedCarOrder order) { this.value = order; } void show() { System.out.println(value.getRequest()); } void execute() { Mailer.sendNote(value.getCustomerID()); } void close() { Queuer.remove(this); } }

As you can see, this design separates the "current application's top-down design", which gave us the abstract Order interface, from the "real data", which is the RepresentedCarOrder value object.

With this design pattern in hand, if we wanted to build a new auditor module we would no longer need to interfere or collide with the Order abstract base class, but we could now write, with no squeamishness or difficulty at all:

class CarOrderAudit implements Audit { private RepresentedCarOrder value; CarOrderAudit(RepresentedCarOrder order) { this.valueObject = order; } boolean auditRequired() { return value.getEstimate() < value.getCost(); } }

This design would guide us to correctly, clearly distinguish any display queue from an audit queue, and would avoid any downcasting nonsense. (And with more rigorous audits, perhaps we will avoid the fate of Arthur Andersen!)

The point, here, is that to build robust systems, there is a point where encapsulation must end and where representation must begin. That point is where your application has long-lasting data for which you cannot anticipate all the future applications.

---

Is Representation a Subset of Encapsulation?

I highly recommend reading Carlos Perez's analysis of loose coupling, which lays out some more differences between the tightly-coupled (similar to what I call encapsulation) approach and the loosely-coupled (similar to what I call representation) approach. Although I believe Carlos would agree with my approach, Carlos disputes my use of the word "representation", pointing out that "Representation really concerns itself with a subset of ... Encapsulation," so I am not correct in suggesting you need to choose between the two. What does he mean? Since every encaspsulating interface must in the end represent its data somewhow, every contract that encapsulates also represents data, so it's not really an either-or choice.

But "representation" really is a whole technique on its own that is quite different from the object-oriented exercise of encapsulation. The very best loosely-coupled contracts are just representations, without all the other baggage of encapsulation such as definition of required behavior, temporal sequencing, or roles and responsibilities. Fully unencumbered by the dictates of any interpretation whatsoever, the best representations transcend the boundaries of time, space, and holy Garner Group classification.

Excel, OLE and POI

Even representations that are not-very-good provide enormous power to cross interoperability boundaries. This success of bad representations stands in stark contrast to the weakness of pretty-good encapsulations, which can have trouble crossing between threads in the same address space!

Consider Excel. Microsoft Excel is a client-side GUI spreadsheet application, written for Windows and Mac in C. Around 1990, Microsoft decided that they were going to bundle Excel together with their word processor other programs in a whole Microsoft Office suite - a business stroke of genius that muscled both Lotus and Word Perfect out into irrelevance. So Microsoft invested huge amounts of time and effort in trying to figure out how to hook together Excel, Word, Powerpoint, Outlook, and Access.

Their solution? Encapsulation. Microsoft engineers defined an encapsulation standard called OLE (aka COM and ActiveX), and then they defined a large number of "OLE" and "OLE Automation" interfaces for Office. In effect, they defined thousands of little behaviors and contracts, allowing programmers to do things like move the Microsoft Word cursor around from a script or change the contents of an Excel spreadsheet from within Access. By encapsulating very complex programs behind interfaces, OLE freed the Excel and Word teams to innovate and evolve their underlying data formats and algorithms while maintaining the same overall ineroperability behavior.

How far does OLE interoperability reach? Not as far as you would hope. Microsoft invested billions in OLE for more than a decade, and they were able to make OLE interfaces reasonably usable on a single machine running Microsoft software. But OLE has had little success on other platforms, and in the details, OLE is still full of many bugs and problems. My judgement is that the problems stem from the fact that OLE interfaces have to support a lot of behavior that comes from bad design descisions derived from its original 1991 design, including "apartment threading", "monikers", "CoTaskMemAlloc", and so on. Microsoft is now replacing OLE with a more modern .NET framework that is garbage collected, multithreaded, and internet aware. The interfaces of OLE encapsulation have failed to stand the test of time.

Did bad design decisions in 1991 doom OLE? No. The reason OLE had a limited lifespan comes down to the difference between encapsulation versus representation.

You can see evidence of this in the saved-Excel-spreadsheet XLS document format. The Excel development team has treated the XLS format (based on the closed OLE CDF format) as something proprietary and undocumented and and not for interchange. And for sure the format suffers from more than its share of bad design decisions. Nevertheless, the XLS format has solved more difficult interoperability problems than the heavily-engineered OLE interfaces ever will.

Unix web servers do not use OLE interfaces to interact with Excel, and OpenOffice does not support OLE Automation interfaces. However, Unix and Java web servers and OpenOffice do both interoperate with the XLS format! Can you really load and save those proprietary XLS files on linux or with Java? You absolutely can! If you haven't tried the POI libraries for Java or the OpenOffice application suite, you should check them out to see the power of representation.

The HTML Lesson

A good representation stands on its own, without imposing hard contracts about "a document containing X must relate in way Y to a particular system Z". A representation that requires an ongoing relationship with a running system is not fulfilling its role.

Consider what we have seen in this article:

1. A good value object implements no behavior.
2. An undocumented file format beats documented interfaces for interchange.

A third example, of course, is HTML itself, the Internet's great example of a victory of representation over encapsulation. HTML is a standard that follows in a long tradition of hypertext innovations starting with Vanevar Bush's original memex vision and Ted Nelson's Xanadu.

However, HTML improved on those early concepts in a key way. In those early concepts, a "hyperlink" was always conceived of as something that was guaranteed to be consistent, in other words, that must always point to a valid target document that actually exists somewhere else in the system.

HTML is incredibly successful partly because, as a representation, it does not require hyperlinks to be consistent. Every HTML document can stand on its own, regardless of the state of the various systems (the millions of internet servers) that happen to be deployed. A valid HTML document is allowed to have a "broken" hyperlink. If HTML documents were not allowed to have broken hyperlinks, then the process of maintaining the consistency of all HTML documents would be impossible to solve.

By eliminating the "behavioral" requirement of cross-document hyperlink consistency, and replacing it with the show-everything-with-no-promises "representation" of HTML with visible hyperlink URLs, HTML got the following benefits:

1. It is now possible to delete an HTML page from the Web without telling everybody.
2. It is now possible to harvest hyperlink relationships in ways that might have been unanticipated by all the server administrators.
3. It is now possible to design new kinds of URL organization for dynamic generation, tracking, or other applications without upgrading all the web browsers in the world.

If HTML had imposed rigid, system-specific behavior requirements on the humble hyperlink, the modern Internet would not have happened.

Good Representations are Loners

The success of the design for HTML comes from the following principle: the meaning and validity of an HTML document can completely understood by looking at just that one document as it stands alone.

Although an HTML document can certainly refer to many other things on the internet, none of those references is a promise or a requirement. A hyperlink is not a guarantee, but an advisory that "if you can resolve this URL, it may be useful to you." So fundmentally an HTML document is a self-contained unit, and it is still valid and useful even if the whole rest of the Internet is down.

This self-contained characteristic of good representations is universal. A good value object or a good document format is self-contained. A good packet format allows each packet to be read by itself. A good written sentence represents an idea that can stand alone.

The lesson? Good representations work without context. A good representation stands on its own.

Posted by David at November 28, 2003 06:25 AM