The (unofficial) WBI StoryWe didn't start out to build a programmable HTTP proxy. We started out to build something that would simply remember where you'd been on the web from one day to the next. We started with the "memory browser" and ended up designing a new way of thinking about and building web applications with intermediaries. Intermediaries are like agents but they don't have to be smart. Intermediaries are like proxies, but they don't have to be dumb. But let's not get ahead of ourselves....
People learn and remember but browsers don't do a very good job of this. And way back in 1995, web browsers did an even worse job of keeping track of your history than they do today. So we thought about this problem of keeping track of web browsing, and we figured that the proxy was a pretty good place from which to spy on the user --- didn't require client code, and didn't require intimate knowledge of the browser.
A series of post-it notes followed in which we sketched out the architecture of a proxy that we could plug computation into. We reasoned that we could do anything we wanted if we could just monitor traffic, edit the returned document, and generate documents ourselves. MEGs were born. Of course, if we'd thought it up in the reverse order (Generate, Edit, Monitor), we'd have a better acronym. But MEG stuck.
So around Christmas, 1995, one of us wrote the first version of WBI in about a week --- in Perl. It was quick and dirty and slow, but it did it all. It was even multithreaded. Of course, we hadn't come up with the name WBI yet, so it was called the Memory Browser, which later mutated into Personal Web Agent (PWA). This last name clung to us for many months. We went around and gave many demos, showing off history and traffic lights and other simple functions to IBM executives and others.
In the beginning of 1996, we started reimplementing the programmable proxy in C++, presumably for speed but at the cost of substantial programming effort. Many months went by, and it sucked up the full time effort of three fairly competent programmers. It was originally written for OS/2 and then ported to Win32. And after much anguish, we hit on the name WBI by flipping the letters IBM over and having the audacity to pronounce it "webby".
We worked feverishly to produce a clean little proxy that people could install on their machines and that would provide simple web history functions. We grappled with all sorts of problems. For instance, where do we put the links that you click on to get to the history functions? Will people mind if we add extra markup to the pages they view, such as a toolbar at the top? And what do we do with framed pages? What about illegal HTML? How do you modify that robustly? And all this is on top of the basic infrastructure for proxying and for plugging computation into the proxy.
But eventually, that summer, July 1 to be exact, we released the first version of WBI on alphaWorks --- it was one of the first three not-quite-ready-for-primetime technologies to be made available there. We were thrilled. We wrote a paper about WBI for the First International Conference on Autonomous Agents. We went on vacation. Not all together, though our boss wanted us to do that too. But that's another story.
Bug reports flowed in rapidly and we worked hard to keep up. Many people downloaded it, installed it, and it immediately sucked up all the cycles on their machines. We tried to fix that. The one big thing we learned from this release was that people will download anything, but that you only get one chance to impress them. Because of the bugs, many people uninstalled WBI and never tried it again. But it turned out that we gained quite a nice following in the OS/2 community.
With our success --- that we had actually produced a program that tens of thousands of people downloaded in the first few months --- came the offer of help from outside. Well, outside of research, but still inside of IBM. A programming group in Endicott, New York, took over development of WBI, freeing us for more academic pursuits. It gets really cold in Endicott in the winter, and we know because we've been there.
WBI development was taken over by Endicott, and they fixed many bugs and improved it substantially over the next year or two. They took the original C++ code and made wrappers so that you could write plugins in C and Java with essentially the same interfaces. That was pretty cool. Finally Endicott produced an all-Java version of WBI. At first, it too was compatible with C and C++ plugins, but maintaining this was a nightmare and we all moved into the brave new world of Java. It was 1997.
The Endicott team spent time thinking about the "business model": How're we going to make money on this thing? They surveyed people on whether they'd buy the history functionality for 30 bucks. Sort of. But it didn't seem like that big a business, from IBM's point of view anyway. And the thing is that WBI is more than a single plugin --- more than history or traffic lights --- it is an architecture for building intermediary applications.
IBM is a big company. It has hundreds of thousands of employees and makes billions of dollars a year. It didn't get that big by taking frivolous chances. We learned this firsthand as we could not convince very many executives or others inside IBM that we had a really nice scheme for programming web applications. It was particularly suited to applications for which you don't have control of the data. We reasoned that infrastructure and plumbing is IBM's bread and butter, so WBI should be a natural. But alas, researchers' wild-eyed visions are not always exactly in line with corporate business planners' bottom lines. However, both species often live in a symbiotic relationship.
Along the way, servlets appeared and IBM took to that. Servlets are fine, but they are missing a few things that WBI can do. It is pretty much a matter of perspective. We think about the wire and the data flowing along it. Servlets (and servers in general) think about the data and producing it. In particular, WBI and servlets differ in at least the following:
One project we can tell you about was created by an IBM company called Edmark, which makes educational software for kids. It was one of these funny situations where one day these Edmark folks asked us for the WBI code and we sent it. Nine months pass. We get a harried call from someone at Edmark asking a technical WBI question, which luckily we could easily answer (actually, we already had a patch for that bug). But when we said we didn't even know they were still using WBI they said, "What do you mean 'using it'? It ships in our product tomorrow!" Stunned, we learned that they had used WBI to build a web filtering system that allows parents to provide kids with an appropriate selection of web destinations, KidDesk Internet Safe. The PageFilter sample plugin provided in the current WBI Development Kit is based on this idea.
Last year, WBI found another niche inside IBM as a programming framework for transcoding, which is a fancy way of saying transforming data from one form into another. The key is that data are in whatever form they are in, say a GIF or XML or something, and the user who wants the data can only view if it is in a certain form, say HTML on a monochrome screen, and some process transforms it from the first into the second. Looks like a job for intermediary-based computation to us. And it seems IBM thinks so too.
We now have a new development team in Raleigh, North Carolina that is building Transcoding Technology based on the WBI framework. We have included in the WBI Development Kit a sample WBI plugin which does transcoding of XML to HTML using XSL. IBM is also involved in other transcoding projects that look pretty exciting.
So in the end, what is WBI? Well, many people think of WBI as the Personal History functions included in our previous alphaWorks download. This understanding is completely reasonable since we called that program "WBI". Others have looked at various WBI plugins we've built and think of WBI as an intelligent agent, a security and privacy tool, a transcoder, or many other things. But we, the creators of WBI, think of it as the framework that supports intermediary programming. And it turns out that thinking about and building intermediaries is a pretty fun thing to do.