Nothing To See Here

Exchanging my Kindle for another Sony Reader

Mon, 16 Jun 2008 09:12:37 EDT

I replaced my Sony Reader with a hipper new Amazon Kindle, and I'm totally disappointed. The Kindle has a few good ideas, but it is terribly executed. I'm returning it today (something Amazon is making pleasant enough) and buying another Reader.

Read on for more detail...

About 6 months ago, I bought a used Sony Portable Reader from a friend of a friend. I travel a lot, and have a lot of open-format stuff I wanted to read (pdfs, html manuals, etc). I've always through electronic ink was interesting technology, and buying used appealed to my cheaper inclinations.

Fast forward to a a month ago, and having an electronic book has changed my life. The Sony fits in my bag nicely, reading it is quite pleasant, and I can easily carry 50 books with me at any time. It has changed the way I read, allowing me to have several books going at once and switch between them based on my mood, something I used to do in high school. libprs500 (now Calibre) is pleasant to use on the mac and open source. I even used the Sony software to buy a few closed-format bestsellers at reasonable prices.

Unfortunately, my Reader broke, possibly due to it's previous owner or just due to being dropped. It will no longer hold a charge or boot properly. I took it apart, following helpful Flickr directions for prs-505 disassembly, but to no avail. If anyone has a project that would benefit from the eink components, which seem undamaged, let me know.

Since I have a vacation coming up, and I'd so enjoyed the experience of having one, I decided to replace it. And being a consummate gadget shopper, I went looking for other options. I settled on the Kindle as the most advanced option, attracted by it having internet access and a way to buy and load books that would require Windows. So I went ahead and primed one.

I received it the morning I left for a trip to New York on Acela. This gave me some time to get started using it. My first reaction is that it is really bulky. It barely fits into the pouch of my laptop bag, while the Reader is about the same size as the Moleskine I keep there all the time. The Kindle came with a leather cover, but it doesn't really fit into it well, and that just ads more bulk. The cover has a big elastic to hold it closed The on/off switch is on the back, where it is blocked by the cover. In contrast, the Reader has a cover that clips on, is held closed by magnets, and a switch on the top that is easy to access.

The Kindle could survive all these misfeatures, if the reading experience were good. It is not. I don't know how they tested this thing, but anyone with normal sized hands can't find a way to hold it without accidentally hitting buttons. They claim it can be held with both hands, but I have to rest it on a surface, or hold it gingerly with my right hand only. The big fear of the designers seems to have been that people couldn't find the next page button, so they made it take up one whole side. A second next page button and a previous page button consume the other side. Which means if you want to hold the book naturally in one hand, you are going to accidentally palm those buttons, and lose your place.

The keyboard, which is necessary to make the buying and web experience possible, also takes up a lot of space on the device and risks accidental key presses. There is really only one point on the front of the device you can safely rest your thumb. And holding the same position for long periods of reading isn't any fun.

On the bright side, the EVDO is a cool feature, and I like the idea of devices that come with infinite bandwidth. The scroll-wheel menu interface is also good, certainly better than the Sony ten-button menu interface. But these don't make up for the difficult reading experience. And with a new iPhone coming, I'll have a portable web browser anyway. I just need my electronic book to be a good book, not a digital convergence device.

I'd love to see a better E-Ink book than the Sony Reader. But the first generation Kindle is certainly not it. Don't buy one.

Buy It Like You Mean It and Vendor Relationship Management

Tue, 3 Jun 2008 21:14:27 EDT

I just got back from the launch party for Buy It Like You Mean It, a startup non-profit that is "enabling the socially conscious consumer". I'm a big fan of what they are doing. As a free-market capitalist, I like to think that the power of markets can solve all kinds of problems. As a realist (and Wall Street technology vendor), I realize that market actors can have wildly different information and expertise.

Consumer goods suffer greatly from this problem. They are produced and distributed by large and complex organizations. Consumers, particularly in traditional retail settings, have little to go on but what it says on the box and the brand. And in recent times, brands have become commodities themselves, with everyone from Martha Stewart to Sesame Street selling their name. In order to make this market work, we need better technology at the point of sale.

Enter Buy It Like You Mean It.

Send them a barcode (via SMS or otherwise) and they will send you back a customized report about that product and how it aligns with your personal political or social inclinations. This information is crowd-sourced, Wikipedia style, from what I imagine will become an active community of corporate watch dogs. In the short term, they are focusing on chocolate, to build a complete database and work the bugs out of their system. But I'm sure it won't be long before they branch out. And probably build an iPhone application.

I'm a huge fan of the application of technology to improving market dynamics, and especially of applying technology to level the playing field between consumers and corporations. Since the first time I heard the idea, I've been excited about Vendor Relationship Management (VRM), the dual of the ubiquitous Customer Relationship Management (CRM). A lot of this stuff is still in the conceptual stages, such as ProjectVRM at Harvard. Buy It Like You Mean It is a refreshingly concrete application of the same ideas.

I imagine a future in which self-organized blocks of consumers, aided by software, can effectively hold their own against even the largest vendors. Chinese "team buying" (Tuangou) is an early example of this. I look forward to using technology to share information with other consumers about my credit card company, auto mechanic, airline, or hotel. Actually, I can already do that. So instead I look forward to using technology to digest and aggregate all the freely available information, and to even more directly manage my relationship with these vendors, all the way to making purchase decisions for me.

Much appreciation to Taza Chocolate for hosting the launch party. They are a stone ground chocolate factory in Somerville, making ultra-high end chocolate. If you have a chance to tour their factory, I highly recommend it.

kdb+ now available for free download

Sat, 12 Apr 2008 10:05:24 EDT

If you've talked to me about programming languages and Wall Street in the last 4 years, I've probably mentioned kx. This is a company which makes a combination programming environment and database based on a language called q which is derived from APL. (Yes, APL, the language invented in 1957 before there was a computer to run it on.) And this environment is in turn used by many of the top quants on Wall Street (and other parts of the financial world) for both research and production systems. Becoming a kx programmer is a good way to double your salary and quadruple your job security.

Well, it's been going around my corner of the blogosphere that kdb+ is now free for personal use. I first heard about it from Marc Adler. You can go download it from the kx download page. This represents a big step towards openness, which I think will be good for everyone.

The q environment is impressive, you have to give them that. There is an emphasis on brevity; the OSX binary of kdb+ is only 227K. That's smaller than the ncurses library it ships with. And brevity doesn't stop there. Utterances in the language are well known for their complexity and impenetrable internal logic. A lot of q code makes obfuscated perl look clear and verbose. It doesn't help that the culture of kx programmers discourages commenting. Error handling is tricky at best, and modularity and maintainability are in short supply. For confusion, q adds a bunch of SQL keywords on top of the previous language, k, in an almost but not quite fully compatible way.

But for all the faults you can kind find some really interesting features in q. And if nothing else, it is an example of a novel programming language, tightly integrated with a data management system, finding commercial success, which is always nice to see. And if you learn it, you might find a sweet job on wall street as a result.

When neither a column store nor a row store is the answer

Fri, 11 Apr 2008 22:34:00 EDT

A few days ago I found myself giving database advice to a friend with a new startup. His problem is a pretty common one: he has a very large corpus of data, over which he will run compute-intensive proprietary algorithms. Both the data and the computation will require a cluster of machines. He has a prototype based on Postgres. His question to me: should I continue to use a row store (Postgres) or should I use new technology, specifically a column store?

If you read the title of the post, you can already guess that my answer was neither.

Traditional row stores are optimized for transaction processing, in which records are updated. They are also sufficient for small data sets and ad hoc queries, as you find in a normal web app. Column stores are optimized for ad hoc queries, but this time against very large sets of data.

In either case, the data store is designed as a shared resource to cope with IO bound operations. But for many applications, the database is neither shared nor IO bound. In this application, the large data store is used by a small set of applications (the data loader and the data reducer), which are long running with predicable access patterns. This is in sharp contrast to the kind of load databases are designed to support.

The answer is to look at much simpler tools like BigTable, which are optimized for precisely this kind of access pattern. BigTable stores data in columns, but spends considerably less effort optimizing the IO of the data. Instead, it assumes the application will access the data in a reasonably efficient way. It's great if you have a single application or single kind of application working over your data. In fact, BigTable even interoperates well with Map/Reduce, which is a reasonable basis for the compute intensive part of this application. The Hadoop implementation of these technologies looks pretty good.

There is an ad hoc query component to the application, but it is ad hoc queries of the results of the data reduction. The results are effectively a pre-aggregation of the large data set. There is a two-tiered storage model already built in to the design.

Now, an idealized database management system, the kind that you get along with your flying car, might make this two tiered design unnecessary, or at least automatic. It ought to be able to index the huge data set and support queries against it, notice patterns in those queries and store pre-aggregated information in order to service common queries quickly. But no readily available system comes even close to that level of self management. In fact, you would have a hard time finding a DBMS that gives you really rich and pleasant to use semantics for maintaining materialized views, never mind automatically creating them.

So, for now we build knowledge of these limitations into our system architecture, take a two tiered approach, and can benefit from the good work done by Google at building a very large, very dumb data store.

Google commences another assault on the traditional database community

Tue, 8 Apr 2008 09:32:19 EDT

Many in the blogosphere noticed when Stonebraker and Dewitt at The Database Column took offense at the idea that map-reduce is the solution to many of life's problems. The idea that a simple idea, promoted by a services company, can blow away 20+ years of distributed database research, bothers them for some reason. Sure, it may not be the best solution to every problem, but it is a sufficient solution to many problems.

Google is quietly at it again, this time with the Google AppEngine.

This time, they are taking a shot at transaction processing. This is particularly amusing, because Stonebraker himself has been predicting an upset in transaction processing for a few years now. But rather than demonstrate tight integration between a app language and the database optimizer, or use traditional stored procedures, Google is again causing problems by building the simplest thing that can possibly work. And simple developers with simple applications are going to like it.

The Google App Engine data store is a non-relational database, with support for an SQL-like but limited language called (unsurprisingly) GQL. There are no joins. The lock management is simplistic. It is basically an entity-relationship database, with fixed and variable attributes associated with each entity. Every query is always a SELECT * ("The SQL syntax is retained for familiarity").

The whole thing is wrapped in a Python object-mapper, which means it looks quite a bit like an object database with a simplified model. This turns out to be what most people who are building web applications, whether in PHP, Ruby on Rails, or Java (Hibernate). Delivering a database which is limited to this functionality puts google in a powerful position to optimize specific use cases for simplicity and performance.

We are at the start of (or possibly in the middle of) a cycle that we haven't seen in over 10 years: Existing assumptions about databases are being questioned, and a thousand flowers are blooming. Stonebraker himself pointed this out in his One Size Doesn't Fit All papers, talks, and columns. However, it is not just the Oracle of the world who need to change or get out of the way. It is also the leaders of the database research community and their dominant paradigms. Things are going to be messy for a while yet. But in the end we will have technology that better fits modern infrastructure and modern usage.

Bungee Connect: What I learned about platform evaluations

Sat, 12 Jan 2008 21:50:13 EST

Catching up on email this Saturday, I got the opportunity to take a survey about my experience evaluating Bungee Connect. Evaluating bungee connect has been something that has popped onto my radar several times. They purport to be an application platform for web applications. They have their own language, their own ui toolkit, their own (browser-based) IDE, and their own code library and source control system. In my day job I create and sell a novel application platform (for streaming applications), so I like to look at new platforms on many levels. I signed up, and was shortly invited into the bungee beta.

Suffice to say the evaluation didn't go well for me. It isn't clear that there were any problems with the product. But I kept getting blocked by other things in the evaluation. I had trouble logging in. Trouble figuring out their programming paradigm. Trouble deciding what the tool is good for. And more.

After taking their survey, I decided to capture for myself what I learned about the my own bungee connect evaluation process.

Knowing what a tool is good for is non-trivial. It helps to give me good examples of problems I would solve with this tool.
Even better is to give me problems I could solve with this tool that will impress other people. Like if I just spent an hour with this on the couch, I should be able to show my wife something. Or my boss.
Samples are really good for learning. Especially samples that can be extended into boss-impressing applications.
Exercises are also a good thing. They are better than samples for getting me hands-on.
Videos are a really slow way to transfer information. I'm an early adopter of programming languages. This means I learn at least one new language per month. I don't have time to sit through a 5 minute video about hello world. If you can't explain hello world in a page of text, you have a problem.
Rather than a video, give me an exercise and a cheat sheet. The cheat sheat should give me instructions for completing the exercise, in excruciating detail (I can skip the obvious parts, since it is just text).
If the exercises are long, or build on one another, give me Julia Child-style opportunities to start with a mostly-complete version of the application.
I'm really busy. During the signup, wait for email, click link to confirm cycle, you are almost guaranteed to lose my attention.
I'm really busy. Things that timeout are at odds with that. Development environments that timeout in a matter of minutes are basically unusable.
Programs are made of files. Anything that tries to hide the files, or make the files too opaque, will make me uncomfortable. Are you listening Smalltalk?
Offering hosting is good. But being the only place I can store my software is scary.
Web services are cool, but the number of public web services is not yet impressive. And their documentation leaves something to be desired. If your software depends on third-party services, document the heck out of those too.
Twenty dollars at Amazon is sufficient incentive to get me to take a survey.

That's the braindump for now. I've already noticed areas where we at StreamBase could be doing better.

Information Wants To Be Half Price

Sat, 12 Jan 2008 09:32:00 EST

If you haven't heard already, Steve Jobs is expected to announce on Monday that iTunes will begin offering 24-hour movie rental for $4. This is widely reported not just by traditional rumor sites, but by people like Salon. And, by and large, people have been complaining about the price. Including me. Digging deeper, I have an armchair economist answer as to why this price feels wrong.

I am a long time Netflix customer and former Netflix shareholder. I got out of the stock because I don't think they are positioned to win in online-rentals. The recent announcement with LG about a set-top box might help, but it's far from a solution. Right now, customers have yet to find a business model they really like. And so companies are serving up a wide variety of business models, hoping to find one that works. As a result, we are in a tornado of different pricing and contract terms. Now Apple, who has gotten this right before, has thrown their hat in.

The goal with all these different business models is for the company to make as much money as possible. Company bosses are likely to see the price of their brick and mortar competition as a baseline. Ideally, they would like to charge more. They provide additional convenience, they say. Ticketmaster is a success story here, charging additional fees for purchasing online. But they sell one of the remaining actually-scarce goods, tickets to live shows in capacity-limited venues. And they are an effective monopoly. And everyone hates them. An online movie provider cannot win by charging extra for convenience.

Customers don't accept the brick and mortar price as the right base price. For customers, the base price is free. And claims of convenience fall on deaf ears. It's quite often more convenient to steal films than to acquire them legitimately online (bad websites, DRM software, advertising). Advertising models might work for older content, but not for big-budget Hollywood films. An online movie provider cannot win (or stay in business) pricing at free.

This leaves frustrated customers and frustrated industry players. Taking a page from Getting to Yes, I went looking for objective criteria to decide the fair price for digital movie rental, based on the current market price for retail movie rental, and the differences that result from going online. I'm going to assume that contract terms will be achieved that for most customers are equivalent to retail rental, and that a software/player stack will be created that offers sufficient convenience.

Customers currently pay about $4 to rent a movie at Blockbuster. But there are a number of additional components of the cost to consumers, beyond simple price. Important components are:

$4 in cash
Travel costs to get to the store
Risk that they won't have the movie
Risk that something else will come up between the time you rent and the time you watch
Risk that the disc will be damaged
Risk that the pimply-faced-youth will frustrate you with incompetence

As you can see, most of these costs are removed by the convenience of the online rental. "Great," says Hollywood et al, "we can charge them more and capture all the value." Well, hold on a second. Let's look at the costs to the rental store:

Cost of acquiring the rights and media for films that get rented
Cost of films that don't get rented
Cost of floor space, fixtures, bricks, mortar
Wages of pimply-faced-youth

As you can see, many of those costs are lower too. "Great," say customers, "we should get access to more movies more easily for less money." Ah, a classic two-party negotiation just waiting to have objective criteria applied.

Pricing is about supply and demand. Pricing is about finding a market-clearing price, where there are equal numbers of buying and sellers at that price, so all goods change hands, and both profit and utility are maximized. At least that's what I learned in Econ 101.

Soft-costs of buying are reduced for buyers. This means that at every price point (lower and higher than $4), there will be more buyers. Production costs are also reduced. So there should also be more supply at every price point.

That means that if the market were working properly, prices would go down, and consumers would be happily buying more of a better product. Instead, prices are staying high, products aren't better yet, and consumers are getting frustrated and opting out of the whole thing.

The punch line is that on the internet, information doesn't need to be free, as Stewart Brand claimed. People are willing to pay. But in the mind of consumers, when you remove the overhead of moving about physical goods, and you are trying to set a fair price, Information Wants to be Half Price.

Book Review: The Change Function

Sat, 1 Dec 2007 20:11:49 EST

Most business books have only one good idea. In second-class business books, it's common to name the book after the idea, in the hopes of building some brand recognition. The Change Function fits right into this cliche, providing one good idea, the change function itself. Helpfully, Pip Coburn has the good taste not to pad is his book with many other ideas. Instead, he applies his one good idea to several case studies. If you just want the one idea, read the rest of this post (information wants to be free, after all). If you want case studies and details about applying the idea to an organization, pick up Coburn's book.

Most new products fail to succeed. They fail because they are not adopted by customers. Adoption generally fails because users don't change their habits to use or purchase the new device or service. People generally avoid change. Creators and purveyors of new products must find a way to motivate change. The focus of this book is on how to understand when users are willing to change.
The Change Function states that the willingness of users is a function of two parameters: Crisis and TPPA. (Readers with mathematical ability will have to forgive Coburn's limited grasp of mathematical notation and terminology). Crisis is a motivating event or condition. Without Crisis, people will keep doing whatever they have been doing. TPPA is Total Perceived Pain of Adoption, which is a good phrase. This is how difficult people expect that the change will be, which is only tangentially related
TPPA comes in many forms. The most obvious is price. But generally price is not most important. More important source of Pain might be the need to learn new things, or the possibility of the system not working out and leaving the buyer feeling stupid. Pain comes from political or career risk. And pain comes from fear of the unknown.
Crisis can also vary. The crisis for purchasers of HDTVs or iPods is a fear of being left out. The crisis for some iPhone purchasers is a need for "internet in my pocket". For others, the iPhone is a way to purchase coolness. Regulatory changes can drive a crisis, for people or businesses. So can life changes, like a new baby.
TPPA can be lowered. Improving usability reduces TPPA. Try-before-you-buy is another popular technique. Allowing users to start small and build up confidence reduces their risk. Initial user experience should deliver minimal pain. And the perception of pain must also be managed, through marketing. If users think the product is difficult to use, or prone to breakage, or many other intangibles, these can be managed with appropriate messaging.
Crisis can be created or discovered. Marketing to CEOs can create a crisis, if they fear their company is being left behind. Marketing to individuals can create desire for a new experience or new device, a kind of crisis. Crisis that already exist can be exploited by modifying the product to address them. Finding a motivating Crisis is how many products that were "before their time" become mainstream. A killer application is often a kind of crisis.
Users (generally unconsciously) weigh the TPPA versus their own Crisis. This sounds obvious, but is pretty fundamental stuff. It means, for example, that If You Build It, They May Well Not Come. For developers of new technology, from eBook readers to programming languages, ask yourself what the motivating crisis is for your users, and whether it exceeds the TPPA. If it does, you need to find different users, create Crisis, or lower TPPA.

OOPSLA 2007: New Ideas, Good Quotes, and Cynical Self-Satisfaction

Sun, 28 Oct 2007 08:26:00 EST

I'm still in Montreal, and still thinking about a lot of what I saw and learned at OOSPLA. But I wanted to jot down some links while the cool things were still fresh in my mind. So here is a short set of links worth following that I picked up while at OOPSLA.

There was of course the workshop on associations as a first class language construct, which I already blogged about he r e.
As part of that, Jiri Soukup talked about his project IN_CODE and in particular his UML Layout tool, which attempts to provide sufficiently good graphical layout of UML that the machine-generated layout can be the only layout. The theme of graphical languages with a single canonical display was recurring. I picked it up again from Jonathan Edwards (author of Subtext), who has been working on a canonical visual representation for conditional constructs, and presented it as No Ifs, Ands, or Buts. While I missed the talk, I like the ideas.
I saw a demo of AmbientTalk/2, which is a cute language that makes event-driven programming the norm. The big idea is that all objects belong to a specific thread of control (an actor), and their methods can only be executed in that thread. At method call sites the caller explicitly calls either synchronously (with a '.' operator) or asynchronously (with a '<-' operator). These ideas may have originated in earlier languages with which I was not familiar (E was mentioned). This definitely buys a chunk of attractive simplicity. However, the demo failed to present any complicated synchronization patterns. I'm forced to wonder if we need better benchmarks for the expressivity of parallel languages. Building an IM system just isn't very hard.
Erik Meijer's Confessions of a Used Programming Languages Salesman was pretty clever, mostly talking about his experience trying to bring fire down off the mount to Visual Basic programmers. Computer Scientists need regular injections of reality when it comes to things like industrial adoption (usually non-adoption) of their ideas. It was also one of several talks at OOPSLA about LINQ and how awesome C# 3.0 is. I'm definitely envious of some of the features of C#, and I hope Java catches up soon (1.7 is looking pretty good in some ways, but certainly not all). Erik talked a lot about the extension points added to C#, but I'm at the moment unable to find anything on the web that tells me how to add LINQ-style keywords to the language for arbitrary languages, like say an extended version of SQL.
The Domain Specific Languages panel was entertaining, but neither conclusive nor particularly inspired. It was the first place the phrase "Silver Bullet" came up, but far from the last. Unfortunately (and ironically) everyone seems to have a different idea of what a Domain Specific Language (DSL) is, leading to wildly different concerns. More on this in a future post.
Sean McDirmid's talk and live coding demo of SuperGlue was the most impressive new thing I saw at the show. SuperGlue is an event driven language and a live programming environment. As you edit programs (in Eclipse, of course) the program is continuously executed and re-executed. Sean built up a game of Pac-man, Julia-Child-style. As each new block of code was added to the program, the game gained new behavior. Even, thanks in part to eclipse, responding to typing successive characters of a numeric constant, like "330". I really hope that he makes his code, or at least videos, available. Unfortunately, I'm afraid that Sean's demonstration and discussion of the benefits of live programming overshadowed his more theoretical contribution of object-oriented signalling. His paper from ECOOP 2006, SuperGlue: Component Programming with Object-oriented Signals may discuss that in more detail. Sean is unlikely to keep working on SuperGlue, which is implemented in Scala on Eclipse, as he is going to work for Microsoft.
Sharing the stage with Sean, linked in the minds of the organizers because both talks had "live" in their title, was Living in the Comfort Zone, which proposes restricting program inputs using a "rectifier" in order to avoid exposing programs to inputs that, while legal, might be unexpected and trigger bugs. Interesting idea.
The Tuesday night performance of 50 in 50 by Guy Steele and Dick Gabriel was indescribably awesome. I expect a video, and will post a link when it becomes available. Various people compared it to Steele's 1998 performance-essay Growing a Language, which I had not previously seen and which I can now recommend (you may not understand what he is getting at until page 4).
Tony Kay presented a way to punctuate transactional-memory transactions to make them nest, which just further convinced me that transactional memory is far from the solution to the difficulty of threaded programming. Their model might be better, but it is at best a patch which solves only some of the problems caused by an insufficient underlying abstraction.
StreamFlex was definitely interesting. Inspired by StreamIT and other stream processing work, it provides an events-and-operators model not unlike Aurora (the prototype for StreamBase), where the operators are kernels of Java code. The main contribution, to me, was the approach to real-time semantics. They instrumented their virtual machine to provide some simple transactional memory features, and then use them to assure real-time performance of the real-time thread. All method calls into objects shared with the real-time thread are logged using transactional memory. In any conflict between the real-time thread and another thread, the transaction is aborted for the non-real-time thread. However, to make this work they introduce a somewhat complicated object-ownership system, enforced by compile-time typechecking. Combining the real-time approach with the object ownership of AmbientTalk I mentioned above might be interesting.
Thomas Kuehne tried to answer the question Can Programming be Liberated from the Two Level Style?, but I remained unconvinced. The two level style means classes and objects. In many programs, the "item-description pattern" is used to allow a richer set of "is-a" relationships than the class hierarchy generally affords. While I support the idea of turning design patterns into language features, the mechanics of DeepJava failed to inspire me with a desire to teach them to the average 6.170 student. Inheritance is tricky enough without meta-inheritance.
No Silver Bullet Reloaded was the Panel Of The Stars at OOPSLA, with Fred Brooks, David Parnas, and other panelists who would have seemed famous in other company. Martin Fowler, in particularly, put on a fabulous performance as The Werewolf (who spent much of his time gloating). The forgone conclusion was that no, there had been no successful silver bullet in the 10 years since Mythical Man Month was updated. As for discussion of what we should do about it, the general idea was that we should do all those good things software engineering pundits have been telling us for years, and that we should make sure to employ smart people. Interestingly, panelists from industry were the ones pushing for employing only smart people, which seems like an ineffective strategy when you are trying to develop software in the large at a company like Cisco. The attitude of cynical self-satisfaction ("industry just needs to do what we already told them to do, which they will never do") seemed distressingly pervasive at OOPSLA this year.
I caught the end of a talk by John McCarthy about Elephant 2000, a new language which he has been developing for over a decade. I didn't really follow his description of Elephant, and after talking to others about it I think it is ill conceived. The second question was, roughly, "Could you talk about your experience inventing LISP instead?". McCarthy seemed happy to do that, and what followed was 30 minutes of entertaining anecdotes about early computing.
There were a number of interesting projects presented only on posters at the conference. Chris Pickett of McGill presented Software Speculative Multithreading for Java, which not only contained several interesting ideas, but was the most interactive poster I've seen (photo here), being made of movable felt pieces that could be used to talk through demonstrations. The general idea was that at each method call site, come up with a result predictor, and spawn a speculation thread that will continue the computation as if that result had been returned. Speculation can be running several levels deep at a time. This required significant VM modification, which he is now turning into a library so that it can be added to the IBM JVM. There is also a lot of work to be done on heuristics for choosing call sites for speculation, and choosing predictors.
Talking to people about startups versus large companies, I got an interesting anecdote from someone involved in VoIP at Nortel: In their estimation, it takes them 2 years from project start to get a product into the marketplace, while it takes their startup competitors only 1 year. However, after 2 years they have a product which meets the needs of their customer base, while it takes the startup 3.5 years. Makes me wonder what these constants look like in other product areas.
I attended the Seaside BoF. I find a lot of the Seaside technology very interesting, but as befits their roots in the Smalltalk community, they have a lot of difficulty marketing themselves. I ended up serving as a resident heretic, which I hope was helpful, as their main topic of discussion was how to make Seaside adoption as easy as Ruby on Rails adoption. My heresies included "What do you mean I have to use a special smalltalk editor instead of emacs?", "Are continuations really scalable? Why can't I get a straight answer?" and "How am I supposed to choose a runtime/database/editor/etc when I don't know anything about Smalltalk yet?" This turned into some extensive discussions about the adoption curve of destabilizing development tools, which I will try to write up later.
I had a slow morning on Thursday and missed Context, Perspective and Programs, which for many people was something of a religious (or drug-induced) experience. That's what I get for not reading speaker bios in addition to abstracts. Hopefully there will be a video.
I sat in on a talk titled Neovictorian Computing, which had a lot of good quotes, but didn't say a whole lot. The best quote, which reflected a lot of what went on at OOPSLA, was "The pedagogy of software engineering is almost entirely about guilt." The speaker is the creator of Tinderbox, a tool for keeping personal notes. The thrust of the talk was that we need to empower our users to be craftspeople, and to produce small software systems that meet their own unique needs. Tinderbox, as described in the talk, seemed potentially the right system for my contact management needs (I've been unsatisfied with all off the shelf contact managers). Unfortunately, I found it difficult to use and not nearly as interesting as the talk had made it seem. Maybe it is the best we can do, when targeting end user programmers (the tool is used mostly by authors to manage book notes), but I would hope we could do better.
I had a couple of talks with Canadians about how their country could benefit from more startups. Unfortunately, most encouragement of startups seems to be government funding, which I think is fundamentally flawed. More on this later.
The final talk of OOPSLA was Brian Foote, revisiting his paper Big Ball of Mud, about the commonness of evolved software systems. He is a very good speaker, and it was an entertaining talk. His focus was on finding the right metaphors to communicate how something is working. My favorite was "sustainable development", suggesting that many deadline-driven software projects take unsustainable shortcuts, much like some farmers in the Amazon, prematurely depleting a programs maintainability. But he was also quick to point out that some techniques that software engineering pundits disdain, like cut and paste, are a fundamental part of nearly all software development, and that when you let reuse become the overriding concern, you can end up with something highly inefficient, like the Space Shuttle (the reuse of the machine, not the software, in that case). He also described at length about how we are losing the "War on Bugs", how attempts to crack down on domestic production have merely driven bug production overseas. Programmers start with "gateway bugs" like unchecked error codes and undocumented exceptions, and move on to harder bugs like cross cutting dependencies and threading. Foote went on like that for a while. I may just pull the quotes out of my notes and post them separately.

That's it for my train of consciousness notes from OOPSLA 2007. It inspired a lot of thoughts, so you may be hearing more from me on some of these topics.

OOPSLA 2007: "The Popularity Cycle of Graphical Tools, UML, and Libraries of Associations" - not the workshop I expected

Tue, 23 Oct 2007 09:59:24 EDT

Today was my first day at OOPSLA 2007 in Montreal. After a brief exposure to the amazing "Underground City" (really a shopping mall that is infecting downtown like a cancer), I crashed a workshop for which I had not signed up, and had a completely different experience than I had planned at The Popularity Cycle of Graphical Tools, UML, and Libraries of Associations.
The title implied it was the workshop about graphical programming languages and tools. As an author of a graphical programming language, StreamBase StreamSQL EventFlow, it seemed like the place to be. My plan had been to pick up some contacts in the graphical languages space, and learn about what has come before. However, the participants (with one vocal exception) had already accepted as a given that UML would eventually decline in popularity, to be replaced by a and they were focused on what would come next.

In fact, the general agreement about what comes next is first class associations. An association is a relationship between two classes that relates instances of the first with instances of the second. These are the lines in UML diagrams. The focus of the workshop was on creating language constructs to allow realization of these lines (associations) in code and the creation of libraries of types of associations.
In UML, most boxes are realized concretely as a Class. This makes translation to and from UML diagrams and other models straightforward for classes. But most lines are realized in an implementation dependent and error prone fashion. For example, there might be a pointer from objects of one class to the other. Or there might be pointers in both directions. Or a set of pointers, or an externally stored map.
Any of these implementations requires maintenance logic, generally embedded in one or both class definitions. This logic is error prone and can get out of sync. If the association is subject to additional constraints, such as a graph being acyclic or conforming to some security model, the maintenance logic can get very complex. At some point a separate class might be created just to track instances of the relationship, but this is a discontinuous change to the program, and impacts all clients of the relation. Much better if there were a concrete realization of associations somewhere in the program, where changes to the association could be made.
Three concrete implementations were presented, for C++, Java, and C#. The C++ implementation, by Jiri Soukup and called incode, was implemented via code generation. The Java and C# systems, presented by James Nobel and Kasper Østerbye respectively, utilized Aspect Oriented Programming (AOP) tools to inject required logic into classes at runtime. Kasper's work is described in a paper, Associations as a Language Construct, James's in Relationship Aspect Patterns. Stephen Nelson also described but did not present his work, and Mike Heffernan presented an in-house system he has developed at OpusEdge.
The OpusEdge system, in particular, focused on using associations and the objects they related to control database schema generation and database access. In this case the associations and objects form a Model in the context of Model, View, Controller. His system, implemented in .NET, provided transparent network access to the model from clients, as well as server side access.
While this was not the workshop I had intended to attend, it was all very interesting to me. The interplay between databases and programming languages, and between the relational model and object oriented models, is a subject which holds long-term interest to me. The question of whether associations should be binary or N-ary is similar to the question of RDF versus fully-relational in databases.
If we expand language functionality to enable associations, or just let the AOP beast out of its cage all together, it would greatly improve the rapid-development experience. Today, many MVC frameworks create degenerate views given a model, and some even take a programming by exception approach to generating more interesting views. WIth associations as first class manipulable constructs, the model can be enriched with more information (by using a library of association subtypes). This can in turn drive better automatic behavior in areas such as view generation, model validation, and persistence.
We are going to continue conversations about associations, electronically, in a forum moderated by Stephen Nelson. I will be interested to see what progress is made. I have two burning questions: How can we leverage associations to improve database interactions? And, once we have associations, what traditional programming language constructs can we drop?