How to Make Our Project Successful?

Here is a longish post on why open source projects fail. We have just got started. I guess it's going to be a challenging but exciting ride. There is a lot of unknown. But I do believe we have the critical ingredients of vision and implementation. But we need to make sure community process works smoothly.

The Web Way

Interesting quote from this post:

The Web Way is a philosophy toward Web-based services:

1. They should be as simple as possible, but no simpler.
2. They should have clean designs for user interfaces and clean designs for programming interfaces.
3. Where it's useful, they should embrace REST.
4. Where it's useful, they should embrace loose coupling.
5. Where it's useful, they should embrace glorious, nonblocking, asynchronous pubsub. ;)
6. Where it's useful, they should embrace microformats, a/k/a lowercase semantic web.
7. Where it's needed, they should embrace the time-tested principles of Scalable Internet Architectures (three simple rules: optimize where it counts, complexity has its costs, and use the right tool).

We are moving in that direction :). I have been thinking about REST, continuation, Cornerstone Action, caching, session and forward propagation model / dependency caching all together these days and realized we may have the best practice of taking all of them into account and implementing a scalable, easy to develop and evolve system architecture.

AspectWerkz R2

One interesting piece of technoloy we need to closely look at is AspectWerkz. Release 2 does load-time (as opposed to compile-time (e.g. AspectJ) and run-time (e.g. DynAOP)) weaving and has excellent performance while not requiring precompiled code or a different language. It can also work with different aspect implementations.

Closure and Continuation

Many good languages, such as JavaScript, Lisp, Smalltalk, etc., but not Java, support closure, which is basically a function (withthe bindings for its parameters) that is a first-class value that can be passed around into other functions/methods. Closure gives the developer tramendous power in writing simpler code.

According to Guy L. Steele (of Common Lisp and Scheme fame),

A Closure needs to retain information about lexical variables, but not return-address information. A continuation needs both.

So a continuation is basically a closure with the extra information of next address to go to. Supporting continuation is quite a hot topic because it greatly simplifies page flow development.

In Cornerstone, an invocation is a closure. When an invocation is put into an invocation controller, it is equivalent to putting it in an expression with lazy evaluation: only when the path of the invocation is executed, the invocation is invoked. Also, an invocation controller basically implements a new language contruct that is otherwise not available in Java. For example, the IfThenElse controller is equavlent to Smalltalk's Boolean>>ifTrue:else: method.

Not only that, a Cornerstone action has an exit, which is the next address to go to. So an action (a special case of invcation) is equivalent to continuation (which is a special case of closure with its extra next address).

Therefore, we inadverdently added support for closure and continuation to Java with invocations and actions, although not directly to the Java language. We need to explore deeper support of continuation in Cornerstone to make web development as simple as possible.

Atom API

The Atom API sounds a lot like what we want to achieve for the Web of Apps, except it is designed for content. Maybe it can evolve into something for communication among applications. Need to dig into it and learn from it.

Blogger, Picasa, Hello and Google

So far the experience with Blogger has been pretty good. I recently bought a Nikon D70 digital SLR camera. It has already changed my view of family photography (my 2.5-year-old son is of the golden age of being all natural without stopping to pose). Some features that set it (and other fine digital SLRs) apart are: instant on, (almost) zero shutter lag and fast continuous shooting (not to mention great quality optics). I realized the volume of pictures in general and sharable picture in particular that I will produce from now on will sky-rocket soon because there is no need to delete anything (storage is cheap) and you can just fire away continuously (no shutter lag and continuous shooting mode). So I need some good software to organize and share my pictures (mostly of my 2.5-year-old son) without hassle. Because I am a Blogger user, it's just so natural for me to try Picasa and Hello (all Google properties). Just installed them and will start trying them out.

What do they have to do with Cornerstone Brainstorm? Well, a lot. They are all Google properties that work well with each other. They all empower the end user to be able to do things and do things to each other easily. What we want to add to the mess is the angle of apps. Writing a little app should be as easy as posting on a blog. Sharing apps with each other should be as easy as pointing permlinks at each other. I envision a day when I view your app on Blogger, move a magnifying glass over your app to see what kind of inputs and outputs it takes and produces, grab them into my blog, add my logic (by picking from a vast repository of prebuilt templates) based on yours and hit Publish App. Then my app is done, which anyone can also point to and use.

Haloscan commenting and trackback have been added to this blog.

Interesting: Driect Web Remoting

Looks like many people are getting into calling server from JavaScript. DWR is just one example, XMLHttpRequest another.

Food for Thought on Customization

Read this post just now. Points to ponder:

Somewhere in all of this there's a balance to be struck. I don't figure any single framework or whatever could be the solution to all problems. There will be some things it can do which save you money but once you're outside of its sweet spot I suspect any cost savings evaporate pretty quickly.

And if that's the case, what does that mean for the long term life of a system? What amount of change can we expect to see? Might it not need reworking on top of a new framework to suit some new need? Or perhaps we have to take a modular approach and build new functionality separately on top of a new framework and integrate it with the old system? What does all of that mean for architecture?

We probably want to think it over on how to integrate with other frameworks, not necessarily assuming them underneath the wings of Cornerstone, which has been our approach, because it might just be true that I don't figure any single framework or whatever could be the solution to all problems.

XMLHttpRequest

We have been using a hidden iframe as the mechanism to send request to server in the background. Here is potentially a better way: XMLHttpRequest. It is supported on IE 5+, Mozilla and Apple's Safari. When we have time, we should investigate whether it simplifies our client-side coding.

Groovy Dying?

Whatever happened to Groovy? The JVM desparately needs a dynamic language (maybe several) that is simple in the syntax, rich in functionality and integrated directly with existing libraries. Does Jython fills the bill already? Got to investigate it.

Securing Invocations

We can easily add security checks to before invocations are invoked. We may consult JAAS and see what kind of API they have. In a nutshell, we associate a secure invocation with a list of principals (each an individual or a group) and give the association priviledges.

End-User Programming

While we are on the subject of language and making it easy for end-users to program, I think we may be moving into a 2-tier world in respects to programming.

• Tier 1: The traditional software professional type who write Cornerstone invocations (services, actions, business rules, templates, etc.) which may require good programming expertise and experience.
• Tier 2: The casual users who just compose or orchestrate existing invocations created by Tier 1 people to put them together in a way suitable for the situation at hand, which should not require much programming expertise in the traditional sense. Many visual aids and metaphors can be utilized here. The user may be just dealing with, e.g., sources that provide data, sinks that consume data and service compositions/orchestrations in the middle to tie them together. For example, user's particular calendar is the union of two other calendars from the same family, in which case the 2 sources are the two other calendars and the sink is his own calendar and the service in the middle is union.

The roles Cornerstone plays in this picture are: 1) make it simple for the tier 1 developers to create invocation; 2) make it simple for tier 2 developers to tie them together. Tier 2 users will be the ones that create the Web of Applications (ref 1 and ref 2).

PS: Some people are advocating that we start over. Interesting to watch.

The Web of Applications (2)

Today Jon's Radio has a piece on Mozilla Calendar. His conclusion:

Bottom line: this solution is workable, but not great. While Calendar could improve matters somewhat, it's ultimately just awkward to push whole calendar files around.

My church has been in the need of a calendar server for a long time. I haven't found something that is easy. I was thinking of the Mozilla Calendar and WebDAV solution. It's good that Jon has tried it and found it to be insufficient. So I don't think I will touch it for a while.

On the other hand, his wishes:

I wish there were a middle ground between this model and the dedicated calendar server. Imagine a WebDAV implementation that could map collections not just to whole-file resources, but to XML elements within files. Given an XML flavor of the iCalendar format, you could achieve more finely-granular control over calendar data. But the same model would work for other applications using other XML formats. And you'd have the option of searching with XPath or XQuery, again leveraging infrastructure not bound to any application domain.

I always envisioned WebDAV working this way. Maybe someday, when filesystems and XML databases merge, it will.

The idea of The Web of Applications goes along the same line but on a different level. We don't really integrate at the data level but rather at the application level. Think of applications as small pieces of logic (services) that work on data. In the example of the calendar, a subset of the whole web should be the global calendar. Each small calendar (personal, group, etc.) has permlinks to other events in other calendars. The interfaces between the calendars are of the service nature, not just at the low data level.

Significance: It's in web's distributed, decentralized, P2P model and you can only expect huge growth out of this model unless you are a Google. If each little piece (in this case, it's the calendar; in other cases other applications) can be as easy as a blog (Web2), then the growth of The Web of Applications can be exponential, just like when people started to created HTML pages (Web1) and link to each other.

Our vision: People will create small applications of all sorts and link to each other. This is the Web of Applications.

The Web of Applications

The web has been extremely successful. Web1 is a web of HTML pages, some broken, but mostly useful and robust. Web2 seems to be at least about a web of blogs, linked by permlinks. Web3 may be about a web of applications.

	Web1	Web2	Web3
web of	HTML pages	blogs	applications
link	URL	URL, permlink	URL, permlink, REST style webservice (optionally looked up through a service)
data type	content	content	data
unit	page	blog	(small) application

Result of webservices may be cached. The web of applications is the biggest implementation of Forward Propagation Model (otherwise known as dependency caching). One common feature of these 3 generations of web is the units are all small, can be created by mere mortals and yet number in millions.

The user model of web of applications should be very simple. It's like the tip of the iceberg. Users don't see and don't care about the foundation underneath it. And Cornerstone is the foundation.