by Apache Cayenne Vice President Andrus Adamchik

As some of you may have noticed already, a major new release 3.0 of Apache Cayenne has been posted not so long ago. I am using this occasion to reflect on the state of Cayenne, mention the release highlights and outline our future direction.

State of Apache Cayenne

With a solid design, 9 years of active development, a dedicated community and a place among the family of the Apache Software Foundation projects, Cayenne is without a doubt a serious player in the Java ORM space. Moreover with the convergence of many existing frameworks around JPA specification, Cayenne stands out as a project that follows its own path. The foundations of Cayenne design are "rich" persistent objects and a clean separation of the mapping model from the Java code. These are the choices that were made consciously back in the day, and they survived POJO/annotation hype rather well, enabling smooth database access experience and a number of rather unique features not found (maybe even not possible) in other ORMs. Let me go through some of those distinctions here, even though most were available prior to 3.0 release:

Transparent and lightweight transactions. Cayenne ObjectContext does not require a transaction wrapper around it to talk to DB. Transactions are created transparently to the user and only for the duration of a DB operation. E.g. if you decide to read a relationship that hasn't been resolved yet, a transaction will be created with a scope of a DB select. In some other case you may change objects properties in memory, and since this does not require a DB operation, a transaction is not created. In other words ObjectContexts, their objects, and changes to those objects are not attached to any open JDBC resources and are also serializable. At the same time it is easy to hook up Cayenne stack to JTA or some other transaction mechanism if desired.

Context nesting. ObjectContexts can be nested, so commit/rollback at the object level can be hierarchically structured. E.g. you may change some object properties in a dialog window, commit back to the parent context attached to the parent window, without committing to DB. Or you may decide to roll back you changes made in a dialog. This can be done without rolling back the parent window context.

Remote Object Persistence. Nesting contexts is great, but there's more to it. A "child" ObjectContext can be located in a remote application, talking to a "parent" over a web service. Now that's a cool use of nesting! We are calling this feature ROP for "Remote Object Persistence". ROP provides a full set of ORM functionality in remote tiers of multi-tier Java applications. Our users have written some cool things using ROP.

Generic objects and dynamic mapping. Since Cayenne fully separates the mapping model from the Java code, and does not require class enhancement or annotation preprocessing, this gives a whole lot of flexibility as to what persistent Java objects can be. One way to take advantage of that is via generic objects. With generic objects, you may treat all or some of your entities as maps of values, only with support for relationships, queries and all other ORM goodies. Why would you want to do that? One real life use is to create an entire mapping in runtime. An application would load metadata of a database schema using Cayenne DB reverse-engineering API and generate a dynamic Cayenne mapping with entities mapped as generic objects. Later an application would query and modify these objects using a standard Cayenne API. A possibility of a fully dynamic mapping is quite different from other ORMs that require precompilation.

Modeling tools. A big help in day-to-day work is CayenneModeler - a cross-platform, IDE-independent GUI mapping tool. It frees you from the need to deal with raw model, provides support for various ORM-related DB operations, and ensures seamless upgrades between the versions of Cayenne. Modeler goes a long way in simplifying learning curve for the new users.

3.0 Highlights

3.0 has 225 new Jiras. This number excludes bug fixes, so it is all new stuff. There is a very long list of things that were added in 3.0 and it is definitely a must-have upgrade for our existing users.

Interestingly, some of the 3.0 additions that we'll mention below, came from toying with the idea of building a JPA provider on top of Cayenne runtime. This goal has been ultimately abandoned (yet again we came to a conclusion that Cayenne's existing design better serves our users). However looking at what others are doing in the ORM space was instrumental in improving Cayenne in a few areas. This is how we got lifecycle events, EJBQL query, Embeddables, etc. So what are the essential new things in 3.0?

The use of generics. As 1.5 is a minimal JDK requirement for Cayenne 3.0, we were able to switch many public APIs to use generics. This process will continue in the following releases.

Flattened attributes and Vertical Inheritance. In the past we supported "flattened" relationships, i.e. relationships mapped across more than 2 tables. Now we added a similar ability for attributes, Object properties can be mapped to columns from a joined table over one or multiple joins. This feature can be used for standalone entity mapping, but when combined with single-table inheritance, it gives us a classical vertical inheritance mapping.

Lifecycle Events. Now you can either declare a callback method on an entity object or declare a listener class to receive entity events, JPA-style. To add a bit more flexibility to this mechanism and decouple business logic often present in the event handlers from the mapping, Cayenne also allows to register listeners via API.

Pluggable Query Cache. To me personally this is easily the most important feature of Cayenne 3.0. This is what makes Cayenne scale to serve gazillions of requests. Application developers can optionally categorize their queries, assigning them one or more "cache groups". Each cache group can then be configured with certain expiration policy, using specific cache provider approach (e.g. OSCache config file). Forced cache invalidation is done by group, and has a very low overhead, even across an application cluster. A user may add calls in the post-commit listeners to flush one or another group when an object is saved. This feature has been available since the early 3.0 alphas around 2006-2007 and is a real life saver.

Object queries as Strings. The above mentioned EJBQLQuery is created from String, as opposed to traditional Cayenne SelectQuery that is created via API. That by itself is not such a big deal (my guess is that most users would actually prefer SelectQuery's object API to string concatenation), but EJBQL has many more capabilities, such as scalar and Object[] result sets, GROUP BY and HAVING. So right now this is the most advanced object query in Cayenne, if not the most friendly.

SelectQuery improvements. SelectQuery traditionally avoids a concept of a "join", which is foreign to the OO design (unlike say EJBQL which has explicit "JOIN" clause). In SelectQuery expressions everything is a path, that is later resolved to appropriate SQL joins by Cayenne. In 3.0 we added extra notation to support paths that resolve to OUTER JOINs (add a "+" sign after the path component, e.g. "addresses+.city" to match people that live in a given city or have no city information), and also a concept of split expressions to control join reuse in situations like "match all objects" (use pipe symbol "|" to separate a common path part from specific parts, e.g. "addresses|country.name" to create an expression matching people who have addresses in multiple countries)... Also SelectQuery now supports all kinds of joint and disjoint prefetching. Prefetching algorithms are optimized for speed.

CayenneModeler. In 3.0 Modeler got a serious facelift. We added undo/redo, copy/paste, contextual menus, model search, autocomplete, merging model changes back to DB, and of course support for mapping of all the new 3.0 concepts like callbacks/listeners, flattened attributes, embeddables, etc.

Performance improvements. Cayenne 3.0 has a significantly improved memory model. ObjectContext is using weak references to all cached but unmodified objects, so users no longer need to worry about creation of too many ObjectContexts, that are often stored in a session and may have leaked memory in the past. Query caches are LRU maps so they have a fixed memory footprint and won't grow indefinitely... Paginated query designed to efficiently fetch huge data sets are now made more lightweight, taking much less memory and much less time to initialize. Paginated results can scale to hundreds of thousands of objects in a single list (or maybe more, we haven't tried). Finally, commit concurrency and throughput of the framework is significantly improved.

Beyond 3.0

Cayenne is a mature and powerful framework, but by no means we think of Cayenne as a "finished product". There are lots of ways to make it even better. Planning and development of another major release 3.1 is well underway. The most exciting new piece that is already available on the SVN trunk is a small and easy to use dependency injection (DI) container. Yes, we know about Spring and Guice, but introducing a new complex runtime dependency looked like a bad idea. A presence of an embedded Cayenne DI container will NOT force a choice of an application-wide DI container or create a version conflict with third-part frameworks (if you are using Spring, just keep using it). Besides our DI container is only 35K in size, i.e. an order of magnitude smaller than any third party alternatives. A DI container made Cayenne configuration infinitely more flexible for us as Cayenne developers and for our users integrating Cayenne into their applications.

Another important thing is to take a hard look at the existing features that appeared in 3.0 as a result of our diversion into the JPA land and see how to improve them and make more "native" to Cayenne design philosophy. E.g. EJBQLQuery and SelectQuery need to be merged into a single query that provides both String and object APIs and supports the features of both current queries. As a part of that process we will also finalize the use of generics in queries.

Then there's a lot of other things being discussed on our mailing lists that we are not quite ready to announce yet. If you are interested, come join the Cayenne dev list and start asking questions :-)