r4 - 25 Nov 2003 - 08:54:46 - AzaZelYou are here: OSAF >  Projects Web  >  CodeInfrastructureIssues? > WhyNotZodb

Why is Chandler not using ZODB?

Bill Seitz:

Are there any references online (wiki pages, list archives) discussing the decision to eliminate ZODB, and a short explanation of what's being used instead? (Or maybe there are multiple non-ZODB back-ends possible, all hidden behind RAP...)


Andi Vajda:

The reasons are multiple but none are exactly a ZODB show stopper. Ultimately it was a decision I made based on the constraints of the project.

  • ZODB is a python object database. While Chandler is written in python, I wanted its data model to be independant of any language in particular and needed it therefore to be more constrained than python's.

  • By making the data model more constrained and by controlling it there are many things that are made easier, in particular queries, interactions with RDF, bi-directional referential integrity, access control, garbage collection, etc...

  • There are also a number of requirements the Chandler designers created for the repository that are made easier to implement if we control the actual representation of the data and constrain the data model, such as replication, remote access, shared access, threaded transaction model, etc...

  • When I started working on this project in May, the repository was late, very late, and the project was stalled because of that. I felt I could get something usable for the project to resume much faster if I started a data model implementation from scratch and persisted it using Sleepycat's Berkeley dbxml and Berkeley db.


Today, the Chandler repository is not really so much an object database as an item XML database combined with large collections of references directly stored in Berkeley DB. Chandler persistence is grafted on top of whatever language binding is chosen to operate it (currently only Python is supported). By that I mean that a programmer's python instances are never persisted in the Chandler repository, only specific attribute values the programmer intended for Chandler.

The trade-off for these decisions and design choices is a somewhat steeper learning curve for programmers expecting a real object database like ZODB. My hope is that this trade-off is well worth the gains.

-- HeikkiToivonen - 05 Nov 2003


The first point you have written here really matters, but probably you personally feel better with bdb than zodb?

Switching from an unconstrained db to a really low level one... seems quite strange to me...but I'll see later if your is good decision.

-- AzaZel - 11 Nov 2003


Someone who wishes to remain anonymous sent me (Ducky) email that said:

I developed and used a Zope site for two years. The impression I had of Zope was as a fundamentally broken tool. Please note that my experiences deal with an older version of Zope (ca. 2000-2001; I have a poor sense for exact dates) and that I rarely dealt with ZODB as a separate entity. Also, some issues listed under "Zope" also belong to "ZODB" and vice versa. Please also note that Zope made it difficult for me to deal directly with ZODB. Additionally was the problem of having my Zope server provided by a zope-hosting service, so I didn't have complete and unrestricted access to the server. Therefore your mileage will vary on these points.

Zope weaknesses:

  • Web-based editing is as fun as sorting through 100,000 email messages using Yahoo mail.
  • Nearly non-existent session management.
  • DTML is evil. It completely negates the advantages of using Python as a language. For example, only a small number of Python symbols are available for use by the developer and those are set by the Zope designers.
  • Using Python functions requires a large amount of coding overhead, similar to using Java. Again limiting the expressiveness and conciseness of Python.
  • Documentation is horrific. What hasn't been documented is criminal. What has been documented is worse.
  • No test suite capabilities for Zope objects.
  • Search capabilities were primitive. No search and replace. Search itself was limited to simple word search. No searching by project.

ZODB weaknesses:

  • Everything is kept within the database, making them inaccessible to the tools and scrutiny Unix programmers have come to know and respect from file system-based applications. No grep, no sed, no visual inspection. Note that this is an area I hope OSAF keeps in mind when choosing data storage formats as it's a mistake that could be repeated using other storage toolkits. People and enterprises will want alternative methods for viewing their data. This is why the mbox file format has been so persistent for so long. In other words, if you use a non-readable file format you'd better be prepared to also supply your own command line tools for doing greplike things.
  • Primitive search capabilities.
  • Object indices must reside within application code itself.
  • Badly documented (not quite as bad as the main app, but still pretty grisly).
  • Prone to corruption. This happened to me several times so I was left with the impression that ZODB was fragile. Not much in the way of recovery tools (at least then).

Apocryphal point regarding ZODB which I myself haven't tested:

  • Legend has it that ZODB performs well in an environment where there are few writes vs. many reads, but performance suffers under heavier write loads. That's typical of most web-based environments. Indeed I believe it was smart for the database to be tuned for that kind of usage pattern (my own web app followed this pattern). But Chandler will be much closer to 1:1 read/writes and you will want an environment which is amenable to your own tuning.

-- as told to DuckySherwood - 13 Nov 2003


This seems almost misinformation to me, and i'm also surprised about someone that wants to remain anonymous... there is no problem for somebody that want to write to this wiki while remaining quite anonymous. And i'm surprised that you, Ducky, choosed to write souch things without verfy them. I'm using Zope since 1998 and I'll answer one to one to your points; i'll report in italic your points:

  • Web-based editing is as fun as sorting through 100,000 email messages using Yahoo mail.

Zope developer agreed with you, and it's the reason of existence for the FTP interface from the beginning, so you can edit objects with your preferred editor. There is also a WebDAV interface since some years and, if you use CMF, you can use DirectoryView? to keep your objects on the filesystem. I'm checking Zope's changelog right now and it says that WebDAV was added on version 2.0.1, dated 17 November 1999.

  • Nearly non-existent session management.

Session management is in, it was available as separate product to download and install since 2000, but it was shipped with Zope for the first time with the version 2.5.0, date 25 January 2002.

  • DTML is evil. It completely negates the advantages of using Python as a language. For example, only a small number of Python symbols are available for use by the developer and those are set by the Zope designers.

Wrong, DTML is a templating language, it isn't Python. Zope Developers set DTML simbols wich are, for example <DTML-IF>, <DTML-IN>, and so on... As you should agree, they can't be python symbols, for obvious sintax reasons. But you can use as much Python as you want into the expressions that you can pass to DTML's statements as the argument...but you should but it into double quotes. The only thing that is evil in DTML is it's namespace pollution due to the use of implicit acquisition, but you can use Page Template instead ;-), even in 2001...

  • Using Python functions requires a large amount of coding overhead, similar to using Java. Again limiting the expressiveness and conciseness of Python.

You should prove what you are saying... Python functions are just...python functions! Similar to using Java? What are you saying here?

  • Documentation is horrific. What hasn't been documented is criminal. What has been documented is worse.

Like other open source efforts, documentation is also a community effort...do you have written any documentation? However, DTML book, ZSQL book an an Administrators Guide where available since 1999. The first Zope Book was available the 30th of April of the 2001.. do you miss it? Now there is also a Developer book, more developer specific.

  • No test suite capabilities for Zope objects.

Neither a python's test suite was available in 2001. Now there is a ZopeTestSuite?, a CMFTestSuite? and a PloneTestSuite?.

  • Search capabilities were primitive. No search and replace. Search itself was limited to simple word search. No searching by project.

Search capabilities are primitive? you have to prove that... ZCatalog is one of most powerful tools avilable to search object hierachies... "No search and replace"? are you confusing a hierarchy of live objects with a chunk of text in a text editor? The search wasn't limited to single word search even in 2000.

ZODB weaknesses:

  • Everything is kept within the database, making them inaccessible to the tools and scrutiny Unix programmers have come to know and respect from file system-based applications. No grep, no sed, no visual inspection. Note that this is an area I hope OSAF keeps in mind when choosing data storage formats as it's a mistake that could be repeated using other storage toolkits. People and enterprises will want alternative methods for viewing their data. This is why the mbox file format has been so persistent for so long.In other words, if you use a non-readable file format you'd better be prepared to also supply your own command line tools for doing greplike things.

mbox has been persistent for so long because there wasn't no alternative. Really, it was a fear for adminitrators due to the potential corruption of using mbox with NFS. We are lucky now that we have alternative email's storages like Maildir and friends... The problem of the introspection is the same with SQL storages and even with other storage types like Berkeley DB. You are fool if you think to keep all your data in a single text file

  • Primitive search capabilities.

ZODB doesn't constraint you on the structure of your objects hierarchy, so having advanced search capabilities at ODB level is near impossible. Really no one of the ODB that doesn't constraint the developer on hierarchy structure has advanced search capabilities. But for examble, if you choose to use a Zope- like object hierarchy, you can use ZCatalog even without Zope.

-- AzaZel - 25 Nov 2003

View topic | Edit |  | WYSIWYG | Attach | Printable | Raw View | Backlinks: Web, All Webs | History: r7 < r6 < r5 < r4 < r3 | More topic actions...
 
Open Source Applications Foundation
Except where otherwise noted, this site and its content are licensed by OSAF under an Creative Commons License, Attribution Only 3.0.
See list of page contributors for attributions.