{"body": {"type": "/type/text", "value": "### Infobase\r\n\r\nBuilding Open Library, we faced a difficult new technical problem. We wanted a database that could hold tens of millions of records, that would allow random users to modify its entries and keep a full history of their changes, and that would hold arbitrary structured data as users added it. Each of these problems had been solved on its own, but nobody had yet built a technology that solved all three together.\r\n\r\nSo we created _Infobase_, a new database framework that gives us this flexibility. Infobase stores a collection of objects, called \"things\". For example, on the Open Library site, each page, book, author, and user is a thing in the database. Each thing then has a series of arbitrary key-value pairs as properties. For example, a book thing may have the key \"title\" with the value \"A Heartbreaking Work of Staggering Genius\" and the key \"genre\" with the value \"Memoir\". Each collection of key-value pairs is stored as a version, along with the time it was saved and the person who saved it. This allows us to store full structured data, as well as travel back thru time to retrieve old versions of it. \r\n\r\nInfobase is built on top of PostgreSQL, but its interface is abstract enough to allow it to be moved to other backends as performance requires. The current schema of Infobase tables looks like:\r\n\r\n    Table site\r\n        id \r\n        name (string)\r\n\r\n    TABLE thing\r\n      id\r\n      site_id (references site)\r\n      key (string)\r\n\r\n      [(site_id, key) combinations are unique]\r\n    \r\n    TABLE version\r\n      id\r\n      revision (int)\r\n      thing_id (references thing)\r\n      author_id (references thing)\r\n      ip (ip address)\r\n      comment (string)\r\n      created (datetime)\r\n\r\n      [(thing_id, revision) combinations are unique]\r\n    \r\n    TABLE datum\r\n      thing_id (references thing)\r\n      begin_revision (int)\r\n      end_revision (int)\r\n      key (string)\r\n      value (string)\r\n      datatype ('string', 'reference', 'int', 'float', or 'date')\r\n      ordering (integer, default null)\r\n\r\nFrom Python, the infobase interface looks like this:\r\n\r\n    # retrieve the book object\r\n    foo = site.get('/foo')\r\n    assert foo.title == \"The Story of Foo\"\r\n\r\n    # query for books by that author\r\n    foos = site.things(dict(author=\"Joe Jacobson\"))\r\n    assert foos[0].title == \"The Story of Foo\"\r\n\r\nInfobase also has a [programmable API](/dev/docs/api), which can be used to build applications using the Open Library data.\r\n\r\n### Infogami\r\n\r\nBut simply building a new database wasn't enough. We needed to build a new wiki to take advantage of it. So we built _Infogami_. Infogami is a cleaner, simpler wiki. But unlike other wikis, it has the flexibility to handle different classes of data. Most wikis only let you store unstructured pages -- big blocks of text. Infogami lets you store structured data, just like Infobase does, as well as use infobase's query powers to sort through it.\r\n\r\nEach infogami _page_ (i.e. something with a URL) has an associated _type_. Each type contains a _schema_ that states what fields can be used with it and what format those fields are in. Those are used to generate _view and edit templates_ which can then be further customized as a particular type requires.\r\n\r\nThe result, as you can see on the Open Library site, is that one wiki contains pages that represent books, pages that represent authors, and pages that are simply wiki pages, each with their own distinct look and edit templates and set of data.\r\n\r\n### OL Technology\r\n\r\nBut infogami is also open to expansion. It has a rich plugin framework that lets us build exciting site-specific features on top of it. So we've added specific Open Library technology to help us handle things like the search engine. We also hope to develop plugins to handle reviews, price checking, and other important features to the site.\r\n\r\n### Find out more\r\n\r\nThere's a lot of exciting new technology here and we suspect it will be confusing at first. But there are places you can go for help:\r\n\r\n * [About the architecture](/about/architecture)\r\n * [Developer information](/dev)\r\n * [Join the technology mailing list](http://mail.archive.org/cgi-bin/mailman/listinfo/ol-tech)\r\n * [Read the source code](http://www.openlibrary.org:9021/)"}, "title": "About the Technology", "last_modified": {"type": "/type/datetime", "value": "2009-03-04T04:42:32.014605"}, "key": "/about/tech", "type": {"key": "/type/i18n_page"}, "id": 9888057, "revision": 3}