Planet CouchDB

April 17, 2014

Joan Touzet

Understanding race-induced conflicts in BigCouch

Distributed databases with a near-real-time multi-master configuration – such as BigCouch, coming soon to Apache CouchDB – must deal with the potential of simultaneous modifications of a single resource. While the approach taken by multiple single-machine Apache CouchDB servers using regular HTTP replication is well understood, the situation changes a little bit when dealing with BigCouch-style internal replication inside a cluster.

I think it’s time to have a better understanding of what this means, and what impact this has on you as an application developer. Most of the time, there’s no change – to your app, a BigCouch-style cluster looks and feels like a single Apache CouchDB node. But when making near-simultaneous writes to the same document from different clients, you may experience document conflicts that you wouldn’t have with an Apache CouchDB 1.x single server.

How does this happen? Bear with me – this gets a bit complex. Hopefully this diagram will help.

The sequence diagram below depicts a scenario where two very closely spaced writes to the same document in a 3-node BigCouch cluster will lead to a document conflict.

Sequence diagram showing Sequence diagram illustrating a document conflict introduced by near-simultaneous writes to a 3-node cluster.

Sequence diagram illustrating a document conflict introduced by near-simultaneous writes to a 3-node cluster.

In this example, the database cluster is a 3-node cluster, with default settings of n=3, r=2 and w=2. (This means that a successful write to a document must write 2 copies to disk before an HTTP 201 status code is returned.) Client 1 and Client 2, both external client processes talking to the cluster, are both trying to update /db/doc, which is currently at revision 3. Client 1 is trying to write rev 4-1, and Client 2 is trying to write rev 4-2.

For the purposes of this example, we are going to state a specific ordering for document writes, and treat the write as being processed serially in a specific order. In reality, writes are issued in parallel with no coordination of writes. The scenario as shown is simply one of the possible event sequences you may run into, in the wild.

Client 1’s write is being mediated by Node A. For this example, Node A issues writes to nodes A, B and C, in that order. Client 2’s write is being mediated by Node C. For this example, Node C issues writes to nodes C, B and A, in that order. Both Clients’ first writes succeed (Client 1/Node A and Client 2/Node C) and return the internal equivalent of an HTTP 201 status code.

Arbitrarily, we say Client 1’s request to Node B arrives prior to Client 2’s request to Node B. (One will always be processed before the other; it doesn’t matter which one gets there first.)  Node B makes the write of revision 4-1, and returns a 201 equivalent to Node A.

At this point, Node A has 2 successful write responses back. Since the cluster is configured with w=2, a 201 is returned to Client 1 and the client disconnects; its work finished. The third write, already issued by Node A to Node C, will eventually be handled, the conflict recorded to disk and a 409 sent back to Node A. Note that both copies of the document are kept, capturing the document conflict on Node C when this write occurs. Node A’s work is now done.

Just after completing its write of revision 4-1, Node B then processes Node C’s write attempt of rev 4-2 from Client 2. This results in the conflict being written to disk, and returns the 409 equivalent to Node C. The same happens when Node C’s write to Node A is processed. Node C now has a non-conflict 201 response from itself, and the 409 responses from Node B and Node A, so it sends the client a 202 status.

At the end of the process, all 3 nodes have both versions of the document recorded, fulfilling CouchDB’s promise of eventual consistency.

Still with me? Good.

So which document “wins”? By design, the document with the higher hash value (the second part of the _rev token, i.e. _rev=##-hash) will win. If 4-1 and 4-2 were the actual _rev values, 4-2 would win. As such, there is no guarantee that the write with a 201 response will be the ‘winner.’ [1]

The closer together writes to the same document occur, the more likely it is that the cluster may still be processing a previous write when the subsequent write comes in. Even with resolution and DELETEs of the losing conflicts, document “tombstones” will be left behind on these leaf nodes to ensure replication results in eventual consistency (CouchDB’s guarantee! [2])

The best approach is to avoid these kinds of document conflicts via an access pattern where simultaneous writes are as unlikely to occur as possible. There are a number of resources out there on how to design apps this way, but one of my favourites is to never re-write a document, store all operations as new documents, and use a view or _all_docs and the startkey/endkey parameters to retrieve the latest state of a given resource.

Barring an application redesign, your CouchDB client should look for 202s and conflicts in documents consistently, and provide evidence of this result to the application layer. [3] You can also create a conflicts view to review all conflicts in a given database.

Resist the temptation to try and resolve the conflict in the CouchDB library! Only at the application layer can you best decide how to deal with a document conflict. You might initially choose to ignore conflicts, but probably it’s in your best interest to perform some sort of manual resolution and write a new, merged version based on data in all the conflicted versions.

If you’re still lost, post in the comments or on the couchdb-user mailing list and I’ll do my best to explain.


[1] https://cloudant.com/for-developers/faq/data/, “My write returned a 201, and yet I have a conflicted document -  what gives?”

[2] Doing better is a provably difficult problem. For reference, start with  L. Lamport’s Time, Clocks and the Ordering of Events in a Distributed System, Communications of the ACM 21, 7 (July 1978), 558-565.

[3] Take note: some CouchDB libraries, such as python-couchdb, do not differentiate between a 201 and a 202 response!

by Wohali at April 17, 2014 05:01 PM

The Official CouchDB Blog

CouchDB Weekly News, April 17

Blog Posts

Weekly CouchDB meeting – summary

  • Bigcouch Merge: work on branches has started again and will make good progress
  • rcouch Merge: code reviews have been restarted; more help with testing is needed; discussion around Windows support, will be continued on Dev-Mailing list
  • Advocate Hub: onboarding at the moment, next steps will be sent to marketing mailing list soon
  • 1.6.0 release: release candidate 3 (rc3) is being prepared now
  • COUCHDB-1986: the issue is being investigated, will be updated soon if its status changes

Major Discussions

1.6.0 changelog review (see thread)

A discussion around what exactly should be included in 1.6.0 (see changelog and documentation for details).

Handling the Welcome Request (see thread)

How to handle the welcome message that appears from a root request and direct the "/" request to another URL (setup: version 1.5 on a Windows machine).

authentication_redirect is not working (see thread)

How to handle the redirection of users after login to other databases that service page requests (and avoid errors; setup: public application with CouchDB-based authentication).

Vote: Release Apache CouchDB 1.6.0 rc2 (see thread)

The vote was aborted because of an issue brought up by the test suite.

Modeling Relationships and providing Transactional Integrity (see thread; ongoing discussion)

… and the question if _bulk_docs is transactional (hint: it's not)

Releases in the CouchDB Universe

Opinions

Use Cases, Questions and Answers

Get involved!

If you want to get into working on CouchDB:
  • rcouch Merge: Erlang hackers and CouchDB users, we need your help with testing and review of the rcouch merge. It's easy! Find the how-to in this post.
  • Here's a list of beginner tickets around our currently ongoing Fauxton-implementation. If you have any questions or need help, don't hesitate to contact us in the couchdb-dev IRC room (#couchdb-dev) – Garren (garren) and Sue (deathbear) are happy to help.
  • You want join us for the updates of CouchDB-Python for Python 3? Take a look at issue 231.
We'd be happy to have you!

Events

Job opportunities for people with CouchDB skills

… and also in the news

Posted on behalf of Lena Reinhard.

by Noah Slater at April 17, 2014 10:03 AM

April 16, 2014

The Official CouchDB Blog

Merging rcouch

rcouch is a variant of CouchDB that is derived from the main Apache CouchDB code base. One way to look at it is as a “future port” of CouchDB, that is for two reasons:

  1. The internals are greatly improved. In simple terms, if CouchDB would be started to day, it would look a lot like rcouch.
  2. It comes with a number of nice new features.

So whether you are a CouchDB user or a CouchDB Erlang developer, rcouch is good news for you!

Since CouchDB has been around for a while, the internal codebase is a little crufty. We are constantly working on improving that, but one of the larger efforts have been done within the rcouch project by Apache CouchDB committer Benoit Chesneau.

At the same time, Benoit added a number of nice little features that benefit CouchDB users, just to name a few:

  • _changes for views.
  • Replication from views.
  • Write-only “Drop-Box” databases.
  • Request validation on-read.
  • _db_changes (actually, we nicked that one for Apache CouchDB 1.4 already :).

For more information, see the rcouch website.

Benoit has generously donated the rcouch project back to Apache CouchDB and has worked on merging the rcouch and the CouchDB codebases. Now it is time to review the merged code and get it into Apache CouchDB proper and put it into a new release.

With that, we need your help.

How you can help, you ask? Easy!

If you are a CouchDB user, and are not afraid to build CouchDB yourself, give the rcouch merge branch a spin:

git clone https://github.com/apache/couchdb.git
cd couchdb
git checkout -b 1994-merge-rcouch origin/1994-merge-rcouch

# Read the DEVELOPERS file to install all dependencies

# test regular build:
make

# test build with static icu library
make icu=static

# test build with shared libraries
make libs=shared

# run test suite
make check

# make a release
make rel

# run:
./rel/apache-couchdb/bin/couchdb start

# now CouchDB is running at http://127.0.0.1:5984/_utils/, as usual, you just won’t see any output.


If you are an Erlang developer, we could use your help with code reviews. Any small chunks will help!

Please see the associated JIRA ticket for more information.

Thanks so much!


FaQ: how does this relate to BigCouch?

It doesn’t really, the BigCouch merge is ongoing and we will announce updates and requests for review when the time is right. Well, actually, if we can convince you to help with the review of the rcouch merge, we can put even more resources on the BigCouch merge :)

by Jan at April 16, 2014 08:50 PM

April 10, 2014

The Official CouchDB Blog

CouchDB Weekly News, 10 April

Releases

Apache CouchDB 1.5.1 released
This is a security release. Download link | Release notes

Blog Posts

Major Discussions

Writing many docs at once: bench marking a 10k write to a CouchDB server (see thread here)
A discussion around handling and measurements of bulk input in CouchDB.
Question about "complex" range queries (see thread here)
Requirements for sort orders of data views.
Is the revision field deterministic? (see thread here)
Quoting Robert Samuel Newson's reply: "Yes, it’s deterministic. The same document with the same history will have the same _rev value. This is an optimization over the previous algorithm where _rev was a random UUID. […] The advantage is that two servers receiving the same update can more optimally replicate. They still have to check that the target has all _id/_rev pairs but will usually be able to skip actually transmitting document and attachment content."

CouchDB Universe

  • Apache Con has taken place from April 7-9. You can find the slides of the talks given here

Releases in the CouchDB Universe

  • New Erlang Version is out
  • mbtiles2couchdb, for using CouchDB as a simple tiles server
  • RCouch has a new home: announcement | the new home
  • PPnet, a minimal social network to drop into websites based on PouchDB
  • sabisu, a sensu web UI, got open sourced by Cloudant last week
  • Janus, an online storage for offline Web Apps built using PouchDB
  • couche 0.0.2, a couchdb client for node, with specific apis
  • mock-couch, an http server pretending to be couchdb, for unit testing
  • couchdb-sync 0.1.1 - a generic couchdb replicator which is basically an eventemitter
  • hackoregon-couch 0.0.2 which exports database information from PostgreSQL and imports into CouchDB
  • overwatch 0.2.7, a deterministic couchdb replication watcher

Opinions

Use Cases, Questions and Answers

Get Involved!

If you want to get into working on CouchDB:
  • here's a list of beginner tickets around our currently ongoing Fauxton-implementation. If you have any questions or need help, don't hesitate to contact us in the couchdb-dev IRC room (#couchdb-dev) – Garren (garren) and Sue (deathbear) are happy to help.
  • Your help on updating CouchDB-Python for Python 3. Wanna join? Please participate here.
We'd be happy to have you!

New PMC Member

  • Joan Touzet joins the Apache CouchDB Project Management Committee today. Joan has made outstanding, sustained contributions to the project. Welcome to the Couch, Joan!

Events

Job opportunities

… and also in the news

Posted on behalf of Lena Reinhard.

by Noah Slater at April 10, 2014 10:59 AM

April 09, 2014

The Official CouchDB Blog

Apache CouchDB 1.5.1 Released

Apache CouchDB 1.5.1 has been released and is available for download.

CouchDB is a database that completely embraces the web. Store your data with JSON documents. Access your documents with your web browser, via HTTP. Query, combine, and transform your documents with JavaScript. CouchDB works well with modern web and mobile apps. You can even serve web apps directly out of CouchDB. And you can distribute your data, or your apps, efficiently using CouchDB’s incremental replication. CouchDB supports master-master setups with automatic conflict detection.

Grab your copy here:

http://couchdb.apache.org/

Pre-built packages for Windows and OS X are available.

CouchDB 1.5.1 is a security release, and was originally published on 2014-04-09.

These release notes are based on the changelog.

Changes

  • Add the max_count option (UUIDs Configuration) to allow rate-limiting the amount of UUIDs that can be requested from the /_uuids handler in a single request.

by djc at April 09, 2014 05:39 PM

Apache CouchDB 1.5.0 Released

Apache CouchDB 1.5.0 has been released and is available for download.

CouchDB is a database that completely embraces the web. Store your data with JSON documents. Access your documents with your web browser, via HTTP. Query, combine, and transform your documents with JavaScript. CouchDB works well with modern web and mobile apps. You can even serve web apps directly out of CouchDB. And you can distribute your data, or your apps, efficiently using CouchDB’s incremental replication. CouchDB supports master-master setups with automatic conflict detection.

Grab your copy here:

http://couchdb.apache.org/

Pre-built packages for Windows and OS X are available.

CouchDB 1.5.0 is a feature release, and was originally published on 2013-11-05.

These release notes are based on the changelog.

Changes

  • COUCHDB-1781: The official documentation has been overhauled. A lot of content from other sources have been merged, and the index page has been rebuilt to make the docs much more accessible. #54813a7
  • A new administration UI, codenamed Fauxton, has been included as an experimental preview. It can be accessed at /_utils/fauxton/. There are too many improvements here to list them all. We are looking for feedback from the community on this preview release.
  • COUCHDB-1888: Fixed an issue where admin users would be restricted by the public_fields feature.
  • Fixed an issue with the JavaScript CLI test runner. #be76882, #54813a7
  • COUCHDB-1867: An experimental plugin feature has been added. See src/couch_plugin/README.md for details. We invite the community to test and report any findings.
  • COUCHDB-1894: An experimental Node.js-based query server runtime has been added. See Experimental Features for details. We invite the community to test and report any findings.
  • COUCHDB-1901: Better retry mechanism for transferring attachments during replication. #4ca2cec

by djc at April 09, 2014 05:37 PM

April 08, 2014

The Official CouchDB Blog

CouchDB and the Heartbleed SSL/TLS Vulnerability

You may or may not have heard about the Heartbleed SSL/TLS vulnerability yet. Without much exaggeration, this is a big one.

What does this mean for CouchDB?

1. If you are using CouchDB with the built-in SSL support, you are at the whim of Erlang/OTP’s handling of SSL. Lucky for you, while they do use OpenSSL for the heavy lifting, they do the TLS/SSL handshake logic in Erlang (Source). That means you are not affected by this issue.

2. If you are using CouchDB behind a third-party proxy server you are at the whim of the SSL library it uses. For the big three Apache, nginx and HAProxy it’s all OpenSSL. So if they are using OpenSSL 1.0.1-1.0.1f with heartbeat support (RFC6520) enabled (the default), you need to take action. As far as I can tell now:

  • 0. Check if you are vulnerable
  • 1. Stop your service.
  • 2. Upgrade to OpenSSL 1.0.1g or recompile OpenSSL without heartbeat support.
  • 3. Request new cert from your SSL cert vendor.
  • 4. Revoke your old cert.
  • 5. Invalidate all existing sessions by changing the CouchDB couchdb_httpd_auth/secret configuration value to a new UUID.
  • 6. Restart your service.
  • 7. Invalidate all your user’s passwords and/or OAuth tokens.
  • 8. Notify your users that any of their data and passwords are potentially compromised.

by Jan at April 08, 2014 03:53 PM

The Little Things(1): Do Not Delete

CouchDB takes data storage extremely seriously. This usually means we work hard to make sure that the CouchDB storage modules are as robust as we can make them. Sometimes though, we go all the way to the HTTP API to secure against accidental data loss, saving users from their mistakes, rather than dealing with hard drives and kernel caches that usually stand in the way of safe data storage.

The scenario:

To delete a document in CouchDB, you issue the following HTTP request:

DELETE /database/docid?rev=12345 HTTP/1.1

A common way to program this looks like this:

http.request('DELETE', db + '/' + docId + '?rev=' + docRev);

So far so innocent. Sometimes though, users came to us and complained that their whole database was deleted by that code.

Turns out the above code creates a request that deletes the whole database, if the docId variable isn’t set correctly. The request then looks like:

DELETE /database/?rev=12345 HTTP/1.1

It looks like an honest mistake, once you check the CouchDB log file, but good old CouchDB would just go ahead and delete the database, ignoring the ?rev= value.

We thought this is a good opportunity to help users not accidentally losing their data. So since late 2009 (yes, this is an oldie, but it came up in a recent discussion and we thought it is worth writing about :), CouchDB will not delete a database, if it sees that a ?rev= parameter is present and it looks like that this is just a malformed request, as database deletions have no business requiring a ?rev=.

One can make an easy argument that the code sample is fairly shoddy and we’d agree. But we are not here to argue how our users use our database beyond complying with the API and recommended use-cases. And if we can help them keep their data, that’s a win in our book

Continuing down this thought, we thought we could do one better. You know that to delete a document, you must pass the current rev value, like you see above. This is to ensure that we don’t delete the document accidentally without knowing that someone else may have added an update to it that we don’t actually want to delete. It’s CouchDB’s standard multi version currency control (MVCC) mechanism at work.

Databases don’t have revisions like documents, and deleting a database is a simple HTTP DELETE /database away. Databases, however, do have a sequence id, it’s the ID you get from the changes feed, it’s an number that starts at 0 when the database is created and increments by 1 each time a document is added, updated or deleted. Each state of the database has a single sequence ID associated with it.

Similar to a rev, we could require the latest sequence ID to delete a database, as in:

DELETE /database?seq_id=6789

And deny database deletes that don’t carry the latest seq_id. We think this is a decent idea, but unfortunately, this would break backwards compatibility with older versions of CouchDB and it would break a good amount of code in the field, so we are hesitant to add this feature. In addition, sequence IDs change a little when BigCouch finally gets merged, so we’d have to look at this again then.

In the meantime, we have the protection against simple coding errors and we are happy that our users keep their hard earned data more often now.

by Jan at April 08, 2014 01:59 PM

April 03, 2014

The Official CouchDB Blog

CouchDB Weekly News, April 3

Major Discussions

Vote on release of Apache CouchDB 1.5.1-rc.1 (will be released as Apache CouchDB 1.5.1 — see thread)

The vote passed.

Importing CSV data into a CouchDB document using Python-Cloudant module (discussion still open — see thread)

Approaches brought up: (1) Transforming a CSV file into a JSON file with Python; (2) rc_scv, a direct rcouch extension; (3) using Google Refine and Max Ogden's refine uploader (see visual example here); (4) CSV2Couch. Further information can also be found in this blog post on "Using Python with Cloudant" and the newer Cloudant-Python interface

CouchDB 1.6.0 proposals (see thread)

Discussion around the open blocker plus how to deal with it and around re-cutting 1.6.x from master. Releasing 1.5.1 will take precedence.

Poll around Erlang whitespace standards (see thread; the poll is still open)

Joan Touzet: "I know many of you are fed up with not being able to auto format in your favourite editor and match the CouchDB Erlang coding standards, or receiving pull requests that are formatted poorly. I'd like to fix that with an appropriate whitespace standard, and supplementary plugins for vi and Emacs that will just Do The Right Thing and let us all stop worrying about whitespace corrections in pull requests."

There's currently a poll around this topic which is still open.

Multiple Concurrent Instances of CouchDB on Mac OS X 10.7.5 (see thread)

Approaches: (1) for a powerful enough machine Vagrant could be used to spin up a few CouchDB VMs, e.g. with CouchDB-Vagrant. Other options could be: (2) using Docker, (3) Node-Multicouch (used in Hoodie) or (4) this script to configure isolated instances, it should be possible to point couchdb to the CouchDB commands inside the .app.

CouchDB Universe

Releases in the CouchDB Universe

  • Wilt 3.0.0 – a browser/server based CouchDB API library based on the SAG CouchDB library for PHP
  • contentful-2-couchdb – a proof of concept for easy data export to CouchDB and replication use
  • Availability of MariaDB 10 announced
  • PouchDB released a new website
  • PouchDB 2.1.0 release includes e.g. (all release notes here):
    • Support optional leveldown builds
    • Replication performance improvements
    • Fix for localStorage detection in Chrome Apps
    • Improved error reporting from replicator et al.

Opinions

Use Cases, Questions and Answers

Getting involved into CouchDB

If you want to get into working on CouchDB: here's a list of beginner tickets you can get started with. These are issues around our currently ongoing Fauxton-implementation. If you have any questions or need help, don't hesitate to contact us in the couchdb-dev IRC room (#couchdb-dev) – Garren (garren) and Sue (deathbear) are happy to help. We'd appreciate having you!

New Committers and PMC Members

… and also in the news

Posted on behalf of Lena Reinhard.

by Noah Slater at April 03, 2014 10:19 AM

March 27, 2014

The Official CouchDB Blog

CouchDB Weekly News, March 27

Welcome to the Apache CouchDB weekly news! From today on, we’ll bring you the highlights of what happened in the CouchDB Universe each thursday.

Major Discussions

BigCouch vs. Couchbase
A comparison between the two, around conflict solving, revision docs, nodes, replicas and hashing. Find the thread here.
Healthcare projects running on CouchDB
Albin Stigö asked for healthcare companies / projects using CouchDB. In the thread, a collection of examples and reasons why this is a good fit was discussed. CouchDB works well in healthcare due to its relative ease of configuration and deployment, easy synchronization, backups, attachment-handling and notifications and mobile-readiness. Some examples of projects / companies using CouchDB: CommCare HQ, Mobius, Neurofoundation. Find his summary and the entire thread here.

Weekly CouchDB meeting – summary

  • 1.6.rc-2: discussion around blocker; will be collated and sent to the mailing list
  • Marketing: there'll be a weekly CouchDB news (which you are currently reading)
  • Fauxton - status: Futon will be removed as soon as Fauxton-implementation is done; following release will be marked as beta
  • Merges: discussion around BigCouch and rcouch merges will also be collated and sent to the mailing list

CouchDB Universe

Opinions

… and also in the news

Posted on behalf of Lena Reinhard.

by Noah Slater at March 27, 2014 03:29 PM

March 24, 2014

Joan Touzet

Hello world!

This is a test post for integration with Planet CouchDB, the aggregator for all things bloggy and CouchDB-related. I’m excited to be a part of the group!

by Wohali at March 24, 2014 01:02 PM

March 13, 2014

The Official CouchDB Blog

On The CouchDB Community

Jan Lehnardt sent a link to the CouchDB marketing@couchdb.apache.org mailing list where Ari Najarian used CouchDB in a five part course to teach beginners how to program. It’s a great read and I invite you to visit the page.

I would like to slice out one quote from the article where Ari described the CouchDB community:

“The personalities and voices in the CouchDB community are, hands down, the most helpful, generous, welcoming, insightful, ethical, principled, future-minded group of developers I’ve ever encountered in one place. As far as online communities go, you can’t find a safer, more encouraging group of advocates. These people are role models that will help beginners to understand what “open-source” can really be about: social justice, diversity, inclusion, and collaboration.”

This is the reason why I became a part of the Apache CouchDB project - because the community is so awesome.

by andywenk at March 13, 2014 04:49 PM

February 24, 2014

The Official CouchDB Blog

The State of CouchDB

This is a rough transcript of the CouchDB Conf, Vancouver Keynote.

Welcome

Good morning everyone. I thank you all for coming on this fine day in Vancouver. I’m very happy to be here. My name is Jan Lehnardt and I am the Vice President of Apache CouchDB at the Apache Software Foundation, but that’s just a fancy title that means I have to do a bunch of extra work behind the scenes. I’m also a core contributor to Apache CouchDB and I am the longest active committer to the project at this point.

I started helping out with CouchDB in 2006 and that feels like a lifetime ago. We’ve come a long way, we’ve shaped the database industry in a big way, we went though a phoenix from the ashes time and came out still inspiring future generations of developers to do great things.

So it is with great honour that I get to be here on stage before you to take a look at the state of CouchDB.

Numbers

I’d like to start with some numbers:

  • In 2013 we added 15 committers to the project, up to a total of 30. Thats 2x the number of people regularly contributing to CouchDB!
  • The year isn’t yet over, but these committers already created 3x the commits of 2012. And they have committed more than in any other year in CouchDB’s history.
  • We have shipped eight releases: 1.0.4 1.1.2, 1.2.1, 1.2.2, 1.3.0, 1.3.1, 1,4.0 and 1.5.0 just this year, that is up from one(!) last year.
    • thanks to our new release schedule we are getting more features to more people faster by focusing on small iterative changes forward.
  • 20% more JIRA tickets and 50% more GitHub issues

We have made a lot of changes in 2012 to make 2013 a great year for CouchDB and it sure looks like we succeeded and that 2014 is only going to trump that.

I’d like to thank everyone on the team for their hard work.

Currently

We’ve just shipped CouchDB 1.5.0 last week and it comes with a few exciting new things as previews, for you to try out and play with and report any issues with back to us. And that is on top of all the regular bug fixing and other improvements.

  1. A completely new developed admin UI, nicknamed Fauxton, that is poised to replace the much-loved, but increasingly dated Futon. I’d like to personally thank the Fauxton team: Sue “Deathbear” Lockwood, Russell “Chewbranca” Branca, Garren Smith and many more volunteers for their work as well as the company Cloudant for sponsoring a good chunk of that work. Great job everyone! Fauxton is going to be replacing Futon in one of the next few releases and will give us the foundation for the next stage of CouchDB’s life.

  2. Plugins. While it was always possible to write plugins for CouchDB, you kind of had to be an expert in CouchDB to get started. We believe that writing plugins is a great gateway drug to getting more people to hack on CouchDB proper, so we made it simpler to build plugins and to install plugins into a running instance of CouchDB. It is still very early days, we don’t even have a plugin registry yet, but we are surely excited about the prospects of installing GeoCouch with a single click of a button in Futon or Fauxton. We also included a template plugin that you can easily extend and make your own, along with a guide to get you started.  

    The plugins effort also supports a larger trend we are starting to follow with the CouchDB core codebase: decide on a well-defined core set of functionality and delegate more esoteric things to a rich plugin system That means we no longer have to decline the inclusion of useful code like we’ve done in the past, because it wasn’t applicable to the majority of CouchDB users. Now we can support fringe features and plugins that are only useful to a few of our users, but who really need them.

  3. A Node.JS query server. CouchDB relies on JavaScript for a number of core features and we want to continue to do so. In order to keep up with the rapid improvements made to the JavaScript ecosystem we have tentative plans to switch from a Spidermonkey-driven query server to a V8-driven one. In addition, the Node.js project has a really good installation story, something that we had trouble with in the past, and includes a few utilities that make it very easy for us to switch the query server over.

    All this however is not to blindly follow the latest trends, but to encourage the community to take on the query server and introduce much needed improvements. The current view server is a tricky mix of JS, Erlang and C and we are not seeing many people daring to jump into that. In a second step we expect these improvements to trickle down to the other query server implementations like Python or PHP and make things better for everyone. For now this is also a developer preview and we are inviting all Node.js developers to join us and build a a better query server.

  4. Docs landed in 1.4.0, but 1.5.0 is seeing a major update to the now built-in documentation system. With major thanks to Alexander Shorin, Dirkjan Ochtmann and Dave Cottlehuber who were instrumental in that effort, CouchDB now has “really good docs” instead of a “really crappy wiki”, that are shipped with every release and are integrated with Futon and Fauxton.

Beyond

The immediate next area of focus for the CouchDB project is the merging of two forks: BigCouch and rcouch.

BigCouch is a Dynamo implementation on top of CouchDB that manages a cluster of machines and makes them look as a single one, adding performance improvements and fault tolerance to a CouchDB installation. This is a major step in CouchDB’s evolution as it was designed for such a system from the start, but the core project never included a way to use and manage a cluster. Cloudant have donated their BigCouch codebase to the Apache project already and we are working on an integration.

rcouch is a what I would call a “future port” of CouchDB by longtime committer and contributor Benoit Chesneau. rcouch looks like CouchDB would, if we started fresh today with a modern architecture. Together with BigCouch’s improvements, this will thoroughly modernise CouchDB’s codebase to the latest state of the art of Erlang projects. rcouch also includes a good number of nifty features that make a great addition to CouchDB’s core feature set and some great plugins.

Finally, we’ve just started an effort to set up infrastructure and called for volunteers to translate the CouchDB documentation and admin interface into all major languages. Driven by Andy Wenk from Hamburg, we already have a handful of people signed up to help with translations for a number of different languages.

This is going to keep us busy for a bit and we are looking forward to ship some great releases with these features.

tl;dr

2013 was a phenomenal year for Apache CouchDB. 2014 is poised to be even greater, there are more people than ever pushing CouchDB forward and there is plenty of stuff to do and hopefully, we get to shape some more of the future of computing.

Thank you!

by Jan at February 24, 2014 05:53 PM

Cloudant and IBM: Our Commitment to Apache CouchDB

IBM have announced that they are acquiring Cloudant.

Adam Kocoloski is co-founder and CTO of Cloudant as well as serving on CouchDB's Project Management Committee. And with his permission, I am re-posting an email he sent to the CouchDB developer list a few hours ago.

In his words:

"Apache CouchDB means a great deal to me, and to Cloudant as a company. Cloudant and CouchDB have grown alongside each other over the past several years in one of the more authentic vendor/community collaborations I can think of in Apache history. Today marks the next step in Cloudant’s growth as we enter into a definitive agreement to become part of IBM.

"What does this mean for CouchDB? I would not have agreed to this transaction if I had any concerns about Cloudant’s ability to continue its contributions and collaboration with Apache CouchDB. IBM has a strong track record in open source software and a productive relationship with Apache; in fact, IBM was instrumental in bringing CouchDB to the ASF many years ago. IBM is fully supportive of our efforts here, and I’m looking forward to bringing increased resources to bear in support of the project.

"CouchDB has the potential to shape the future of distributed data management and computing. 2013 was a year of tremendous progress, as we doubled our committer base and shipped no fewer than eight releases. Already in 2014 we’ve seen amazing progress on long-standing initiatives to enhance the core of the system. The timing is right — in the market and for the community — to take the next big step forward. With your help, that is exactly what we will do.

"Truly, the future of CouchDB is CouchDB."

Thanks Adam!

I'd like to extend my hearty congratulations to everyone at Cloudant.

The future for CouchDB looks very interesting...

by Noah Slater at February 24, 2014 05:47 PM

February 14, 2014

The Official CouchDB Blog

Board Report (Februrary 2014)

Description

CouchDB submits a board report every quarter. Here's a copy of our February board report.

Releases

No releases in this time period.

Release has been blocked on a number of issues, but is actively being worked on.

Recent Activity

  • Number of the core devs met in Vienna to discuss the pending merges.
  • Finalising the merge of Benoit Chesneau’s rcouch fork.
  • Working on the BigCouch merge from Cloudant.
  • Created a replication list to discuss specific replication topics.
  • New marketing list about to be created for new marketing team.
  • Confluence wiki set up, and migration being planned.
  • Review Board instance set up, and discussion about review ongoing.
  • Translation work going well.
  • Community-provided packages added for Ubuntu Precise and upcoming Trusty release.

Community

Including the following additions, CouchDB has 31 committers and 9 PMC members.

New committers:

  • Nick North

No new PMC members. Most recent PMC addition Nov 9th, 2012.

Mailing list stats:

  • announce
    • 151 subscribers (+20)
    • 1 message since May (-1)
  • user
    • 1405 subscribers (-26)
    • 1086 messages since May (+280)
  • erlang
    • 154 subscribers (+8)
    • 14 messages since May (-8)
  • dev
    • 602 subscribers (-4)
    • 1977 messages since May (+1097)
  • commits
    • 104 subscribers (-1)
    • 2735 messages since May (+1694)
  • l10n
    • 31 subscribers (+10)
    • 207 messages since May (+104)
  • replication
    • 47 subscribers
    • 35 messages since May
  • Issues

    None.

    by Noah Slater at February 14, 2014 05:24 PM

    Board Report (November 2013)

    CouchDB submits a board report every quarter. Here's a copy of our November board report.

    Description

    Apache CouchDB is a database that uses JSON for documents, JavaScript for MapReduce queries, and regular HTTP for an API.

    Releases

    Recent Activity

    • Finalising the merge of Benoit Chesneau’s rcouch fork.
    • Preparation of CouchCamp Vienna, a community run weekend hackathon.
    • Successful run of CouchDB Conf Vancouver with ~80 attendees, 13 speakers. Videos will be available.
    • Created an I18N list and translation team to drive translation efforts.
    • Passed a proposal on using Influitive to boost our community engagement.

    Community

    Including the following additions, CouchDB has 29 committers and 9 PMC members.

    New committers:

    No new PMC members. Most recent PMC addition Nov 9th, 2012.

    Mailing list stats:

    • announce
      • 131 subscribers (+37)
      • 2 message since August (+1)
    • user
      • 1431 subscribers (-10)
      • 806 messages since August (+310)
    • erlang
      • 146 subscribers (+25)
      • 22 messages since August (15)
    • dev
      • 606 subscribers (+7)
      • 880 messages since August (-130)
    • commits
      • 105 subscribers (-2)
      • 1041 messages since August (+344)
    • l10n
      • 21 subscribers (+21)
      • 103 messages since August (+103)

    Issues

    None.

    by Noah Slater at February 14, 2014 05:20 PM

    February 13, 2014

    The Official CouchDB Blog

    Packages for Ubuntu 12.04 Precise

    I’m delighted to announce the availability of Apache CouchDB Ubuntu Precise (12.04) packages, with integrated upstart support.

    These have been contributed with support from Mobius Medical Systems, LP, who use Apache CouchDB in their product Mobius3D/FX for oncology and radiology, and funded the development of this work through the Couch Firm.

    This would not have been possible without the assistance of Jason DeRose (who has lead the 1.5.0 package going into the next Ubuntu Trusty release), and Noah Slater, who was surprised on reviewing the package to find so much of his original work there!

    Installation should be straightforward, usual caveats around making backup before upgrading precautions apply, noting that between 1.0.1 and 1.5.0 there are a number of key changes, including the on-disk data format used by the .couch files was upgraded. Although it should keep your dbs intact and do the right thing for your local.ini files, always have a backup - or two!

    sudo apt-get install python-software-properties -y
    sudo add-apt-repository ppa:couchdb/stable -y
    sudo apt-get update -y
    # remove any existing couchdb
    sudo apt-get remove couchdb couchdb-bin couchdb-common -yf
    # see my shiny 1.5.0 goodness
    sudo apt-get install -V couchdb
        Reading package lists... Done
        Building dependency tree
        Reading state information... Done
        The following extra packages will be installed:
           couchdb-bin (1.5.0-0ubuntu2)
           couchdb-common (1.5.0-0ubuntu2)
           couchdb (1.5.0-0ubuntu2)
         … Y …
    
    # manage via upstart
    sudo stop couchdb
        couchdb stop/waiting
    sudo start couchdb
        couchdb start/running, process 3541
    …
    

    You can of course modify or duplicate the /etc/init/couchdb.conf script, to support running multiple CouchDB instances or users on the same box.

    Note there's also a couchdb/dev PPA which will be used for testing and updating the next release, e.g. the upcoming 1.6.0, before switching it over to the stable branch once it's had enough eyes on it.

    by dch at February 13, 2014 12:19 PM

    December 11, 2013

    The Official CouchDB Blog

    CouchDB Conf Videos

    Videos from CouchDB Conf are now available.

    CouchDB everywhere with PouchDB
    Dale Harvey, Mozilla
    PouchDB is CouchDB written in JavaScript, that means it runs everywhere, I mean everywhere. Find out about why this changes everything for CouchDB and app developers, as well as the latest updates and status of the project.

    CouchDB Writ Large
    Mike Miller, Cloudant
    In this talk, Mike Miller describes how CouchDB fits into Cloudant's software stack as one of many open source projects the company uses to run distributed database clusters around the world.

    Replication, FTW!
    Benjamin Young, Cloudant
    Jan's "call to arms" earlier this year made replication the central focus of Apache CouchDB. Replication is the feature that makes CouchDB so distinctive in any marketplace (open or otherwise) and sets it apart from the NoSQL pack. We'll take a look at the various replicating databases, how we can extend the reach of replication, and how we can help others find their way into this world of JSON juggling.

    10 Common Misconceptions about CouchDB
    Joan Touzet, Atypical
    I've consulted with hundreds of people who use CouchDB, and the same sorts of questions keep coming up. Come to this talk if you want to know more about the kinds of mistakes many users make when thinking about how to use the database in their application. I'll talk a bit about the "rough edges" of CouchDB, and how to work them to your advantage.

    Experiences using CouchDB inside Microsoft's Windows Azure team
    Will Perry and Brian Benz
    Will Perry, a developer on the Windows Azure team, is part of a small team at Microsoft that has been using CouchDB extensively on an internal project. In this talk, he'll share motivation for using CouchDB, experiences using and running CouchDB in Azure, design and architectural patterns the team used and a quick run-down of some of the things they love and struggle with on a team more used to relational SQL DBs.

    Deep Dive into a Shallow Write Pool
    Jason Johnson, IBM SoftLayer
    When storing data safely, disk I/O is almost always a major bottleneck. In this talk, we'll examine some of CouchDB's write patterns and explore how we can tune underlying storage media to complement those patterns.

    Say Hello to Fauxton, the New Web Dashboard for CouchDB
    Russell Branca, Cloudant
    In this talk, Cloudant Engineer Russell Branca discusses the new Web dashboard that will soon be released for Apache CouchDB.

    Making CouchDB Plugins
    Jason Smith, Nodejitsu
    CouchDB now has plugin support! This talk explains the design of the plugin system and demonstrates extending CouchDB by building example plugins. From "my first Erlang project" to major CouchDB changes, plugins have something for everybody.

    Personal Web Apps
    Nathan Vander Wilt, freelance product developer
    CouchDB is great at keeping track of all sorts of personal data — everything from notes and contacts, to sharing a live stream of photos from a balloon 250 feet above a construction site. I'll demo of some of the more interesting apps I've been building for my "personal cloud" and talk about the CouchDB features which make them easy. We'll also explore what CouchDB *doesn't* do and how to get those things done anyway.

    How CouchDB is powering the next-generation cloud storage
    Sascha Reuter, doctape / tape.io GmbH
    This talk will give you an overview on how we leverage CouchDB to be the heart of our cloud-storage service. It will cover a whole bunch of fascinating real-life experiences, as well as some nifty tricks on how to unleash CouchDB's full potential in production. I'll show off how the mighty changes-stream helps us interconnecting different components of our service, as well as pushing realtime UI updates & notifications to our users. I'll also tell you about things to avoid, to guarantee full performance, data consistency and high availability in production. To the end, I'll show off how we use CouchDB even as our global queue for processing & conversion jobs and why this is actually a pretty good idea.

    Open Data with Couch, Pouch, and Cloudant: Liberating the Laws of Massachusetts
    Calvin Metcalf
    As a member of Code for Boston, the Boston Code for America Brigade I scrapped the complete laws of Massachusetts to JSON, put them into a CouchDB database on Cloudant, and then used Cloudant to implement full-text search and PouchDB to make a demo app for people to browse the laws. Learn how couch can help distribute open data when that data is too big to just dump into github.

    War stories from npmjs.org
    Charlie Robbins & Jason Smith, Nodejitsu
    Q&A with Nodejitsu and npmjs.org, the package manager for the Node.js server-side JavaScript platform — running CouchDB on the back end.

    Check out the full playlist on YouTube.

    Thank you to Cloudant for recording and making these videos available.

    by Noah Slater at December 11, 2013 03:53 PM

    December 04, 2013

    The Official CouchDB Blog

    Board Report (August 2013)

    CouchDB submits a board report every quarter. Here's a copy of our August board report.

    Description

    Apache CouchDB is a database that uses JSON for documents, JavaScript for MapReduce queries, and regular HTTP for an API.

    Releases

    Recent Activity

    • Support added for latest Erlang/OTP releases.
    • Preparation for 1.4.0 release cycle has begun.
    • Work is underway to document an official project vision.
    • Work is underway to release in Cloudant's Fauxton overhaul of the CouchDB admin interface.
    • Work is underway to merge in Cloudant's BigCouch fork which will add clustering capabilities to CouchDB.
    • Work is underway to merge in parts of Benoît Chesneau's rcouch fork which improves the build system, refactored internal applications, and additional user-facing functionality such as bonjour/zeroconf awareness, global db changes feeds, replications and changes feeds across views.
    • Work is underway on a plugin system, with early results already shared with the community.
    • Cloudant announced the BigCouch merge, generating lots of press activity.
    • Cloudant hosted a CouchDB booth and CouchDB party at OSCON.
    • Cloudant are organising 'Apache CouchDB Conf, Vancouver' on November 13th, 2013 (in line with ASF branding and CouchDB PMC cooperation).

    Our GitHub comment notifications have not been set up yet, due to lack of progress with necessary infrastructure tasks.

    Community

    Including the following additions, CouchDB has 28 committers and 9 PMC members.

    New committers:

    No new PMC members.

    Mailing list stats:

    • announce
      • 94 subscribers (+38)
      • 1 message since May (-1)
    • user
      • 1441 subscribers (-15)
      • 496 messages since May (-561)
    • erlang
      • 121 subscribers (+16)
      • 7 messages since May (-7)
    • dev
      • 599 subscribers (-2)
      • 1010 messages since May (-996)
    • commits
      • 107 subscribers (+6)
      • 697 messages since May (-262)

    Issues

    No issues for the board at this time.

    by Noah Slater at December 04, 2013 11:13 PM

    November 09, 2013

    The Official CouchDB Blog

    CouchDB will go multilingual - looking for contributors!

    The Apache CouchDB project is pleased to announce the creation of a localization team. The l10n@couchdb.apache.org mailing list has been set up to cover everything l10n- and i18n-related in Apache CouchDB.

    The first steps will be to set up some tools to enable easy translation of the documentation and (at a later point) the administration interface (Futon/Fauxton). We are looking for people who want to help out. Would you like to read the CouchDB documentation in another language, and have some time to spare? Please help out! Bonus points if you have experience with translation or localization-related tools and best practices.

    We’re just getting started: we have some ideas on how to get started, but we need more help. If you’re interested, please subscribe to l10n@couchdb.apache.org and let us know!

    Cheers from the Apache CouchDB I10n team

    by andywenk at November 09, 2013 10:09 PM

    November 05, 2013

    The Official CouchDB Blog

    CouchDB Conf is Next Week

    It’s almost time for CouchDB Conf, held in Vancouver on November the 13th. That’s the day before Cascadia JS. If you don’t have your ticket to CouchDB Conf yet, there’s still time secure a place. Come meet the community, and learn about the future of CouchDB. The ticket includes a reception at Cascadia JS, which starts the day after CouchDB Conf.

    Purchase your ticket for CouchDB Conf now

    The conference will include sessions that span Apache CouchDB and related technologies — covering everything from getting started to advanced features as well as CouchDB internals, application development, and best practices. Attendees will also get updates on other projects in the community such as PouchDB, a Web browser-based database capable of replicating data with Apache CouchDB and other compatible databases. Featured speakers include Jan Lehnardt, Hoodie; Joan Touzet, Atypical; Brian Benz, Microsoft; and Dale Harvey, Mozilla.

    Featured Discussions:

    • Apache CouchDB is the database that replicates. CouchDB-style replication is now supported in a wide range of databases that live in browsers (PouchDB), mobile (TouchDB), and the cloud. Many of the talks will cover various aspects of using replication and modeling your data to work with it.
    • Fauxton is the new Futon, CouchDB's Web-based administration console. Fauxton brings a fresh, modular approach to UI for Apache CouchDB. It is available at /_fauxton in Apache CouchDB 1.5, and will eventually become the new /_utils UI.

    Apache CouchDB Conf is the event to attend to expand knowledge, meet project committers and contributors, and find out where Apache CouchDB is headed.

    Visit the conference website for more information.

    We hope to see you there!

    by Noah Slater at November 05, 2013 05:04 PM

    October 10, 2013

    The Official CouchDB Blog

    Registration Is Open for CouchDB Conf, Nov 13th

    CouchDB Conf is a one-day conference in Vancouver, Canada on November 13, 2013. I am excited to let you know that registration is now open! Early bird tickets are $150, and are available until October 13th. (So get yours quick!)

    Purchase your ticket for CouchDB Conf now

    CouchDB Conf will convene 200 developers and technology evangelists. They will come together to learn about CouchDB, related technologies, and new projects in the community. The agenda will cover everything from getting started to advanced features and internals, app development, and best practices.

    View the confirmed speakers

    We expect the conference to sell out quickly, so register today to avoid disappointment!

    by Noah Slater at October 10, 2013 05:41 PM

    Apache CouchDB 1.4.0 Released

    Apache CouchDB 1.4.0 has been released and is available for download.

    CouchDB is a database that completely embraces the web. Store your data with JSON documents. Access your documents with your web browser, via HTTP. Query, combine, and transform your documents with JavaScript. CouchDB works well with modern web and mobile apps. You can even serve web apps directly out of CouchDB. And you can distribute your data, or your apps, efficiently using CouchDB’s incremental replication. CouchDB supports master-master setups with automatic conflict detection.

    Grab your copy here:

    http://couchdb.apache.org/

    Pre-built packages for Windows and OS X are available.

    CouchDB 1.4.0 is a feature release, and was originally published on 2013-09-03.

    These release notes are based on the changelog.

    Upgrade Notes

    • We now support Erlang/OTP R16B and R16B01; the minimum required version is R14B.
    • User document role values must now be strings. Other types of values will be refused when saving the user document.

    Changes

    • COUCHDB-1684: Support for server-wide changes feed reporting on creation, updates and deletion of databases. #917d8988
    • COUCHDB-1139: it’s possible to apply list functions to _all_docs view. #54fd258e
    • Automatic loading of CouchDB plugins. #3fab6bb5
    • Allow storing pre-hashed admin passwords via _config API. #c98ba561
    • COUCHDB-1772: Prevent invalid JSON output when using all_or_nothing _bulk_docs API. #dfd39d57
    • Add a configurable whitelist of user document properties. #8d7ab8b1
    • COUCHDB-1852: Support Last-Event-ID header in EventSource changes feeds. #dfd2199a
    • Much improved documentation, including an expanded description of validate_doc_update functions (commit:ef9ac469) and a description of how CouchDB handles JSON number values (#bbd93f77).
    • COUCHDB-1632: Ignore epilogues in multipart/related MIME attachments. #2b4ab67a
    • Split up replicator_db tests into multiple independent tests.

    by djc at October 10, 2013 12:07 PM

    October 04, 2013

    Till Klampäckel

    SQL MAX() and GROUP BY for CouchDB

    While re-writing a couple SQL statements into CouchDB we got stuck when we wanted to do a SELECT MAX(...), id ... GROUP BY id in CouchDB.

    MySQL

    Imagine the following SQL table with data:

    mysql> SHOW FIELDS FROM deploys;
    +-------------+-------------+------+-----+---------+-------+
    | Field       | Type        | Null | Key | Default | Extra |
    +-------------+-------------+------+-----+---------+-------+
    | project     | varchar(10) | NO   |     | NULL    |       |
    | deploy_time | datetime    | NO   |     | NULL    |       |
    +-------------+-------------+------+-----+---------+-------+
    2 rows in set (0.01 sec)
    

    In order to get the latest deploy for each project, I'd issue:

    mysql> SELECT MAX(deploy_time), project FROM deploys GROUP BY project;
    +---------------------+----------+
    | MAX(deploy_time)    | project  |
    +---------------------+----------+
    | 2013-10-04 22:01:26 | project1 |
    | 2013-10-04 22:02:17 | project2 |
    | 2013-10-04 22:02:20 | project3 |
    +---------------------+----------+
    3 rows in set (0.00 sec)
    

    Simple. But what do you do in CouchDB?

    CouchDB

    My documents look like this:

    {
      "_id": "hash",
      "project": "github-user/repo/branch",
      "deploy_time": {
         "date": "2013-10-04 22:02:20",
         /* ... */
      },
      /* ... */
    }
    

    So, after more than a couple hours trying to wrap our heads around map-reduce in CouchDB, it's working.

    Here's the view's map function:

    function (doc) {
      if (doc.project) {
        emit(doc.project, doc.deploy_time.date);
      }
    }
    

    This produces nice key value pairs — in fact, multiple — for each project.

    And because the map-function returns multiple, we need to reduce our set.

    So here is the reduce:

    function (doc, values, rereduce) {
      var max_time = 0, max_value, time_parsed;
      values.forEach(function(deploy_date) {
        time_parsed = Date.parse(deploy_date.replace(/ /gi,'T'));
        if (max_time < time_parsed) {
          max_time = time_parsed;
          max_value = deploy_date;
        }
      });
      return max_value;
    }
    

    The little .replace(/ /gi,'T') took especially long to figure out. Special thanks to Cloudant's seńor JS-date-engineer Adam Kocoloski for helping out. ;-)

    Step by step

    • iterate over values
    • fix each value (add the "T") to make Spidermonkey comply
    • parse, and compare
    • return the "latest" in the end

    A note of advice: To save yourself some trouble, install a local copy of Spidermonkey and test your view code in there and not in your browser.

    Open the view in your browser: http://localhost/db/_design/deploys/_view/last_deploys?group=true

    {
      "rows":[
        {
          "key":"testowner/testname/testbranch",
          "value":"2013-09-13 11:41:03"
        },
        {
          "key":"testowner/testname/testbranch2",
          "value":"2013-09-12 16:48:39"
        }
      ]
    }
    

    Yay, works!

    FIN

    That's all for today.

    by Till Klampaeckel (till@php.net) at October 04, 2013 08:40 PM

    August 13, 2013

    The Official CouchDB Blog

    Announcing CouchDB Conf in Vancouver

    We’re excited to announce the next Apache CouchDB Conf will be in Vancouver, Canada on November 13, 2013. Big thanks to Cloudant to helping to make this happen!

    Sign up to get notified when CouchDB Conf tickets go on sale.

    We’re building the schedule with talks from all corners of the CouchDB community and hope to see you there this fall. This is a follow-up to our CouchDB Conf in Berlin, where we laid the groundwork for the BigCouch merge and other exciting features you'll hear about in Vancouver (geospatial data and PouchDB for mobile devices to name two).

    We’ll be in Vancouver, BC the day before CascadiaJS 2013, which will be held November 14-15. Sign up to get notified when the next batch of CascadiaJS tickets go on sale. The Early Bird tickets sold out in 20 minutes!

    There's lots of crossover between the JavaScript and CouchDB communities. Data-driven web apps and CouchDB makes it a natural fit. So we hope you'll be able to stick around and experience what both conferences have to offer.

    We’ll plan to keep running these Apache CouchDB Confs. Got suggestions? Feedback? Wanna help us with our next one? Get in touch publicly, or privately. You can also tweet us at @CouchDB. Whatever suits!

    by Noah Slater at August 13, 2013 09:17 PM

    July 25, 2013

    The Official CouchDB Blog

    Apache CouchDB is supporting the Google Summer of Code

    The Apache Software Foundation is participating in GSoC again this year, and CouchDB is on the project list of course!

    By the end of the summer, we hope you've above all, had fun, and we've enabled you to make a valued contribution, and ideally help you continue in your involvement with the CouchDB project.

    I'm a student, how do I start?

    Check out some of the ideas or suggest your own ones. Please subscribe to our developer mailing list and join us on IRC at #couchdb-dev on irc.freenode.net to talk about what you're interested in. Don't forget to read the ASF GsOC guide.

    You'll need to pick your projects and submit proposals from 22nd April to 3rd May.

    I'm interested in being a mentor for the CouchDB project!

    Thanks! Please read up on being a mentor and contact dev@couchdb.apache.org for more details. Previous mentors suggest the effort is around 5h per week on average. Please review the topics list and feel free to make any adjustments.

    Please pass this on to your local user groups, universities, etc. Thanks!

    For more information on Google Summer of Code itself, timelines and proposals, visit the Melange website.

    For more information on the Apache Software Foundation, visit our ASF site.

    by dch at July 25, 2013 06:49 PM

    Welcome BigCouch

    Good news! Cloudant has announced the completion of the BigCouch merge. This is a huge step forward for CouchDB. So thank you to Cloudant, and thank you to the committers (particularly Robert Newson and Paul Davis) who slogged (and travelled the world to pair with each other) to make this happen.

    What does this mean? Well, right now, the code is merged, but not released. So hold your clicks just a moment! Once the code has been tested, we will include it in one of our regular releases. (If you want to help us test, hop on to the dev@ mailing list!)

    What’s new? The key accomplishment of the merged code is that BigCouch’s clustering capability, along with the rest of Cloudant’s other enhancements to CouchDB’s code base, will now be available in Apache CouchDB. This also includes improvements in compaction and replication speed, as well as boosts for high-concurrency access performance.

    Painless replication has always been CouchDB's biggest feature. Now we get to take advantage of Cloudant’s experience running large distributed clusters in production for four years. With BigCouch merged in, CouchDB will be able to replicate data at a much larger scale.

    But wait! That’s not all! Cloudant has decided to terminate their BigCouch fork of CouchDB, and instead focus future development on Apache CouchDB. This is excellent news for CouchDB, even more excellent news for the CouchDB community.

    Here’s the original press release:

    Cloudant Contributes Database Scalability and Fault-Tolerance Framework to Apache CouchDB™

    And here are some highlights from the press:

    Sofa, so good ... BigCouch relaxes into comfy Apache CouchDB (The Register)

    Cloudant pitches one CouchDB for all (GigaOM)

    Cloudant Merges BigCouch Into Apache CouchDB (eWeek)

    Cloudant Breathes Life Into CouchDB - Merges BigCouch into Apache Project (CMSwire)

    CouchDB Gets Cloudant's 'Super-Size Me' Scaling (InformationWeek)

    Cloudant Merges BigCouch Code into Apache Open Source Database (The WHIR)

    Cloudant Merges BigCouch Database into Open Source Apache CouchDB (The VAR Guy)

    Cloudant merges distributed BigCouch code with Apache CouchDB (JAXenter)

    Cloudant contributes database scalability and fault-tolerance framework to Apache CouchDB (SD Times)

    P.S. Cloudant is also hosting a party at OSCON on July 25 to celebrate the news. (They’re booth #305 if you’re at the show.) And if all that wasn’t enough, stay tuned for news about a CouchDB conference that we should be announcing shortly…

    by Noah Slater at July 25, 2013 06:43 PM

    June 27, 2013

    The Official CouchDB Blog

    Apache CouchDB 1.3.1 Released

    Apache CouchDB 1.3.1 has been released and is available for download.

    CouchDB is a database that completely embraces the web. Store your data with JSON documents. Access your documents with your web browser, via HTTP. Query, combine, and transform your documents with JavaScript. CouchDB works well with modern web and mobile apps. You can even serve web apps directly out of CouchDB. And you can distribute your data, or your apps, efficiently using CouchDB’s incremental replication. CouchDB supports master-master setups with automatic conflict detection.

    Grab your copy here:

    http://couchdb.apache.org/

    Pre-built packages for Windows and OS X are available.

    CouchDB 1.3.1 is a bugfix release, and was originally published on 2013-06-27.

    These release notes are based on the CHANGES file.

    Replicator

    • Tolerate missing source and target fields in _replicator docs (COUCHDB-1788).

    Log System

    • Don't log about missing .compact files.

    • Fix bug in WARN level logging from 1.3.0 (COUCHDB-1794).

    View Server

    • Fix the -S option to couchjs to increase memory limits (COUCHDB-1792).

    Misc

    • Improve documentation: better structure, improve language, less duplication.

    • Improvements to test suite and VPATH build system.

    by Noah Slater at June 27, 2013 06:35 PM

    April 09, 2013

    The Official CouchDB Blog

    Apache CouchDB 1.0.4 Released

    Apache CouchDB 1.0.4 has been released and is available for download.

    Grab your copy here:

    http://www.apache.org/dyn/closer.cgi?path=/couchdb/1.0.4/apache-couchdb-1.0.4.tar.gz

    These release notes are based on the NEWS file.

    Log System

    • Fix file descriptor leak in _log.

    HTTP Interface

    • Fix missing revisions in _changes?style=all_docs.
    • Fix validation of attachment names.

    View Server

    • Avoid invalidating view indexes when running out of file descriptors.

    Replicator

    • Fix a race condition where replications can go stale.

    More Information

    See the CHANGES file and the Git commit log for more information.

    by Jan at April 09, 2013 07:41 PM

    Apache CouchDB 1.1.2 Released

    Apache CouchDB 1.1.2 has been released and is available for download.

    Grab your copy here:

    http://www.apache.org/dyn/closer.cgi?path=/couchdb/1.1.2/apache-couchdb-1.1.2.tar.gz

    These release notes are based on the NEWS file.

    HTTP Interface

    • ETag of attachment changes only when the attachment changes, not the document.
    • Fix retrieval of headers larger than 4k.
    • Allow OPTIONS HTTP method for list requests.
    • Don't attempt to encode invalid json.

    Replicator

    • Fix pull replication of documents with many revisions.
    • Fix replication from an HTTP source to an HTTP target.

    View Server

    • Avoid invalidating view indexes when running out of file descriptors.

    Log System

    • Improvements to log messages for file-related errors.

    Build System

    • Windows: don't ln the couchjs install target.
    • Windows: remove icu version dependency.
    • Improve SpiderMonkey version detection.

    More Information

    See the CHANGES file and the Git commit log for more information.

    by Jan at April 09, 2013 07:41 PM

    Apache CouchDB 1.2.0 Released

    Apache CouchDB 1.2.0 has been released and is available for download.

    Grab your copy here:

    http://couchdb.apache.org/

    Windows packages are now available. Grab them at the same download link.

    This release also coincides with a revamped project homepage!

    This is a big release with lots of updates. Please also note that this release contains breaking changes.

    These release notes are based on the NEWS file.

    Performance

    • Added a native JSON parser

      Performance critical portions of the JSON parser are now implemented in C. This improves latency and throughput for all database and view operations. We are using the fabulous yajl library.

    • Optional file compression (database and view index files)

      This feature is enabled by default.

      All storage operations for databases and views are now passed through Google's snappy compressor. The result is simple: since less data has to be transferred from and to disk and through the CPU & RAM, all database and view accesses are now faster and on-disk files are smaller. Compression can be changed to gzip compression with options that specify the compression ratio or it can be fully disabled as well.

    • Several performance improvements, especially regarding database writes and view indexing

      Combined with the two preceding improvements, we made some less obvious algorithmic improvements that take the Erlang runtime system into account when writing data to databases and view index files. The net result is much improved performance for most common operations including building views.

      The JIRA ticket (COUCHDB-976) has more information.

    • Performance improvements for the built-in changes feed filters _doc_ids and _design

    Security

    The security system got a major overhaul making it way more secure to run CouchDB as a public database server for CouchApps. Unfortunately we had to break a bit of backwards compatibility with this, but we think it is well worth the trouble.

    • Documents in the _users database can no longer be read by everyone

      Documents in the _users databases can now only be read by the respective authenticated user and administrators. Before, all docs were world-readable including their password hashes and salts.

    • Confidential information in the _replication database can no longer be read by everyone

      Similar to documents in the _users database, documents in the _replicator database now get passwords and OAuth tokens stripped when read by a user that is not the creator of the replication or an administrator.

    • Password hashes are now calculated by CouchDB instead of the client

      Previously, CouchDB relied on the client to hash and salt the user's password. Now, it accepts plain text passwords and hashes them before they are committed to disk, following traditional best practices.

    • Allow persistent authentication cookies

      Cookie based authentication can now keep a user logged in over a browser restart.

    • OAuth secrets can now be stored in the users system database

      This is better for managing large numbers of users and tokens than the old, clumsy way of storing OAuth tokens in the configuration system and configuration system.

    • Updated bundled erlang_oauth library to the latest version

      The Erlang library that handles OAuth authentication has been updated to the latest version.

    Build System

    • cURL is no longer required to build CouchDB as it is only required by the command line JavaScript test runner

      This makes building CouchDB on certain platforms easier.

    HTTP API

    • Added a data_size property to database and view group information URIs

      With this you can now calculate how much actual data is stored in a database file or view index file and compare it with the file size that is already being reported. The difference is CouchDB-specific overhead most of which can be reclaimed during compaction. This is used to power the automatic compaction feature (see below).

    • Added optional field since_seq to replication objects/documents

      This allows you to start a replication from a certain database update sequence instead from the start.

    • The _active_tasks API now exposes more granular fields for each task type

      The replication and compaction tasks, e.g. report their progress in the task info.

    • Added built-in changes feed filter _view

      With this you can use a view's map function as a changes filter instead of duplicating.

    Core Storage

    • Added support for automatic compaction

      This feature is disabled by default, but it can be enabled in the configuration page in Futon or the .ini files.

      Compaction is a regular maintenance task for CouchDB. This can now be automated based on multiple variables:

      • A threshold for the file_size to disk_size ratio (say 70%)
      • A time window specified in hours and minutes (e.g 01:00-05:00)

      Compaction can be cancelled if it exceeds the closing time. Compaction for views and databases can be set to run in parallel, but that is only useful for setups where the database directory and view directory are on different disks.

      In addition, if there's not enough space (2 × data_size) on the disk to complete a compaction, an error is logged and the compaction is not started.

    Replicator

    • A new replicator implementation that offers more performance and configuration options

      The replicator has been rewritten from scratch. The new implementation is more reliable, faster and has more configuration than the previous implementation. If you have had any issues with replication in previous releases, we strongly recommend giving 1.2.0 a spin.

      Configuration options include:

      • Number of worker processes
      • Batch size per worker
      • Maximum number of HTTP connections
      • Number of connection retries

      See default.ini for the full list of options and their default values.

      This allows you to fine-tune replication behaviour tailored to your environment. A spotty mobile network connection can benefit from a single worker process and small batch sizes to reliably, albeit slowly, synchronise data. A full-duplex 10GigE server-to-server connection on a LAN can benefit from more workers and higher batch sizes. The exact values depend on your particular setup and we recommend some experimentation before settling on a set of values.

    Futon

    • Futon's Status screen (active tasks) now displays two new task status fields: Started on and Updated on

    • Simpler replication cancellation

      Running replications can now be cancelled with a single click.

    Log System

    • Log correct stack trace in all cases

      In certain error cases, CouchDB would return a stack trace from the log system itself and hide the real error. Now CouchDB always returns the correct error.

    • Improvements to log messages for file-related errors

      CouchDB requires correct permissions for a number of files. Error messages related to file permission errors were not always obvious and are now improved.

    Various Bugfixes

    • Fixed old index file descriptor leaks after a view cleanup
    • Fixes to the _changes feed heartbeat option when combined with a filter. It affected continuous pull replications with a filter
    • Fix use of OAuth with VHosts and URL rewriting
    • The requested_path property of query server request objects now has the path requested by clients before VHosts and rewriting
    • Fixed incorrect reduce query results when using pagination parameters
    • Made icu_driver work with Erlang R15B and later
    • Improvements to the build system and etap test suite
    • Avoid invalidating view indexes when running out of file descriptors

    Breaking Changes

    This release contains breaking changes:

    http://wiki.apache.org/couchdb/Breaking_changes

    It is very important that you understand these changes before you upgrade.

    More Information

    See the CHANGES file and the Git commit log for more information.

    by Noah Slater at April 09, 2013 07:40 PM

    Apache CouchDB 1.2.1 Released

    Apache CouchDB 1.2.1 has been released and is available for download.

    Grab your copy here:

    http://www.apache.org/dyn/closer.cgi?path=/couchdb/1.2.1/apache-couchdb-1.2.1.tar.gz

    These release notes are based on the NEWS file.

    HTTP Interface

    • Fix various bugs in the URL rewriter when recursion is involved.

    Platform Integration

    • Fix couchdb start script.

    Futon User Interface

    • Disable buttons that aren't available for the logged-in user.

    Replicator

    • Fix potential replication timeouts.

    View Server

    • Change use of signals to avoid broken view groups.

    More Information

    See the CHANGES file and the Git commit log for more information.

    by Jan at April 09, 2013 07:35 PM

    Apache CouchDB 1.2.2 Released

    Apache CouchDB 1.2.2 has been released and is available for download.

    CouchDB is a database that completely embraces the web. Store your data with JSON documents. Access your documents with your web browser, via HTTP. Query, combine, and transform your documents with JavaScript. CouchDB works well with modern web and mobile apps. You can even serve web apps directly out of CouchDB. And you can distribute your data, or your apps, efficiently using CouchDB’s incremental replication. CouchDB supports master-master setups with automatic conflict detection.

    Grab your copy here:

    http://couchdb.apache.org/

    Pre-built packages for Windows and OS X are available.

    This is a bugfix release

    These release notes are based on the CHANGES file.

    HTTP Interface

    • Reset rewrite counter on new request, avoiding unnecessary request failures due to bogus rewrite limit reports.

    Build System

    • Fixed issue in couchdb script where stopped status returns before process exits.

    by Noah Slater at April 09, 2013 07:32 PM

    Apache CouchDB 1.3.0 Released

    Apache CouchDB 1.3.0 has been released and is available for download.

    CouchDB is a database that completely embraces the web. Store your data with JSON documents. Access your documents with your web browser, via HTTP. Query, combine, and transform your documents with JavaScript. CouchDB works well with modern web and mobile apps. You can even serve web apps directly out of CouchDB. And you can distribute your data, or your apps, efficiently using CouchDB’s incremental replication. CouchDB supports master-master setups with automatic conflict detection.

    Grab your copy here:

    http://couchdb.apache.org/

    Pre-built packages for Windows and OS X are available.

    This release comes with a CouchDB manual, hosted directly out of Futon. A PDF version of the manual is also distributed.

    This is a feature release.

    These release notes are based on the CHANGES file.

    HTTP Interface

    • No longer rewrites the X-CouchDB-Requested-Path during recursive calls to the rewriter.

    • Limit recursion depth in the URL rewriter.

      Defaults to a maximum of 100 invocations but is configurable.

    • Fix _session for IE7.

    • Added Server-Sent Events protocol to db changes API.

      See http://www.w3.org/TR/eventsource/ for details.

    • Make password hashing synchronous when using the /_config/admins API.

    • Include user name in show/list ETags.

    • Experimental support for Cross-Origin Resource Sharing (CORS).

      See http://www.w3.org/TR/cors/ for details.

    Replicator

    • The replicator will use a new server-wide UUID in checkpoint IDs to improve the chances of an efficient resume.

    Storage System

    • Fixed unnecessary conflict when deleting and creating a document in the same batch.

    View Server

    • Additional response headers may be varied prior to send().

    • GetRow() is now side-effect free.

    Futon

    • Disabled the link to the Futon test suite.

      These tests were causing problems when run from a browser, and are now available via the CLI instead.

    • Added view request duration to Futon.

    • Disable buttons for actions that the user doesn't have permissions to.

    Security

    • Passwords are now hashed using the PBKDF2 algorithm with a configurable work factor.

    Test Suite

    • Moved the JS test suite to the CLI.

    • Improved tracebacks printed by the JS CLI tests.

    • Improved the reliability of a number of tests.

    UUID Algorithms

    • Added the utc_id algorithm.

    URL Rewriter & Vhosts

    • Database name is encoded during rewriting (allowing embedded /'s, etc).

    • Reset rewrite counter on new request, avoiding unnecessary request failures due to bogus rewrite limit reports.

    Build System

    • C/C++ compiler detection has been improved.

    • Autoconf v2.63 is now required if building from Git checkout directly.

      See DEVELOPERS file for more details.

    • Fixed issue in couchdb script where stopped status returns before process exits.

    by Noah Slater at April 09, 2013 06:30 PM

    February 19, 2013

    EasyBib

    Cloudant's NYC meet-up & webinar!

    Cloudant’s NYC meet-up will take place at our office this Wednesday (tomorrow)! Please RSVP using the following link:

    http://cloudantimagineeasy.eventbrite.com/

    The office’ address is 12 West 31 St:


    View Larger Map

    If you can’t make the meet-up, make sure to check out the webinar on Thursday:

    https://www1.gotomeeting.com/register/648768041

    by tillinside at February 19, 2013 07:23 PM

    February 12, 2013

    EasyBib

    EasyBib and Cloudant

    Slides from a talk I gave at the CouchDB conference in Berlin last month. As always, the slides are on speakerdeck:

    https://speakerdeck.com/till/easybib-and-cloudant

    — Till

    by tillinside at February 12, 2013 02:44 PM

    February 06, 2013

    IrisCouch

    Announcing Premium Server

    We’re excited to announce our latest product—a Premium Server option for Iris Couch users. Premium Server gives serious users an option to increase the stability and response time of their couches. It costs $75/month (in addition to the regular, metered fees), and buys you oh-so-much-more peace of mind... and performance.

    A Premium server is more responsive and reliable than a standard CouchDB server. Premium servers run on isolated, dedicated clusters.

    Interested? Log in and enable Premium Server in our billing page.

    Once you have a credit card on file, you will see a Premium Server option. If for any reason you’re dissatisfied with the service, you can cancel at any time.

    More Details

    What, exactly, is a Premium server?

    Premium servers run permanently in specialized clusters, with dedicated CPU and memory. Premium servers run on a common physical server, with dedicated CPU, memory, and i/o resources. Nothing is “oversold.”

    Premium servers never pause when idle. Typically, if a couch goes unused for too long, we pause it, freeing resources to share with others. Unfortunately, infrequent usage suffers high latency. Premium servers do not have this issue.

    Premium servers are monitored. Our automated tools monitor everything. But people watch every premium server 24/7. If we see any server issues, we respond immediately.

    Premium servers are isolated. Each cluster runs in a dedicated DNS zone and uses dedicated HTTP infrastructure. Issues in one cluster do not affect another.

    Premium servers work. Thanks to many of our users for months of beta testing. Premium service already supports many couches, such as the Node.js npm registry, a mission-critical CouchDB service, processing over 100 million monthly requests.

    by Jeff (jeff@iriscouch.com) at February 06, 2013 02:08 PM

    February 01, 2013

    Jan Lehnardt

    The State of CouchDB

    Slides to go with the headings.


    Hi, welcome to CouchDB Conf.

    CouchDB means a lot to me and it makes me very happy that you are all here today.

    Who am I

    My name is Jan Lehnardt and I am the elected project management committee chair of the Apache CouchDB project. If you don’t know what this means, don’t worry, it means I sign up to deal with politics and paperwork voluntarily.

    I have been with CouchDB since 2006. My first contribution was the initial UNIX build system to CouchDB. I stuck around ever since.

    My main job back then was PHP/MySQL consulting and Open Source work as my own 20% project. In 2007, I started giving talks about CouchDB at user groups and conferences and quickly became addicted. People called me the CouchDB Chief Evangelist, although I don’t really like the title. From 2008 I started getting contracting jobs specifically for CouchDB and things blew up quickly that it became clear soon, that an organisation of some sort was needed to handle this. I eventually joined forces with CouchDB’s original inventor and J Chris Anderson to form the company Couch.io. We initially weren’t keen on taking venture capital, but when the whole NoSQL thing blew up, it was very hard to say no and on January 1st 2010 we were in business.

    Couch.io turned into CouchOne and was eventually merged with Membase into Couchbase. Unfortunately, at this point, there isn’t much Couch-y about Couchbase aside from some underlying technology, so the name can be confusing.

    I parted ways with Couchbase a week ago in order to concentrate and focus more on Apache CouchDB and related technologies.

    CouchDB isn’t Dead

    In late 2011 two major supporters of CouchDB withdrew their support: Canonical’s UbuntuOne service, on paper a poster-child use-case for CouchDB and Saucelab’s Selenium platform publicly made clear that they were moving to other technology. It wasn’t a big deal if you looked closely, the one project didn’t have enough resources to integrate with CouchDB the right way and the other diverged so far from the original system design, that CouchDB was no longer a good fit. We were and we remain good friends with the respective developers, so we didn’t feel like pointing that out specifically. The result though was that people who didn’t look closely saw that CouchDB had popular failures where it hadn’t. This wasn’t all bad, but set the stage for another incident a few weeks later.

    CouchDB’s original inventor publicly announced that he’d move on from Apache CouchDB to Couchbase fully. That new technology would only go into Couchbase. If you are in the know, this wasn’t a big deal again, he hadn’t contributed to CouchDB in two years, Couchbase was a different thing and he wanted to rally support for that. The casual reader though took away one lesson: “CouchDB is dead”.

    With the added confusion of Couchbase’s name, and continuing their CouchConf conferences, things looks dire.

    CouchDB 2012

    With this in the wild, the active CouchDB committers got together and put out an excellent 1.2.0 release that was accompanied by a new, modern website design. We thought instead of writing counter-statements and start unnecessary he-said-she-said discussions, we’d rather concentrate on shipping good software. 1.2.0 was well received and the whole thing lead the team to come together as a whole in Boston and two weeks later in a smaller group in Dublin in April 2012.

    We came up with a long list of of features that we wanted to get into CouchDB. Some work happened over the summer, but I personally took a bit of a break to avoid burn-out from CouchDB. In fall, things started moving faster again when more community members started pushing things along. New contributors came and we finally came up with a way to go forward.

    We decided to not be insane. The definition of insanity is trying the same thing over and over again and expecting different results. While it wasn’t all bad in CouchDB land we could do better and to do better we had to change a few things.

    CouchDB Reboot

    We determined that CouchDB needs a fresh mission statement. I signed up to come up with a few drafts, but nothing happened yet, so this one is still out there cough.

    We decided to release more often. With the upcoming 1.3.0 release we will switch to a quarterly release cycle. A new minor version will come out approximately every 90 days, even if it is just a few bug fixes that a small number of people direly need.

    CouchDB 1.3.0 will ship with top-notch API documentation right inside of Futon, the built in admin interface.

    We will start to set up teams where anyone can sign up to work on a specific thing: docs, our JavaScript components, the Erlang core, the admin interface (built in HTML+JS). We want to make sure anyone who has the right skills can contribute in their area of expertise and not be scared away by the Erlang.

    We put some emphasis on QA, automated testing, building binaries and packages for releases as well as in-development versions, so people can try new features before they come out more easily and as a result report issues early.

    A few of the core team will put some focus on encouraging contributions rather than coding themselves. We hope to get more people on board with CouchDB development.

    We also started doing roughly quarterly in-person meetings (at least as smaller groups) and weekly status meetings on IRC to give everything a little more focus and transparency.

    The next few slides show a long list a few features that we’ll be working on. If you think anything is missing, let us know!

    The more prominent things are: Merging rcouch, BigCouch, the new Futon, the plugin system and v8/node integration.

    rcouch is a cutting edge distribution of CouchDB with many experimental features. A lot of that work is useful for Apache CouchDB and we plan to fold those back into the project.

    BigCouch is a industry-strength clustering solution for CouchDB to allow seamless scaling and storing of lots of data. BigCouch is going to be merged into mainline CouchDB as well.

    The admin interface Futon is getting a complete refresh with modern web development technologies.

    There are many features hidden in some developer’s git repository because, while useful, it wasn’t generally applicable to CouchDB. With the plugin system, we hope that we can fuel a repository of plugins to CouchDB that are easily installed (think browser extensions, right from Futon). For example, you could install GeoCouch with the click of a button.

    Finally, we are currently discussing the move from SpiderMonkey and to V8 or even Node. The jury is still out on which one it is going to be, but we hope that this is going to open up a lot more opportunities for CouchDB in terms of features and new contributors.


    Community

    CouchDB would be nothing without you, the community. People who know little about CouchDB might like it or not, but people who really get to know CouchDB are extremely passionate about it.

    CouchDB’s core idea, the promise that data can be anywhere it is needed, to make applications easier, more approachable and effective for people in modern computing environments, that idea is a big one. It’s the one that brings this community together.

    Thanks again for coming, you are all awesome. CouchDB means a lot to me and I get a little teary seeing you all here.

    If you like any of the above todos, or have some of your own, please help out, get in touch about how.

    Friends of CouchDB

    The CouchDB ecosystem is full of exciting projects that support CouchDB in one way or another.

    PouchDB

    PouchDB is the newest project on the scene. PouchDB is a pure JavaScript implementation of CouchDB targeted at the browser.

    Let me say that again: CouchDB in JavaScript, for the browser.

    PouchDB is the oldest idea, but it took the longest to realise because browsers weren’t ready for it. Now is the time though that we can build and ship a CouchDB compatible data store and replication engine that lives entirely in your browser. If you want to build web apps that can seamlessly synchronise data between a laptop, a phone and a server and not treat one as a “master” and the other as “slaves”, PouchDB is your solution.

    TouchDB

    TouchDB-iOS and -Android are Objective-C and Java implementations of the same idea, but for mobile phone platforms. They natively integrate into your iOS and Android applications and allow synchronisation between themselves, multiple devices, CouchDB’s on the server and even PouchDB in a browser. Again, a milestone in fulfilling the CouchDB vision: data where it is needed, not on some server, far away over a slow and unreliable or even unavailable connection.

    Node.js

    This might not be too obvious to people who don’t do Node, but CouchDB is big, if not essential here, too. As any programming language & platform, Node.js comes with a package manager called NPM. As far as package managers go (and I’ve used them all except I think ant) NPM is the one that sucks least, or rather, it is pretty good.

    If you are invested in Node.js programming NPM isn’t just a tool you can use, it is essential to the development and deployment workflow of your applications. It is that good.

    And here is the kicker: NPM’s (de)-central registry is build on CouchDB. NPM is effectively a CouchApp. The Node elders love CouchDB.

    I suggest, and plan to turn this into an opportunity for CouchDB. Say, for example, we need benchmarking tools for CouchDB. The Node folks are great at that. If we were to go and say, “hey, do you want to help make NPM faster? Help us build benchmarking tools, so we can speed up CouchDB for you.” — And that’s just one idea, the potential collaboration is immense. Add to that that we maybe get native Node support inside CouchDB, we can do all sorts of crazy shit like writing core add-ons to CouchDB in JavaScript.


    Finally, I’d like to mention Hoodie, a project that Gregor, Alex, I and a few friends have been working on on and of over the past year. It is frontend web app framework that uses Node and CouchDB. We are happy to show that to you on the weekend.

    The Node Firm

    I am going to mention Node.js one more time. Some of you might know The Node Firm, but for those who don’t, think of it as a company made up of the core contributors to Node.js that come together as a collective of professionals to offer services around Node.js that are complementary to the open source project. Things like consulting, technical vetting, training, things that an open source project rarely can do itself.

    The key part for me to like The Node Firm (I joined recently) is that, and I repeat, it is complementary to the open source project. It doesn’t try to extract any value from the project, but merely satisfies demand for services that the project can provide. Finally, proceeds are going right back into the community, in the form of conference support, swag, stickers, t-shirts, and so on.

    I think this is a brilliant set up for an open source company and I hope many will fork this model.

    With that as a preamble, I am happy to announce that together with Dave Cottlehuber and the preliminary support of a few community members, we are launching The Couch Firm, modelled after The Node Firm.

    It is early days for us, but expect the same setup, a business complementary to the open source project.

    Conclusion

    Above all, Open Source is about transparency and community, and I hope you feel you have an accurate picture of where we’re at right now. We had a hard year, lost our traction, and we still came out on top.

    With this, I’ll leave the stage for a day of exciting speakers with all sorts of crazy cool topics.

    Thank you all for being here!

    February 01, 2013 08:00 AM

    CouchDB Conf Recap

    On January 25-27th 2013 we ran CouchDB Conf Berlin. A one-day conference and two-day CouchHack event dedicated to the Apache CouchDB community and technology.

    Why

    CouchDB Conf was born out of the desire to run an Apache CouchDB Technology conference. Couchbase’s CouchConf used to be that, but has moved away, while sadly keeping the name.

    The CouchDB community is a busy bunch and any excuse to get together is taken up with enthusiasm. CouchDB Conf was sold out with ~70 participants in a wintery Berlin.

    Conferences in general are great for various reasons:

    • the core developers get some face time to have a good time, but also to get neck deep into discussions and hacking that require in-person communication. Lots of future plans can be hashed out quickly as well. We made good use of that.

    • the user community gets to hang out with the core developers, learn about how they work and most importantly, get to ask all the fun, hard, silly and important questions they never dared ask before. We also made good use of that.

    • Everybody gets to show of what they are working on and get everybody else excited about it. We made extensive use of that. Exictement was rampant.

    Eventify captured a few good impressions.


    Talks

    The slides for the talks are all linked up on the conference site.

    One common thread through all presentations that CouchDB’s replication feature is most the popular thing CouchDB has to offer. HTTP and JSON are nice and simple, but replication is the one thing all would go out of their way to point out that it is the thing that makes CouchDB most attractive.

    CouchDB’s vision early on was to be able to run on small clients, big servers, with lots of data and very little data, mobile phones and really any device that one can carry around, and crazily enough also in the browser. For the longest time though this was a bit of a pipe dream, as the Erlang CouchDB implementation wasn’t really suited to run on low-power devices and definitely not in a browser.

    This now changed. BigCouch makes the start, as it handles CouchDB’s BigData and clustering aspirations. It has been around for a while, but is now finally being integrated into Apache CouchDB proper.

    Hot on the heels is TouchDB, which comes in two flavours: iOS and Android, or Objective-C and Java respectively. Both are native reimplementations of CouchDB for mobile devices. TouchDB-iOS is production ready (a few apps are already live), the Android version is a little behind and both are open for contributions.

    The newest player on the scene is PouchDB, a pure JavaScript implementation of CouchDB targeted at modern desktop and mobile browsers. Albeit being the newest member of the CouchDB family, it has the highest activity and was recently selected to act as a sample project in the Stanford Computer Science course. It is on the verge of being production ready.

    All of this finally brings CouchDB’s vision together. Lots of data can live on BigCouch clusters, regular data can live on regular servers, user-data can live on mobile handsets, tablets and even in the browser and data can float freely between all of them. Application developers no longer have to worry about how to get a user’s data in front of them, handle concurrent updates, sync with offline devices and online servers. Even peer-to-peer sync is possible.

    Towards the end of the day, when the whole vision inadvertently came together, the energy levels were very high. And nothing fuels developers better than seeing users getting excited.

    Shit is getting real, as the kids say.

    CouchHack

    The following weekend was dedicated to hands on hacking, some more cool show & tell type talks and a Skype-powered video talk from Bangkok.

    Among the highlights are:

    • CouchDB/rcouch running on the RaspberryPi with Zeroconf auto-networking and everything.
    • A preliminary port to BlackBerry 10.
    • Wallaby, a native QT-based app framework that uses CouchDB for data storage & sync.
    • Hoodie, a pet project of ours.
    • CouchDB Operations.
    • Hands on hacking with PouchDB.
    • Hands on hacking with TouchDB.
    • Hands on hacking with PouchDB and TouchDB.
    • Clearing out 30+ issues in preparation for the 1.3.0 release.
    • Adding GeoCouch functionality to PouchDB.

    Use the Time Slider feature to see the topics for all days.

    Thanks

    As the organiser, I couldn’t be more happy about how CouchDB Conf has turned out. Of course, I couldn’t have it all done by myself, so I’d like to thank the following people and companies:

    The Speakers

    Katrin Apel, Dale Harvey, David Zuelke, Martin Hilbig, Nuno Job, Volker Mische, Oliver Kurowski, Jason Smith, Robert Newson, Joan Touzet, Stephan Strohmeier, Till Klampaeckel and Randall Leeds.

    The Helpers

    Gregor Martynus, Dave Cottlehuber and Jan Friedensreich.

    The Companies

    Co.Up for hosting us, Tito.io for the flawless ticketing, GitHub & Cloudant for the drinks, Nodejitsu, FreshX, Mozilla, EasyBib and IrisCouch for sending people or donating company time.

    Next

    There are preliminary plans to set up a CouchDB Conf in Seattle later this year. We’ll keep you posted.

    Members of the CouchDB Community will be attending the following conferences:

    You are welcome to join us and nerd out about CouchDB :)

    Thanks!
    Jan

    February 01, 2013 08:00 AM

    Changes and New Beginnings

    About two weeks ago I parted ways with Couchbase’s day to day business. Not that I was involved much in the end, but it was time to move on. I remain a co-founder and shareholder, but my regular involvement is over.

    This wasn’t an easy decision, and as those go, it took me a good while to make it.

    About a year ago I started noticing a change. Things I’ve always cared about intensified manyfold and my work situation didn’t leave me with much time and energy to pursue them.

    Couchbase made every attempt to accommodate me, let me work on Apache CouchDB for most of the past year, and help with an arrangement where I could spend some of my time at my discretion (with no pay, but enough work to keep me afloat) and I am very grateful for that opportunity.

    I spare you the details (ask me over a drink), but in the end I had to answer the question of whether I am a CouchDB person or a Couchbase person. I had to answer that for my working relationship with Couchbase and ultimately I had to answer that for myself.

    My heart knew, of course, but leaving a company that you helped founding isn’t the easiest of things to consider. I carried that conflict through most of 2012 and I tried to make things work in a compromise, but ultimately failed, realising that I am, at heart, a CouchDB person.


    2012 wasn’t the best of years for CouchDB. That helped me realise how much I believe in the idea and the community behind it.

    I learned a lot about myself, turned 30 in December. Around that time, the introspective period of the year turned into action. I was done contemplating what to do with newfound revelations and had a vague plan of the direction things should go towards.


    There are a number of distinct moments that helped me figure out what is important to me.

    Rails Girls

    The Rails Girls movement teaches non-programmers web programming for free. Volunteers from the Rails community would spend their weekends teaching, regular week-nights coaching development groups. All stuff I thought was honourable, but I didn’t see how I could fit that into my schedule, along my regular work and have enough energy to do it all. Yet I saw others doing just that. This created an urgency for changes, so I could, while not teaching Rails Girls, do things I deem important. More on that later.

    In the spirit of Rails Girls we started a JavaScript workshop JSFAB (JavaScript for Absolute Beginners) which, along with other similar initiatives eventually culminated in OpenTechSchool, a roof organisation for volunteer learning and teaching. The idea spreads like fire, still, into other tech communities, and other cities and countries. This is truly a tech education innovation. I am very proud to be a part of this.

    Gleicherlohn.de

    Around the US Equal Pay Day Gina Trapani put up Narrow the Gapp. Anne suggested there should be something like that for Germany, so I spent a total of four or so hours to rip off the project’s UI (open source) and made it so Anne could put in the hard data into a JSON file on GitHub.

    Gleicherlohn.de was a mini project with a large effect. The lesson for me though was that web technology, while incredibly accessible already, is still not easy enough to handle for regular people, even technically inclined non-programmers struggle. Why, I thought, would Anne not be able to put this up all by herself.

    Then I realised what my original motivation for working on CouchDB and eventually founding Couch.io was: empowering people to use technology to their advantage. I wanted to get back to that.

    Giving Back

    While talking to friends last winter I made the realisation that we technology people are a rather privileged bunch. Not that there is anything wrong with that, but for me that means finding ways to give back to the communities that enable me to live such a privileged life. Teaching and making technology more accessible fit right in there, so it wasn’t hard to figure out what to do. Instead of working five or seven days a week, I tried to find a way to work as much as I need to live comfortably, while having enough time and energy to pursue pro-bono work. Again, Couchbase was very accommodating here.

    Diversity

    One last topic that found its way to me is diversity in tech communities. The numbers don’t look particularly good, so I tried to find ways to get this improved. This lead to helping Tiffany to get We Are All Awesome set up, start our diversity-friendly Call for Papers for JSConf EU and the result that we were able to organically get 25% women speakers, an absolute novelty for hardcore tech conferences.

    More importantly though is that we inspired many more events to follow in our footsteps, improve on what we did and share the results. I lost count of how many conferences took after us, but is easily 20+. The resulting cultural change is immense. I couldn’t be more proud of that.


    I want to make sure I can continue to do this work and I hope I can inspire more people to do the same in the way that works best for them. In terms of work, I want to be able to work on things that matter and get paid fairly, but also not over-do it. Leave enough time to work outside of work.

    To that end, I am doing three things:

    The Node Firm is a complementary company to the Node.js open source project. A collaboration of professionals to support the open source projects in areas it can’t provide itself.

    The Couch Firm is exactly that, but for CouchDB. It is early days, I’ll keep you posted.

    Hoodie is a thing to be revealed at a later time, but it is not exactly a secret, if you know where to look.


    2013 looks mighty exciting from where I am sitting.

    I can’t wait to get started!

    February 01, 2013 08:00 AM

    The State of CouchDB

    Slides to go with the headings.


    Hi, welcome to CouchDB Conf.

    CouchDB means a lot to me and it makes me very happy that you are all here today.

    Who am I

    My name is Jan Lehnardt and I am the elected project management committee chair of the Apache CouchDB project. If you don’t know what this means, don’t worry, it means I sign up to deal with politics and paperwork voluntarily.

    I have been with CouchDB since 2006. My first contribution was the initial UNIX build system to CouchDB. I stuck around ever since.

    My main job back then was PHP/MySQL consulting and Open Source work as my own 20% project. In 2007, I started giving talks about CouchDB at user groups and conferences and quickly became addicted. People called me the CouchDB Chief Evangelist, although I don’t really like the title. From 2008 I started getting contracting jobs specifically for CouchDB and things blew up quickly that it became clear soon, that an organisation of some sort was needed to handle this. I eventually joined forces with CouchDB’s original inventor and J Chris Anderson to form the company Couch.io. We initially weren’t keen on taking venture capital, but when the whole NoSQL thing blew up, it was very hard to say no and on January 1st 2010 we were in business.

    Couch.io turned into CouchOne and was eventually merged with Membase into Couchbase. Unfortunately, at this point, there isn’t much Couch-y about Couchbase aside from some underlying technology, so the name can be confusing.

    I parted ways with Couchbase a week ago in order to concentrate and focus more on Apache CouchDB and related technologies.

    CouchDB isn’t Dead

    In late 2011 two major supporters of CouchDB withdrew their support: Canonical’s UbuntuOne service, on paper a poster-child use-case for CouchDB and Saucelab’s Selenium platform publicly made clear that they were moving to other technology. It wasn’t a big deal if you looked closely, the one project didn’t have enough resources to integrate with CouchDB the right way and the other diverged so far from the original system design, that CouchDB was no longer a good fit. We were and we remain good friends with the respective developers, so we didn’t feel like pointing that out specifically. The result though was that people who didn’t look closely saw that CouchDB had popular failures where it hadn’t. This wasn’t all bad, but set the stage for another incident a few weeks later.

    CouchDB’s original inventor publicly announced that he’d move on from Apache CouchDB to Couchbase fully. That new technology would only go into Couchbase. If you are in the know, this wasn’t a big deal again, he hadn’t contributed to CouchDB in two years, Couchbase was a different thing and he wanted to rally support for that. The casual reader though took away one lesson: “CouchDB is dead”.

    With the added confusion of Couchbase’s name, and continuing their CouchConf conferences, things looks dire.

    CouchDB 2012

    With this in the wild, the active CouchDB committers got together and put out an excellent 1.2.0 release that was accompanied by a new, modern website design. We thought instead of writing counter-statements and start unnecessary he-said-she-said discussions, we’d rather concentrate on shipping good software. 1.2.0 was well received and the whole thing lead the team to come together as a whole in Boston and two weeks later in a smaller group in Dublin in April 2012.

    We came up with a long list of of features that we wanted to get into CouchDB. Some work happened over the summer, but I personally took a bit of a break to avoid burn-out from CouchDB. In fall, things started moving faster again when more community members started pushing things along. New contributors came and we finally came up with a way to go forward.

    We decided to not be insane. The definition of insanity is trying the same thing over and over again and expecting different results. While it wasn’t all bad in CouchDB land we could do better and to do better we had to change a few things.

    CouchDB Reboot

    We determined that CouchDB needs a fresh mission statement. I signed up to come up with a few drafts, but nothing happened yet, so this one is still out there cough.

    We decided to release more often. With the upcoming 1.3.0 release we will switch to a quarterly release cycle. A new minor version will come out approximately every 90 days, even if it is just a few bug fixes that a small number of people direly need.

    CouchDB 1.3.0 will ship with top-notch API documentation right inside of Futon, the built in admin interface.

    We will start to set up teams where anyone can sign up to work on a specific thing: docs, our JavaScript components, the Erlang core, the admin interface (built in HTML+JS). We want to make sure anyone who has the right skills can contribute in their area of expertise and not be scared away by the Erlang.

    We put some emphasis on QA, automated testing, building binaries and packages for releases as well as in-development versions, so people can try new features before they come out more easily and as a result report issues early.

    A few of the core team will put some focus on encouraging contributions rather than coding themselves. We hope to get more people on board with CouchDB development.

    We also started doing roughly quarterly in-person meetings (at least as smaller groups) and weekly status meetings on IRC to give everything a little more focus and transparency.

    The next few slides show a long list a few features that we’ll be working on. If you think anything is missing, let us know!

    The more prominent things are: Merging rcouch, BigCouch, the new Futon, the plugin system and v8/node integration.

    rcouch is a cutting edge distribution of CouchDB with many experimental features. A lot of that work is useful for Apache CouchDB and we plan to fold those back into the project.

    BigCouch is a industry-strength clustering solution for CouchDB to allow seamless scaling and storing of lots of data. BigCouch is going to be merged into mainline CouchDB as well.

    The admin interface Futon is getting a complete refresh with modern web development technologies.

    There are many features hidden in some developer’s git repository because, while useful, it wasn’t generally applicable to CouchDB. With the plugin system, we hope that we can fuel a repository of plugins to CouchDB that are easily installed (think browser extensions, right from Futon). For example, you could install GeoCouch with the click of a button.

    Finally, we are currently discussing the move from SpiderMonkey and to V8 or even Node. The jury is still out on which one it is going to be, but we hope that this is going to open up a lot more opportunities for CouchDB in terms of features and new contributors.


    Community

    CouchDB would be nothing without you, the community. People who know little about CouchDB might like it or not, but people who really get to know CouchDB are extremely passionate about it.

    CouchDB’s core idea, the promise that data can be anywhere it is needed, to make applications easier, more approachable and effective for people in modern computing environments, that idea is a big one. It’s the one that brings this community together.

    Thanks again for coming, you are all awesome. CouchDB means a lot to me and I get a little teary seeing you all here.

    If you like any of the above todos, or have some of your own, please help out, get in touch about how.

    Friends of CouchDB

    The CouchDB ecosystem is full of exciting projects that support CouchDB in one way or another.

    PouchDB

    PouchDB is the newest project on the scene. PouchDB is a pure JavaScript implementation of CouchDB targeted at the browser.

    Let me say that again: CouchDB in JavaScript, for the browser.

    PouchDB is the oldest idea, but it took the longest to realise because browsers weren’t ready for it. Now is the time though that we can build and ship a CouchDB compatible data store and replication engine that lives entirely in your browser. If you want to build web apps that can seamlessly synchronise data between a laptop, a phone and a server and not treat one as a “master” and the other as “slaves”, PouchDB is your solution.

    TouchDB

    TouchDB-iOS and -Android are Objective-C and Java implementations of the same idea, but for mobile phone platforms. They natively integrate into your iOS and Android applications and allow synchronisation between themselves, multiple devices, CouchDB’s on the server and even PouchDB in a browser. Again, a milestone in fulfilling the CouchDB vision: data where it is needed, not on some server, far away over a slow and unreliable or even unavailable connection.

    Node.js

    This might not be too obvious to people who don’t do Node, but CouchDB is big, if not essential here, too. As any programming language & platform, Node.js comes with a package manager called NPM. As far as package managers go (and I’ve used them all except I think ant) NPM is the one that sucks least, or rather, it is pretty good.

    If you are invested in Node.js programming NPM isn’t just a tool you can use, it is essential to the development and deployment workflow of your applications. It is that good.

    And here is the kicker: NPM’s (de)-central registry is build on CouchDB. NPM is effectively a CouchApp. The Node elders love CouchDB.

    I suggest, and plan to turn this into an opportunity for CouchDB. Say, for example, we need benchmarking tools for CouchDB. The Node folks are great at that. If we were to go and say, “hey, do you want to help make NPM faster? Help us build benchmarking tools, so we can speed up CouchDB for you.” — And that’s just one idea, the potential collaboration is immense. Add to that that we maybe get native Node support inside CouchDB, we can do all sorts of crazy shit like writing core add-ons to CouchDB in JavaScript.


    Finally, I’d like to mention Hoodie, a project that Gregor, Alex, I and a few friends have been working on on and of over the past year. It is frontend web app framework that uses Node and CouchDB. We are happy to show that to you on the weekend.

    The Node Firm

    I am going to mention Node.js one more time. Some of you might know The Node Firm, but for those who don’t, think of it as a company made up of the core contributors to Node.js that come together as a collective of professionals to offer services around Node.js that are complementary to the open source project. Things like consulting, technical vetting, training, things that an open source project rarely can do itself.

    The key part for me to like The Node Firm (I joined recently) is that, and I repeat, it is complementary to the open source project. It doesn’t try to extract any value from the project, but merely satisfies demand for services that the project can provide. Finally, proceeds are going right back into the community, in the form of conference support, swag, stickers, t-shirts, and so on.

    I think this is a brilliant set up for an open source company and I hope many will fork this model.

    With that as a preamble, I am happy to announce that together with Dave Cottlehuber and the preliminary support of a few community members, we are launching The Couch Firm, modelled after The Node Firm.

    It is early days for us, but expect the same setup, a business complementary to the open source project.

    Conclusion

    Above all, Open Source is about transparency and community, and I hope you feel you have an accurate picture of where we’re at right now. We had a hard year, lost our traction, and we still came out on top.

    With this, I’ll leave the stage for a day of exciting speakers with all sorts of crazy cool topics.

    Thank you all for being here!

    February 01, 2013 08:00 AM

    The State of CouchDB

    Slides to go with the headings.


    Hi, welcome to CouchDB Conf.

    CouchDB means a lot to me and it makes me very happy that you are all here today.

    Who am I

    My name is Jan Lehnardt and I am the elected project management committee chair of the Apache CouchDB project. If you don’t know what this means, don’t worry, it means I sign up to deal with politics and paperwork voluntarily.

    I have been with CouchDB since 2006. My first contribution was the initial UNIX build system to CouchDB. I stuck around ever since.

    My main job back then was PHP/MySQL consulting and Open Source work as my own 20% project. In 2007, I started giving talks about CouchDB at user groups and conferences and quickly became addicted. People called me the CouchDB Chief Evangelist, although I don’t really like the title. From 2008 I started getting contracting jobs specifically for CouchDB and things blew up quickly that it became clear soon, that an organisation of some sort was needed to handle this. I eventually joined forces with CouchDB’s original inventor and J Chris Anderson to form the company Couch.io. We initially weren’t keen on taking venture capital, but when the whole NoSQL thing blew up, it was very hard to say no and on January 1st 2010 we were in business.

    Couch.io turned into CouchOne and was eventually merged with Membase into Couchbase. Unfortunately, at this point, there isn’t much Couch-y about Couchbase aside from some underlying technology, so the name can be confusing.

    I parted ways with Couchbase a week ago in order to concentrate and focus more on Apache CouchDB and related technologies.

    CouchDB isn’t Dead

    In late 2011 two major supporters of CouchDB withdrew their support: Canonical’s UbuntuOne service, on paper a poster-child use-case for CouchDB and Saucelab’s Selenium platform publicly made clear that they were moving to other technology. It wasn’t a big deal if you looked closely, the one project didn’t have enough resources to integrate with CouchDB the right way and the other diverged so far from the original system design, that CouchDB was no longer a good fit. We were and we remain good friends with the respective developers, so we didn’t feel like pointing that out specifically. The result though was that people who didn’t look closely saw that CouchDB had popular failures where it hadn’t. This wasn’t all bad, but set the stage for another incident a few weeks later.

    CouchDB’s original inventor publicly announced that he’d move on from Apache CouchDB to Couchbase fully. That new technology would only go into Couchbase. If you are in the know, this wasn’t a big deal again, he hadn’t contributed to CouchDB in two years, Couchbase was a different thing and he wanted to rally support for that. The casual reader though took away one lesson: “CouchDB is dead”.

    With the added confusion of Couchbase’s name, and continuing their CouchConf conferences, things looks dire.

    CouchDB 2012

    With this in the wild, the active CouchDB committers got together and put out an excellent 1.2.0 release that was accompanied by a new, modern website design. We thought instead of writing counter-statements and start unnecessary he-said-she-said discussions, we’d rather concentrate on shipping good software. 1.2.0 was well received and the whole thing lead the team to come together as a whole in Boston and two weeks later in a smaller group in Dublin in April 2012.

    We came up with a long list of of features that we wanted to get into CouchDB. Some work happened over the summer, but I personally took a bit of a break to avoid burn-out from CouchDB. In fall, things started moving faster again when more community members started pushing things along. New contributors came and we finally came up with a way to go forward.

    We decided to not be insane. The definition of insanity is trying the same thing over and over again and expecting different results. While it wasn’t all bad in CouchDB land we could do better and to do better we had to change a few things.

    CouchDB Reboot

    We determined that CouchDB needs a fresh mission statement. I signed up to come up with a few drafts, but nothing happened yet, so this one is still out there cough.

    We decided to release more often. With the upcoming 1.3.0 release we will switch to a quarterly release cycle. A new minor version will come out approximately every 90 days, even if it is just a few bug fixes that a small number of people direly need.

    CouchDB 1.3.0 will ship with top-notch API documentation right inside of Futon, the built in admin interface.

    We will start to set up teams where anyone can sign up to work on a specific thing: docs, our JavaScript components, the Erlang core, the admin interface (built in HTML+JS). We want to make sure anyone who has the right skills can contribute in their area of expertise and not be scared away by the Erlang.

    We put some emphasis on QA, automated testing, building binaries and packages for releases as well as in-development versions, so people can try new features before they come out more easily and as a result report issues early.

    A few of the core team will put some focus on encouraging contributions rather than coding themselves. We hope to get more people on board with CouchDB development.

    We also started doing roughly quarterly in-person meetings (at least as smaller groups) and weekly status meetings on IRC to give everything a little more focus and transparency.

    The next few slides show a long list a few features that we’ll be working on. If you think anything is missing, let us know!

    The more prominent things are: Merging rcouch, BigCouch, the new Futon, the plugin system and v8/node integration.

    rcouch is a cutting edge distribution of CouchDB with many experimental features. A lot of that work is useful for Apache CouchDB and we plan to fold those back into the project.

    BigCouch is a industry-strength clustering solution for CouchDB to allow seamless scaling and storing of lots of data. BigCouch is going to be merged into mainline CouchDB as well.

    The admin interface Futon is getting a complete refresh with modern web development technologies.

    There are many features hidden in some developer’s git repository because, while useful, it wasn’t generally applicable to CouchDB. With the plugin system, we hope that we can fuel a repository of plugins to CouchDB that are easily installed (think browser extensions, right from Futon). For example, you could install GeoCouch with the click of a button.

    Finally, we are currently discussing the move from SpiderMonkey and to V8 or even Node. The jury is still out on which one it is going to be, but we hope that this is going to open up a lot more opportunities for CouchDB in terms of features and new contributors.


    Community

    CouchDB would be nothing without you, the community. People who know little about CouchDB might like it or not, but people who really get to know CouchDB are extremely passionate about it.

    CouchDB’s core idea, the promise that data can be anywhere it is needed, to make applications easier, more approachable and effective for people in modern computing environments, that idea is a big one. It’s the one that brings this community together.

    Thanks again for coming, you are all awesome. CouchDB means a lot to me and I get a little teary seeing you all here.

    If you like any of the above todos, or have some of your own, please help out, get in touch about how.

    Friends of CouchDB

    The CouchDB ecosystem is full of exciting projects that support CouchDB in one way or another.

    PouchDB

    PouchDB is the newest project on the scene. PouchDB is a pure JavaScript implementation of CouchDB targeted at the browser.

    Let me say that again: CouchDB in JavaScript, for the browser.

    PouchDB is the oldest idea, but it took the longest to realise because browsers weren’t ready for it. Now is the time though that we can build and ship a CouchDB compatible data store and replication engine that lives entirely in your browser. If you want to build web apps that can seamlessly synchronise data between a laptop, a phone and a server and not treat one as a “master” and the other as “slaves”, PouchDB is your solution.

    TouchDB

    TouchDB-iOS and -Android are Objective-C and Java implementations of the same idea, but for mobile phone platforms. They natively integrate into your iOS and Android applications and allow synchronisation between themselves, multiple devices, CouchDB’s on the server and even PouchDB in a browser. Again, a milestone in fulfilling the CouchDB vision: data where it is needed, not on some server, far away over a slow and unreliable or even unavailable connection.

    Node.js

    This might not be too obvious to people who don’t do Node, but CouchDB is big, if not essential here, too. As any programming language & platform, Node.js comes with a package manager called NPM. As far as package managers go (and I’ve used them all except I think ant) NPM is the one that sucks least, or rather, it is pretty good.

    If you are invested in Node.js programming NPM isn’t just a tool you can use, it is essential to the development and deployment workflow of your applications. It is that good.

    And here is the kicker: NPM’s (de)-central registry is build on CouchDB. NPM is effectively a CouchApp. The Node elders love CouchDB.

    I suggest, and plan to turn this into an opportunity for CouchDB. Say, for example, we need benchmarking tools for CouchDB. The Node folks are great at that. If we were to go and say, “hey, do you want to help make NPM faster? Help us build benchmarking tools, so we can speed up CouchDB for you.” — And that’s just one idea, the potential collaboration is immense. Add to that that we maybe get native Node support inside CouchDB, we can do all sorts of crazy shit like writing core add-ons to CouchDB in JavaScript.


    Finally, I’d like to mention Hoodie, a project that Gregor, Alex, I and a few friends have been working on on and of over the past year. It is frontend web app framework that uses Node and CouchDB. We are happy to show that to you on the weekend.

    The Node Firm

    I am going to mention Node.js one more time. Some of you might know The Node Firm, but for those who don’t, think of it as a company made up of the core contributors to Node.js that come together as a collective of professionals to offer services around Node.js that are complementary to the open source project. Things like consulting, technical vetting, training, things that an open source project rarely can do itself.

    The key part for me to like The Node Firm (I joined recently) is that, and I repeat, it is complementary to the open source project. It doesn’t try to extract any value from the project, but merely satisfies demand for services that the project can provide. Finally, proceeds are going right back into the community, in the form of conference support, swag, stickers, t-shirts, and so on.

    I think this is a brilliant set up for an open source company and I hope many will fork this model.

    With that as a preamble, I am happy to announce that together with Dave Cottlehuber and the preliminary support of a few community members, we are launching The Couch Firm, modelled after The Node Firm.

    It is early days for us, but expect the same setup, a business complementary to the open source project.

    Conclusion

    Above all, Open Source is about transparency and community, and I hope you feel you have an accurate picture of where we’re at right now. We had a hard year, lost our traction, and we still came out on top.

    With this, I’ll leave the stage for a day of exciting speakers with all sorts of crazy cool topics.

    Thank you all for being here!

    February 01, 2013 08:00 AM

    CouchDB Conf Recap

    On January 25-27th 2013 we ran CouchDB Conf Berlin. A one-day conference and two-day CouchHack event dedicated to the Apache CouchDB community and technology.

    Why

    CouchDB Conf was born out of the desire to run an Apache CouchDB Technology conference. Couchbase’s CouchConf used to be that, but has moved away, while sadly keeping the name.

    The CouchDB community is a busy bunch and any excuse to get together is taken up with enthusiasm. CouchDB Conf was sold out with ~70 participants in a wintery Berlin.

    Conferences in general are great for various reasons:

    • the core developers get some face time to have a good time, but also to get neck deep into discussions and hacking that require in-person communication. Lots of future plans can be hashed out quickly as well. We made good use of that.

    • the user community gets to hang out with the core developers, learn about how they work and most importantly, get to ask all the fun, hard, silly and important questions they never dared ask before. We also made good use of that.

    • Everybody gets to show of what they are working on and get everybody else excited about it. We made extensive use of that. Exictement was rampant.

    Eventify captured a few good impressions.


    Talks

    The slides for the talks are all linked up on the conference site.

    One common thread through all presentations that CouchDB’s replication feature is most the popular thing CouchDB has to offer. HTTP and JSON are nice and simple, but replication is the one thing all would go out of their way to point out that it is the thing that makes CouchDB most attractive.

    CouchDB’s vision early on was to be able to run on small clients, big servers, with lots of data and very little data, mobile phones and really any device that one can carry around, and crazily enough also in the browser. For the longest time though this was a bit of a pipe dream, as the Erlang CouchDB implementation wasn’t really suited to run on low-power devices and definitely not in a browser.

    This now changed. BigCouch makes the start, as it handles CouchDB’s BigData and clustering aspirations. It has been around for a while, but is now finally being integrated into Apache CouchDB proper.

    Hot on the heels is TouchDB, which comes in two flavours: iOS and Android, or Objective-C and Java respectively. Both are native reimplementations of CouchDB for mobile devices. TouchDB-iOS is production ready (a few apps are already live), the Android version is a little behind and both are open for contributions.

    The newest player on the scene is PouchDB, a pure JavaScript implementation of CouchDB targeted at modern desktop and mobile browsers. Albeit being the newest member of the CouchDB family, it has the highest activity and was recently selected to act as a sample project in the Stanford Computer Science course. It is on the verge of being production ready.

    All of this finally brings CouchDB’s vision together. Lots of data can live on BigCouch clusters, regular data can live on regular servers, user-data can live on mobile handsets, tablets and even in the browser and data can float freely between all of them. Application developers no longer have to worry about how to get a user’s data in front of them, handle concurrent updates, sync with offline devices and online servers. Even peer-to-peer sync is possible.

    Towards the end of the day, when the whole vision inadvertently came together, the energy levels were very high. And nothing fuels developers better than seeing users getting excited.

    Shit is getting real, as the kids say.

    CouchHack

    The following weekend was dedicated to hands on hacking, some more cool show & tell type talks and a Skype-powered video talk from Bangkok.

    Among the highlights are:

    • CouchDB/rcouch running on the RaspberryPi with Zeroconf auto-networking and everything.
    • A preliminary port to BlackBerry 10.
    • Wallaby, a native QT-based app framework that uses CouchDB for data storage & sync.
    • Hoodie, a pet project of ours.
    • CouchDB Operations.
    • Hands on hacking with PouchDB.
    • Hands on hacking with TouchDB.
    • Hands on hacking with PouchDB and TouchDB.
    • Clearing out 30+ issues in preparation for the 1.3.0 release.
    • Adding GeoCouch functionality to PouchDB.

    Use the Time Slider feature to see the topics for all days.

    Thanks

    As the organiser, I couldn’t be more happy about how CouchDB Conf has turned out. Of course, I couldn’t have it all done by myself, so I’d like to thank the following people and companies:

    The Speakers

    Katrin Apel, Dale Harvey, David Zuelke, Martin Hilbig, Nuno Job, Volker Mische, Oliver Kurowski, Jason Smith, Robert Newson, Joan Touzet, Stephan Strohmeier, Till Klampaeckel and Randall Leeds.

    The Helpers

    Gregor Martynus, Dave Cottlehuber and Jan Friedensreich.

    The Companies

    Co.Up for hosting us, Tito.io for the flawless ticketing, GitHub & Cloudant for the drinks, Nodejitsu, FreshX, Mozilla, EasyBib and IrisCouch for sending people or donating company time.

    Next

    There are preliminary plans to set up a CouchDB Conf in Seattle later this year. We’ll keep you posted.

    Members of the CouchDB Community will be attending the following conferences:

    You are welcome to join us and nerd out about CouchDB :)

    Thanks!
    Jan

    February 01, 2013 08:00 AM

    CouchDB Conf Recap

    On January 25-27th 2013 we ran CouchDB Conf Berlin. A one-day conference and two-day CouchHack event dedicated to the Apache CouchDB community and technology.

    Why

    CouchDB Conf was born out of the desire to run an Apache CouchDB Technology conference. Couchbase’s CouchConf used to be that, but has moved away, while sadly keeping the name.

    The CouchDB community is a busy bunch and any excuse to get together is taken up with enthusiasm. CouchDB Conf was sold out with ~70 participants in a wintery Berlin.

    Conferences in general are great for various reasons:

    • the core developers get some face time to have a good time, but also to get neck deep into discussions and hacking that require in-person communication. Lots of future plans can be hashed out quickly as well. We made good use of that.

    • the user community gets to hang out with the core developers, learn about how they work and most importantly, get to ask all the fun, hard, silly and important questions they never dared ask before. We also made good use of that.

    • Everybody gets to show of what they are working on and get everybody else excited about it. We made extensive use of that. Exictement was rampant.

    Eventify captured a few good impressions.


    Talks

    The slides for the talks are all linked up on the conference site.

    One common thread through all presentations that CouchDB’s replication feature is most the popular thing CouchDB has to offer. HTTP and JSON are nice and simple, but replication is the one thing all would go out of their way to point out that it is the thing that makes CouchDB most attractive.

    CouchDB’s vision early on was to be able to run on small clients, big servers, with lots of data and very little data, mobile phones and really any device that one can carry around, and crazily enough also in the browser. For the longest time though this was a bit of a pipe dream, as the Erlang CouchDB implementation wasn’t really suited to run on low-power devices and definitely not in a browser.

    This now changed. BigCouch makes the start, as it handles CouchDB’s BigData and clustering aspirations. It has been around for a while, but is now finally being integrated into Apache CouchDB proper.

    Hot on the heels is TouchDB, which comes in two flavours: iOS and Android, or Objective-C and Java respectively. Both are native reimplementations of CouchDB for mobile devices. TouchDB-iOS is production ready (a few apps are already live), the Android version is a little behind and both are open for contributions.

    The newest player on the scene is PouchDB, a pure JavaScript implementation of CouchDB targeted at modern desktop and mobile browsers. Albeit being the newest member of the CouchDB family, it has the highest activity and was recently selected to act as a sample project in the Stanford Computer Science course. It is on the verge of being production ready.

    All of this finally brings CouchDB’s vision together. Lots of data can live on BigCouch clusters, regular data can live on regular servers, user-data can live on mobile handsets, tablets and even in the browser and data can float freely between all of them. Application developers no longer have to worry about how to get a user’s data in front of them, handle concurrent updates, sync with offline devices and online servers. Even peer-to-peer sync is possible.

    Towards the end of the day, when the whole vision inadvertently came together, the energy levels were very high. And nothing fuels developers better than seeing users getting excited.

    Shit is getting real, as the kids say.

    CouchHack

    The following weekend was dedicated to hands on hacking, some more cool show & tell type talks and a Skype-powered video talk from Bangkok.

    Among the highlights are:

    • CouchDB/rcouch running on the RaspberryPi with Zeroconf auto-networking and everything.
    • A preliminary port to BlackBerry 10.
    • Wallaby, a native QT-based app framework that uses CouchDB for data storage & sync.
    • Hoodie, a pet project of ours.
    • CouchDB Operations.
    • Hands on hacking with PouchDB.
    • Hands on hacking with TouchDB.
    • Hands on hacking with PouchDB and TouchDB.
    • Clearing out 30+ issues in preparation for the 1.3.0 release.
    • Adding GeoCouch functionality to PouchDB.

    Use the Time Slider feature to see the topics for all days.

    Thanks

    As the organiser, I couldn’t be more happy about how CouchDB Conf has turned out. Of course, I couldn’t have it all done by myself, so I’d like to thank the following people and companies:

    The Speakers

    Katrin Apel, Dale Harvey, David Zuelke, Martin Hilbig, Nuno Job, Volker Mische, Oliver Kurowski, Jason Smith, Robert Newson, Joan Touzet, Stephan Strohmeier, Till Klampaeckel and Randall Leeds.

    The Helpers

    Gregor Martynus, Dave Cottlehuber and Jan Friedensreich.

    The Companies

    Co.Up for hosting us, Tito.io for the flawless ticketing, GitHub & Cloudant for the drinks, Nodejitsu, FreshX, Mozilla, EasyBib and IrisCouch for sending people or donating company time.

    Next

    There are preliminary plans to set up a CouchDB Conf in Seattle later this year. We’ll keep you posted.

    Members of the CouchDB Community will be attending the following conferences:

    You are welcome to join us and nerd out about CouchDB :)

    Thanks!
    Jan

    February 01, 2013 08:00 AM

    Changes and New Beginnings

    About two weeks ago I parted ways with Couchbase’s day to day business. Not that I was involved much in the end, but it was time to move on. I remain a co-founder and shareholder, but my regular involvement is over.

    This wasn’t an easy decision, and as those go, it took me a good while to make it.

    About a year ago I started noticing a change. Things I’ve always cared about intensified manyfold and my work situation didn’t leave me with much time and energy to pursue them.

    Couchbase made every attempt to accommodate me, let me work on Apache CouchDB for most of the past year, and help with an arrangement where I could spend some of my time at my discretion (with no pay, but enough work to keep me afloat) and I am very grateful for that opportunity.

    I spare you the details (ask me over a drink), but in the end I had to answer the question of whether I am a CouchDB person or a Couchbase person. I had to answer that for my working relationship with Couchbase and ultimately I had to answer that for myself.

    My heart knew, of course, but leaving a company that you helped founding isn’t the easiest of things to consider. I carried that conflict through most of 2012 and I tried to make things work in a compromise, but ultimately failed, realising that I am, at heart, a CouchDB person.


    2012 wasn’t the best of years for CouchDB. That helped me realise how much I believe in the idea and the community behind it.

    I learned a lot about myself, turned 30 in December. Around that time, the introspective period of the year turned into action. I was done contemplating what to do with newfound revelations and had a vague plan of the direction things should go towards.


    There are a number of distinct moments that helped me figure out what is important to me.

    Rails Girls

    The Rails Girls movement teaches non-programmers web programming for free. Volunteers from the Rails community would spend their weekends teaching, regular week-nights coaching development groups. All stuff I thought was honourable, but I didn’t see how I could fit that into my schedule, along my regular work and have enough energy to do it all. Yet I saw others doing just that. This created an urgency for changes, so I could, while not teaching Rails Girls, do things I deem important. More on that later.

    In the spirit of Rails Girls we started a JavaScript workshop JSFAB (JavaScript for Absolute Beginners) which, along with other similar initiatives eventually culminated in OpenTechSchool, a roof organisation for volunteer learning and teaching. The idea spreads like fire, still, into other tech communities, and other cities and countries. This is truly a tech education innovation. I am very proud to be a part of this.

    Gleicherlohn.de

    Around the US Equal Pay Day Gina Trapani put up Narrow the Gapp. Anne suggested there should be something like that for Germany, so I spent a total of four or so hours to rip off the project’s UI (open source) and made it so Anne could put in the hard data into a JSON file on GitHub.

    Gleicherlohn.de was a mini project with a large effect. The lesson for me though was that web technology, while incredibly accessible already, is still not easy enough to handle for regular people, even technically inclined non-programmers struggle. Why, I thought, would Anne not be able to put this up all by herself.

    Then I realised what my original motivation for working on CouchDB and eventually founding Couch.io was: empowering people to use technology to their advantage. I wanted to get back to that.

    Giving Back

    While talking to friends last winter I made the realisation that we technology people are a rather privileged bunch. Not that there is anything wrong with that, but for me that means finding ways to give back to the communities that enable me to live such a privileged life. Teaching and making technology more accessible fit right in there, so it wasn’t hard to figure out what to do. Instead of working five or seven days a week, I tried to find a way to work as much as I need to live comfortably, while having enough time and energy to pursue pro-bono work. Again, Couchbase was very accommodating here.

    Diversity

    One last topic that found its way to me is diversity in tech communities. The numbers don’t look particularly good, so I tried to find ways to get this improved. This lead to helping Tiffany to get We Are All Awesome set up, start our diversity-friendly Call for Papers for JSConf EU and the result that we were able to organically get 25% women speakers, an absolute novelty for hardcore tech conferences.

    More importantly though is that we inspired many more events to follow in our footsteps, improve on what we did and share the results. I lost count of how many conferences took after us, but is easily 20+. The resulting cultural change is immense. I couldn’t be more proud of that.


    I want to make sure I can continue to do this work and I hope I can inspire more people to do the same in the way that works best for them. In terms of work, I want to be able to work on things that matter and get paid fairly, but also not over-do it. Leave enough time to work outside of work.

    To that end, I am doing three things:

    The Node Firm is a complementary company to the Node.js open source project. A collaboration of professionals to support the open source projects in areas it can’t provide itself.

    The Couch Firm is exactly that, but for CouchDB. It is early days, I’ll keep you posted.

    Hoodie is a thing to be revealed at a later time, but it is not exactly a secret, if you know where to look.


    2013 looks mighty exciting from where I am sitting.

    I can’t wait to get started!

    February 01, 2013 08:00 AM

    Changes and New Beginnings

    About two weeks ago I parted ways with Couchbase’s day to day business. Not that I was involved much in the end, but it was time to move on. I remain a co-founder and shareholder, but my regular involvement is over.

    This wasn’t an easy decision, and as those go, it took me a good while to make it.

    About a year ago I started noticing a change. Things I’ve always cared about intensified manyfold and my work situation didn’t leave me with much time and energy to pursue them.

    Couchbase made every attempt to accommodate me, let me work on Apache CouchDB for most of the past year, and help with an arrangement where I could spend some of my time at my discretion (with no pay, but enough work to keep me afloat) and I am very grateful for that opportunity.

    I spare you the details (ask me over a drink), but in the end I had to answer the question of whether I am a CouchDB person or a Couchbase person. I had to answer that for my working relationship with Couchbase and ultimately I had to answer that for myself.

    My heart knew, of course, but leaving a company that you helped founding isn’t the easiest of things to consider. I carried that conflict through most of 2012 and I tried to make things work in a compromise, but ultimately failed, realising that I am, at heart, a CouchDB person.


    2012 wasn’t the best of years for CouchDB. That helped me realise how much I believe in the idea and the community behind it.

    I learned a lot about myself, turned 30 in December. Around that time, the introspective period of the year turned into action. I was done contemplating what to do with newfound revelations and had a vague plan of the direction things should go towards.


    There are a number of distinct moments that helped me figure out what is important to me.

    Rails Girls

    The Rails Girls movement teaches non-programmers web programming for free. Volunteers from the Rails community would spend their weekends teaching, regular week-nights coaching development groups. All stuff I thought was honourable, but I didn’t see how I could fit that into my schedule, along my regular work and have enough energy to do it all. Yet I saw others doing just that. This created an urgency for changes, so I could, while not teaching Rails Girls, do things I deem important. More on that later.

    In the spirit of Rails Girls we started a JavaScript workshop JSFAB (JavaScript for Absolute Beginners) which, along with other similar initiatives eventually culminated in OpenTechSchool, a roof organisation for volunteer learning and teaching. The idea spreads like fire, still, into other tech communities, and other cities and countries. This is truly a tech education innovation. I am very proud to be a part of this.

    Gleicherlohn.de

    Around the US Equal Pay Day Gina Trapani put up Narrow the Gapp. Anne suggested there should be something like that for Germany, so I spent a total of four or so hours to rip off the project’s UI (open source) and made it so Anne could put in the hard data into a JSON file on GitHub.

    Gleicherlohn.de was a mini project with a large effect. The lesson for me though was that web technology, while incredibly accessible already, is still not easy enough to handle for regular people, even technically inclined non-programmers struggle. Why, I thought, would Anne not be able to put this up all by herself.

    Then I realised what my original motivation for working on CouchDB and eventually founding Couch.io was: empowering people to use technology to their advantage. I wanted to get back to that.

    Giving Back

    While talking to friends last winter I made the realisation that we technology people are a rather privileged bunch. Not that there is anything wrong with that, but for me that means finding ways to give back to the communities that enable me to live such a privileged life. Teaching and making technology more accessible fit right in there, so it wasn’t hard to figure out what to do. Instead of working five or seven days a week, I tried to find a way to work as much as I need to live comfortably, while having enough time and energy to pursue pro-bono work. Again, Couchbase was very accommodating here.

    Diversity

    One last topic that found its way to me is diversity in tech communities. The numbers don’t look particularly good, so I tried to find ways to get this improved. This lead to helping Tiffany to get We Are All Awesome set up, start our diversity-friendly Call for Papers for JSConf EU and the result that we were able to organically get 25% women speakers, an absolute novelty for hardcore tech conferences.

    More importantly though is that we inspired many more events to follow in our footsteps, improve on what we did and share the results. I lost count of how many conferences took after us, but is easily 20+. The resulting cultural change is immense. I couldn’t be more proud of that.


    I want to make sure I can continue to do this work and I hope I can inspire more people to do the same in the way that works best for them. In terms of work, I want to be able to work on things that matter and get paid fairly, but also not over-do it. Leave enough time to work outside of work.

    To that end, I am doing three things:

    The Node Firm is a complementary company to the Node.js open source project. A collaboration of professionals to support the open source projects in areas it can’t provide itself.

    The Couch Firm is exactly that, but for CouchDB. It is early days, I’ll keep you posted.

    Hoodie is a thing to be revealed at a later time, but it is not exactly a secret, if you know where to look.


    2013 looks mighty exciting from where I am sitting.

    I can’t wait to get started!

    February 01, 2013 08:00 AM

    August 06, 2012

    Cloudant

    Search 2.0

    Background

    Before I joined Cloudant, I worked in what we now simply call “Enterprise”. Not, you understand, the one where you go into space and meet aliens (and maybe learn a little about the human condition) but the other one, where you go into meetings and meet managers (and learn a lot about the human condition). It was there that I first encountered Lucene.

    This was before the days of Solr, and way before the days of Elastic Search, a time when a Lucene index was an island, entire of itself. I was involved in building a proprietary clustered search platform and saw up close just how difficult that can be. Later, I was searching for some foundational technology to build a distributed archival system, having failed to create one of my own (a few times). Happily, this was around the dawn of “NoSQL”, as we no longer call it, and I found CouchDB. This database made so many good decisions that I was able to overlook that it was written in some arcane language I’d never heard of. I had already come to value the crash-only mentality that CouchDB enforces (tragically, by building a complicated system with elaborate and brittle shutdown code) as well as the append-only nature of disk writes and the all-in-one file format. The degree to which CouchDB’s API anticipates clustering was considerable too (albeit incomplete).

    I could use this to build what I needed. To cut a long and winding story short, I didn’t. I joined Cloudant instead, who already had.

    Before that, however, I had become an active CouchDB contributor and, a little later, a committer and community member. Further, I felt there was a gap that I was well-positioned to fill; the ability to index CouchDB databases with Lucene. This project, with the remarkably bland name, is, of course, couchdb-lucene.

    Search 1.0

    The first question to answer about Search 2.0 is “What was so wrong with Search 1.0?”. I could write a lot about this but much of it would be pedantry (not yet an Olympic sport, so my trophy shelf remains empty) but two important points remain. Firstly, the code base was large (relative to the feature set) and the original author had left. Secondly, and much more importantly, was the choice of on-disk format for the indexes.

    For those familiar with Lucene, it suffices to say that Search 1.0 does not use the familiar index format you know. For those that don’t, or wouldn’t object to a reminder, Lucene’s on disk format is something splendid. It’s highly optimized for full-text search access patterns where you necessarily have to examine every document you have looking for the top-scoring matches. Specifically, it’s an inverted index where the keys are words (“cat”, for example) followed by a contiguous, ordered list of document id’s that contain that word (and where they contain it). A separate index enables you to efficiently find the index for a given word. There are many more things in there than I’ve said, and more optimizations, but this is the core of it.

    Search 1.0 used the Lucene library for analysis but used clustered CouchDB views for the index. The idea, of course, being to reuse all of Cloudant’s considerable infrastructure around managing secondary indexes. It was anticipated that the less optimized data structure of the view would be compensated by the parallelism introduced from sharding.

    This worked adequately but never spectacularly, and we wanted to do better.

    What Search 2.0 Isn’t

    Search 1.0 is written in Java and uses Jinterface to enable interprocess communication between the JVM and our Erlang nodes running everything else. Java is a verbose language in general, and profligately so from the Erlang perspective. Mindful that my coworkers have no Java history, I investigated other JVM languages for a better fit.

    I didn’t choose Scala but I did choose Scalang (which happens to be written in Scala). Scalang provides a natural interaction with Erlang, notably because it follows the actor model for concurrency. Of course it was necessary to learn Scala in order to develop against Scalang and, indeed, to make some substantial contributions to the Scalang project (process monitoring being the main one).

    Search 2.0 isn’t based on Apache Solr or Elastic Search. Why not? Well, Cloudant already has production quality solutions to the clustering problems we face; sharding, fail-over, read repair, and the like. Search 1.0 was right to use that infrastructure.

    What Search 2.0 Is

    This seems a good time to note that I don’t call this “Search 2.0” at all. What we’re branding as Search 2.0 is, internally, two components, Dreyfus and Clouseau. Dreyfus is about 800 lines of Erlang and deals with plugging into existing Cloudant technologies like Chttpd, Fabric and Mem3. Clouseau is about 1000 lines of Scala which provides access to the Lucene library via Scalang. For (a not entirely fair) comparison, Search 1.0 was a little under 7000 lines of Java.

    Lucene is an excellent library and to use it well is to largely stay out of its way and not be stupid. Lucene 3.6, the latest at the time, is a much more refined release than when I started, back in the 1.4 days. Search 2.0 deliberately exposes only a part of the Lucene feature set so that we can make the core bullet-proof but there are quite a few exciting things I have my eye on, which will have to wait for a future blog post.

    Readers familiar with couchdb-lucene will know that it uses Rhino for its javascript processing. Search 2.0 does not, it uses the same SpiderMonkey as the rest of Cloudant. This ensures we evaluate Javascript the same way in views, update handlers, and index functions and leverages our tools for managing and monitoring couchjs processes. Those readers will also notice that the Search 2.0 API is not the same as couchdb-lucene’s. I took the opportunity to design a simpler API, I hope you like it (Some details here). There are some planned enhancements to that API that will be deployed when the features they enable are ready for production.

    Today, Search 2.0 provides a rich query language into your clustered indexes, including a minor tweak to the standard Lucene query syntax, namely that a range query where both ends look like numbers are transformed into a numerical range query. So queries for price ranges, or latitude/longitude ranges will work as you’d expect. Simon Metson kindly produced a nice demo of this here.

    In addition to rich searching is a feature I’ve called “Bookmarks” for reasons I hope become apparent. We’re all familiar with the notion of “pagination” if only because of the Gooooooogle links we never click. In Search 2.0, every search result returns a “bookmark” value in addition to the number of matches we found and the first “page” of matches. Passing this value back in will return results on the next page. There’s a caveat, though. It’s quite possible that, between two searches, a new document (or an update to an existing document, or removal of a document) will alter the scoring such that you will see results from an earlier page.

    We will be posting a full description of the API and all the features in the For Developers of our website.

    by Robert Newson at August 06, 2012 07:00 AM

    August 01, 2012

    Cloudant

    We're on a Mission to Make it Easy

    Cloudant isn’t like other databases. So it goes that the content we present to developers in our newly minted For Developers section isn’t like other tech documentation; i.e. it’s not static. More on that in a minute.

    First, a bit about the world we live in…

    Cloudant is a hosted, distributed database service. Apps write to and read from Cloudant using our RESTful API. The payload is lightweight JSON. For all of that, you may be thinking it’d take a lot time to get Cloudant set up in order to play with it, and kick the tires. Nope. All it takes is the five or so seconds to fill out the sign up form.

    And so…

    So long, static

    More than just docs, the For Developers section is a live demo of the data layer at work. Click a couple buttons to CRUD, get JSON via secondary indexes and search the sample database that you replicated into your account. It’s a breeze.

    secondary indexes screenshot

    Technical references (including handy curl commands) run inline with each of the demos.

    So long, friction

    As a newb noob N00B who is developing his first application on Cloudant, it is no small comfort to me to play with live demonstrations alongside the reference material in here. Not only are the docs in-step with the product (whew), but also it’s clear right away that the thing just works. And fast.

    Last night, I had the Olympics on in the background as I worked on my first Cloudant-driven app. When I picked up my head to look around, prime-time coverage was long over. Kayaking was on. It was 2am.

    The hours that disappeared on me weren’t spent banging my simple brain against complex SQL statements, and writing and rewriting blocks of server-side code. The time was spent playing around with the data coming from my Cloudant database, moving it around the interface I’m working on. Which is a long way of saying that time flies when you’re having fun.

    Getting started

    I went from setting up my database to playing with the interface in minutes. Here’s how I got there.

    Step 1 I put my data (JSON) into Cloudant with a bulk insert.

    Step 2 I built my search index inside my Cloudant design doc (the same place I’ll keep my secondary indexes).

    {
      "_id": "_design/people",
      "_rev": "13-24d8a5edfd54d61dacb0bc4230eee28a",
      "language": "javascript",
      "indexes": {
        "searchByName": {
          "index": "function(doc) {
            if(doc.docType=='person') {
              index('default', doc.last);index('default', doc.first);
            }
          }"
        }
      }
    }
    

    (My search index, searchByName, indexes the values of the first and last names of the people in my database so I can search for them by either of those or both in my application.)

    Step 3 A little JavaScript to deliver it to the interface.

    // Dependencies: JQuery and sag.js (http://www.saggingcouch.com/)
    $(function() {
      // since I want to work locally (but use Cloudant data), I've set up a reverse proxy
      // more on reverse proxy: http://wiki.apache.org/couchdb/Apache_As_a_Reverse_Proxy
      // use sag.js to connect
      var user_db = 'my-Cloudant-database-name';
      var couch = sag.server('my-server'); // localhost
      couch.setDatabase(user_db);
    
      //  capture the search form submission
      $('form.search').submit(function(e){
        //  kill default action
        e.preventDefault();
        // get the value of the search input
        var query = $('#last').val();
        //  get data via sag.js
        couch.get({
          //  query my Cloudant search index (see above) and account for wildcards (partial string searches)
          url: '/_design/people/_search/searchByName?include_docs=true&q=' + query + '*',
          callback: function(resp, succ) {
            // vars for iterating over the results
            var rows = '', i;
            //  create the rows for my table
            for(i in resp.body.rows) {
              rows += '<tr><td>';
              rows += '<a href="#" class="view-detail data-cloudant-id="' + resp.body.rows[i].id + '">';
              rows += resp.body.rows[i].doc.first + ' ' + resp.body.rows[i].doc.last;
              rows += '</a></td></tr>';
            }
            //  drop the rows into a table and append it to the results container in my interface
            $('#results').html('<table id="people" class="table table-striped"><tbody>' + rows + ' </tbody></table>');
          }
        });
      });
    });
    

    The launch of the For Developers section is a small but important step for us. A first step toward a full-featured library of useful content for people like me and, we hope, people like you. To that end, if you’re a developer using Cloudant and have ideas about docs or demos you’d like to see, please reach out. We’d love to hear from you.

    by Brad Noble at August 01, 2012 07:00 AM

    July 23, 2012

    Cloudant

    Big Science, Big Data

    Big Science, Big Data

    Simon Metson started working for Cloudant in March 2012 as an “ecology engineer” and became a full time employee in July. He is based in the UK.

    Before I joined Cloudant, I worked on the Compact Muon Solenoid (CMS), one of the two Large Hadron Collider (LHC) detectors that announced their discovery of the Higgs Boson earlier this month. I was a member of this collaboration for 10 years, from the start of my PhD through a bunch of post doc positions (all based in the UK at Bristol Uni), so was really excited and pleased to see the result being announced. It is the result of an amazing endeavor, where everything (hardware, software, wetware) has been pushed beyond known limits and, more importantly, delivered.

    Candidate Higgs event - copyright CERN/CMS

    Candidate Higgs event - copyright CERN/CMS See the original

    For most of my time with CMS, I was involved in one way or another with data and workflow management (DMWM); designing and delivering software that would be able to process, catalog and transfer the tens of petabytes of data that the detector would deliver each year, and for the last four and a half years I ran the DMWM project with a colleague from Fermi Lab.

    CMS has a tiered, distributed computing system. Initial processing happens in the Tier 0 (at CERN). The resulting data is shipped to 7 national labs (Tier 1’) located in US, EU and Asia for second pass processing and offsite tape backup. Once on tape the data is made available to the collaboration; any CMS collaborator can request the transfer of any dataset to their local Tier 2 cluster. Last time I check we’d moved over 100PB of data around and had about 50PB of data active in the system.

    CMS 2011 transfer link map - copyright Paul Rossman

    CMS 2011 transfer link map - copyright Paul Rossman See the original

    The DMWM project writes the software that manages these transfers, records and manages the metadata for all the datasets and submits and tracks all the processing jobs through the various batch and grid systems for both organized central processing and more ad hoc user analysis.

    Getting the job done

    Watching how the computing system of CMS evolved over the last ten years from something that required a lot of hand holding to something that could be relied upon to deliver, often well beyond specification, was a great experience; seeing the sites, organization, software and people mature has had a profound effect on how I think about the design, implementation and management of software projects.

    The nature of these experiments is such that any and all tools are available for use. We have a complex problem to solve, which results in a complex solution. While we rationalized a lot of things (making sensible decisions can save you thousands of lines of code) we still ended up with quite a range of supported technologies. Our stack used (from memory and in no particular order):

    • Languages: Python, Perl, C++ and a bit of Java
    • Databases: Oracle, MySQL, CouchDB, MongoDB
    • Messaging: ZeroMQ, MonALISA
    • Grid middleware stacks: EGEE, LCG, OSG, and ARC
    • Batch schedulers: PBS, LSF, Torque, Maui, SGE, Condor
    • Distributed file systems: DPM, HDFS, CASTOR, dCache, Lustre, GPFS).

    Wrangling that stack sure was entertaining…

    I think CMS was the first experiment to start using NoSQL tools in production; HDFS is used at some of the T2 sites for their storage system, MongoDB is used as a smart cache aggregating data from various data services and CouchDB is used for various state machine and monitoring tasks. CouchDB’s replication feature was especially useful in CMS’s distributed computing system. Being able to quickly build aggregated views of monitoring or state information was great, as was having everything served via a REST HTTP API. Having it all in one application was even better!

    From Higgs to Cloudant

    I met Mike Miller at CERN in 2007 (well, in a fondue restaurant, I think), had followed Cloudant from its inception and we’d kept in touch after he left CMS to go look at neutrinos and dark matter at University of Washington, build a company and a set of desks. He contacted me at the end of 2011 to see if I’d be interested in joining Cloudant.

    Like a number of my former colleagues, I felt that it was time to move on. As Mike once said to me “we’re builders”, and I wanted to be building new things again. I’d experienced working in a large (CMS has about 4000 people involved, and is spread over nearly 120 institutions), collaborative environment with all the benefits and issues that they bring and decided that seeing what life is like in a startup would be interesting.

    Am I sad that I left CMS before the discovery announcement? Not really. The nature of these collaborations is that everyone who makes a contribution is recognized; the papers are signed by “the CMS collaboration” and I am a member for the next two years. While I wasn’t directly involved in the analyses that were presented on the 4th, the software that I was responsible for enabled that research. That’s something I’m quietly proud of.

    What’s great about joining Cloudant now is that I get to see that whole evolution and scale up cycle again. Everyone in the team knows that we’re just at the beginning of a very interesting journey and is really excited to see where the company goes over the coming years. I’m really looking forward to helping the company scale up; the software, the organization and, importantly, the customers.

    by Simon Metson at July 23, 2012 07:00 AM

    July 19, 2012

    Cloudant

    Cloudant Helps Light Up the 2012 London Olympics

    As much as we’d like to say Cloudant athletes will be lighting up the 2012 London Olympic track, playing fields, or pool…instead, Cloudant technology will be used to help light up…the Ferris wheel.

    Tonight at 9PM local time in London, Energy of the Nation, the world’s first social-media-driven light show will launch at the EDF Energy London Eye. The nightly light show, and accompanying realtime Energy of the Nation infographic and “Energy2012” mobile app will measure the mood of the UK during the Olympics and Paralympics by performing sentiment analysis on Olympics-related tweets sourced from the UK’s 10 million Twitter users.

    London Eye Olympic Light show (source: Sosolimited.com)

    The campaign was conceived and developed by the creative geniuses down the street from us in Boston, Sosolimited in conjunction with UK-based brand experience consultancy, Ignite. The project was sponsored by EDF Energy, which is a sponsor of both the London Eye and the Olympics.

    How Energy of the Nation Works

    Sosolimited has written software that captures Olympics- and Paralympics-related tweets in the UK. It has been capturing tweets since May 1st of this year. The tweets, and a vast library of emotional words, punctuation and emoticons, are stored using the Cloudant Data Layer service (hosted on SoftLayer cloud infrastructure in Amsterdam).

    The Sosolimited software runs sentiment analysis algorithms on the Tweets in Cloudant to determine their emotional content. The analysis output reflects the rising and falling mood of the nation with respect to the games and is charted throughout the day on a large LED wall next to the London Eye. Every night at 9pm, that data is to be played back in a spectacular, 20-minute, low-carbon light show featuring full-color architectural lighting fixtures around the Eye and large, ground-based search beams, all powered by EDF Energy. There’s more good info on how this all works located on the EDF Energy website.

    We’re psyched for the Olympic games to begin, and even more psyched that Sosolimited chose Cloudant to play a role in the event. If you haven’t seen Sosolimited’s catalog of interactive and multisensory design work, check it out!

    by Andy Ellicott at July 19, 2012 07:00 AM

    July 17, 2012

    Cloudant

    Managing applications on Cloudant

    A fairly common question we get asked is “how do I manage my view code in insert your favorite version control system here?”. As people develop more complicated apps this question becomes increasingly important. Fortunately there are a set of tools that make transferring code from a version controlled directory into your database easy.

    At the heart of Cloudant is Apache CouchDB. Applications that run out of CouchDB are known as “CouchApps”. At their most basic these applications are nothing more than a design document containing a view. There are a raft of tools that simplify the development of these applications that are served directly out of CouchDB. They each target slightly different use cases, from a minimal wrapper to upload code into the database to a fully featured development platform.

    At the most basic level these tools enable developers to version control their views and other code, and “compile” and push them into the server. More advanced use enables development of “two-tier” applications, pulling in various libraries to build complex, dynamic applications; even ones available offline.

    The tools

    Below is a quick summary of the various tools that let you build and push code from your local system into the database. It’s by no means exhaustive, and I’d love any corrections/clarifications/additions.

    • CouchApp.py The original and reference implementation for CouchApp clients this python application is no longer under active development (Benoit has moved on to Erica) but it is still widely used and recieves occasional updates. It turns the structure of a design document into a filesystem layout, mapping the various functions of your application to files in a defined location on your disk.
    • Erica Erica is a CouchApp client built in erlang, effectively the next version of CouchApp.py. It has basically the same feature set as CouchApp.py.
    • Soca Soca takes a different approach to CouchApp.py/Erica, allowing any directory structure for the app and compiles the application into a single json file following a layout defined in a file, rather than the filesystem itself. It builds applications around Sammy.js by default.
    • node.couchapp.js node.couchapp.js is a node application for building CouchApps. It requires all the functions (views, lists etc) of the application be defined in a single json file, with attachments (images, html, external scripts) loaded from a defined location.
    • Reupholster Reupholster is a Java application that builds CouchApps in the same style as node.couchapp.js. It provides an end-to-end development tool, managing creation of the application, automated testing and pushing to the server.
    • Kanso Kanso is another node application, described as “the NPM for CouchApps”. It has a very comprehensive set of packages that enable easy management of various third party libraries and frameworks.
    • Situp situp.py is modeled after the python setup.py install script. It is a single self contained file with no dependencies. It’s the tool I use for my own applications that I develop on or for Cloudant, but it’s also written by me so you’d hope I would use it! It aims to support development of more complex CouchApps, and can pull in Kanso packages.
    • Mouch Mouch takes the metaphor of a build file a step further, resulting in a minimal application “for people who love purism. For people who keep complete control over their bytes. For people who love things speeding fast, keeping dependencies as minimal as possible.”

    Summary

    CouchDB has supported running applications out of the database since 2009, and the associated tools are in a reasonable state of maturity. It’s nice to see other databases getting similar functionality now; it can only be a benefit to developers creating two-tier, dynamic applications.

    For very simple tasks (e.g. pushing a view into the database) there isn’t much to differentiate between the various tools; it’s a very subjective choice. As you go “further down the rabbit hole” you may find one tool fits your application or workflow better than another.

    by Simon Metson at July 17, 2012 07:00 AM

    June 30, 2012

    Cloudant

    A Good Week for PHP and Cloudant

    Little known fact: about 20% of Cloudant Data Layer users are PHP developers. So, last week we were excited to announce that Cloudant is now available as an add-on at cloudControl PHP hosting platform, a cloud application platform for PHP developers (with Ruby and Python support coming soon).

    The announcement follows several weeks of integration work and testing on both sides of the partnership, and we hope PHP developers can develop faster and scale their apps and databases easier as a result of the effort.

    cloudControl account holders can add Cloudant to their service via the Cloudant Add-On page at cloudControl: https://www.cloudcontrol.com/add-ons/cloudant

    Cloudant featured in a book on CouchDB & PHP Development Coincidentally, we also got a note from author Tim Juravich (@timjuravich) last week letting us know he had just published a book on developing PHP web applications on CouchDB. In the book, he walks developers through the creation and deployment of a simple social networking app using tools like PHP, CouchDB, Sag, Bootstrap, jQuery, PHPFog, and Cloudant! You can get the book here.

    by Andy Ellicott at June 30, 2012 07:00 AM

    June 26, 2012

    Cloudant

    Lean and Technical-- How we chose and furnished an office the YCombinator way.

    Cloudant’s growing, and we recently opened offices in Seattle and the UK (in addition to our headquarters in Boston, near Cloudant’s MIT birthplace). As a YCombinator company, Cloudant is conditioned to run lean, but we also have an exceptionally technical team with an appreciation for good design. This made finding and furnishing our new office particularly challenging. Here’s the story of how we did it in Seattle, aka “Cloudant West”.

    Location.

    In his seminal 2005 article “How to Start a Startup”, Paul Graham writes, “for a startup location is very important.” We couldn’t agree more, and it’s why we surveyed five different neighborhoods in Seattle: Downtown, Pioneer Square, Capital Hill, Fremont/Ballard, and South Lake Union. Our criteria were loosely based on:

    • Environment. The energy of the neighborhood, what other companies (large and small) are nearby, and where do we like to spend time?
    • Food. Access to a diverse selection of good, cheap food is critical to a startup. Lunch is often where the best ideas are born, let’s make sure we step away from our laptops long enough to think and talk.
    • Space. Where can we find access to high ceilinged lofts with plenty of light and just enough partitioning into rooms?
    • Cost. Real estate costs vary significantly.
    • Connectivity. We wanted to be at most one switch from a fiber drop.
    • Commute. Seattle doesn’t have a subway and neighborhoods are separated primarily by water. A central location is key.

    Environment concerns quickly narrowed down the list. Downtown is rather soulless during and after business hours. Fremont/Ballard and Capital Hill are amazing places to live and hang out, but they don’t have the necessary concentration of other companies. I wanted to love South Lake Union, primarily because it’s close to the lake and Amazon and biotech are absolutely transforming the neighborhood. Everything is shiny, new, and most intersections have their own cop to direct traffic (AMZ makes things happen). Unfortunately, aside from Techstars it doesn’t have many companies in our space. The kicker was the small selection of food and the surprisingly high costs. That left us with Pioneer Square, which luckily maps well onto space, cost, connectivity and commute. There are a multitude of lofts with just enough brick, priced well, and close to the all-important fiber. To boot, we have several customers around the corner, Future Advisor (YC10) across the street, easy access from three directions, and are right next to Century Link and Safeco fields.

    Next step – find a good space.

    Space.

    Ben Anderson took the lead at scouting locations. I should say that commercial real estate is due for disruption. The fact that agents still control the listings and cost is absurd and creates an inefficient market. That said, we followed advice from several advisors and were introduced to a trusted agent that put plenty of good properties in front of us. We found it challenging to envision how we would lay out an office, so we called in the experts – my fiancee LJ. I delivered a blueprint to LJ and before I knew it she gave us a Google Sketchup model for layout and a video to send back east for the final sign-off. Here’s the video from the winning space; thanks LJ.

    We decided that we could nicely lay this space out for 10 people without feeling crowded. We could get desks (more on that below) that would be easily re-arrangeable, and we could have space for sales and bizdev, meetings, open discussions, and door-closed, heads-down coding.

    Furnishings.

    This was hard. Our office in Boston came furnished. Since this time around we had a blank slate and some time to think, we decided to be creative. Ikea desks are actually OK, I've had many in my time. But, we wanted something just a little different, with superior function, fit/finish, and an imprint of the Cloudant brand: technical, well made, and aesthetically pleasing. We decided to build instead of buy, and to improve on Amazon's famous door desk design. Blueprints, specifications and instructions are available upon request. I'd like to say that I had a big hand in this, but really LJ led the charge (startups are a family affair). For the first prototype she selected a beautiful recycled door that she bought for $20. It required stripping and staining. Here's a look at the prototype in our living room in March. This door had recessed panels. We investigated simply covering the door with a sheet of glass but found that it was expensive, prone to finger prints, and sharp on the edges. Instead she filled the recessed portions with plywood. The inserts are held from below by screws and then the entire surface (door + insert) sanded to a uniform height and finish. Using multiple stains gave us a cool two-tone finish and then the entire door was coated with multiple layers of varnish.

    This prototype turned out to be beautiful but labor intensive, so we instead chose to purchase solid core doors from Home Depot (~$40 each) and proceed to sand, stain, and varnish them all. We (meaning LJ) used the lumber purchased for the desk frame and legs to ingeniously stack doors so that we could varnish them in our tiny garage. It's critical to varnish the doors in a level horizontal position and in a particulate free environment. We recommend two coats of stain minimum and three coats of varnish for a uniform and smooth finish.

    The support frame to turn the door into a desk was also a challenge. Several configurations were tried and we settled on running a single 2x4 on edge underneath the door running the length for support. That board is screwed into the underside of the door using small right angle metal tabs and provides a mount point for legs on each end of the door. The legs are cut and assembled from 2x4 pieces as well. The bonus is that all of the support structure can be cut, drilled, and assembled ahead of time. Assembly consists of screwing the support 2x4 into the underside of the door, connecting the legs to the support, and securing the legs to the underside of the door with right angle brackets.

    We had quite a bit of fun making these desks. We got help from a longtime friend and advisor of Cloudant Ross Rosen and YCombinator's own Tony Wright (seen pulling quite a face below left). Tony, always the designer, augmented LJ's initial tapered leg design by rounding the corners (trying his best to make the table look like a big beveled "call to action" button). We ended up with a stack of gorgeously rounded legs, but they were a bit tricky to hold together, drill, and screw together without someone holding them in place with inventive yoga moves.

    These desks are huge; you can comfortably fit 4 laptops if you use both sides, but we give one to each employee. It’s highly conducive to teamwork, as it’s easy to pull up a chair and work with somebody to solve a problem, review a design or test an interface. The rest of the office we furnished primarily from Costco business, and we’ve been particularly happy with the ProGrid chairs (highly adjustable, robust, comfortable) for under a third of their list value.

    It took some work, but Cloudant West is a space that speaks to the ideals of YCombinator and Cloudant – build great things that people really like. I love our new colony in Seattle, and I really love my homemade desk.

    by Mike Miller at June 26, 2012 07:00 AM

    June 16, 2012

    Volker Mische

    OSGeo Code Sprint Bolsena 2012

    The OSGeo Code Sprint in Bolsena/Italy was great. Many interesting people sitting the whole day in front of their laptops surrounded by a beautiful scenery. This year I spent most of my time on GeoCouch, CouchDB, Couchbase, deegree, Imposm and GeoNetwork.

    Already on the first hacking day we had a result, a Couchbase backend for the deegree blob storage. This means that you can now store your rich features (e.g. from INSPIRE) in Couchbase and serve them up as a WFS with deegree. In case you wonder what rich features are, it's the new, shiny and more descriptive name for complex features.

    In the following days I worked together with Oliver Tonnhofer on a CouchDB/GeoCouch backend for Imposm. You are now able to store your OpenStreetMap data in CouchDB and make queries on it through GeoCouch. I've created a small demo that displays the some data import from Andorra with directly with MapQuery, without the need of any additional server/service. The CouchDB backend should be easily adaptable to Couchbase, if you want to help, let me know.

    I've then spent some time on the GeoNetwork project and helped translating the language file to German. I cleaned it up a bit and fixed major wrong translations. It's not perfect yet, as I've only spent little time on it, but at least it should be way more understandable (and sadly less funny) than the original version which was generated by Google Translate.

    When it was time for presentations, I give a quick overview over the Couch* ecosystem. From CouchDB to GeoCouch, BigCouch, PouchDB, TouchDB (TouchDB-iOS, TouchDB-Android), Couchbase Syncpoint and Couchbase. You can find the slides as PDF here.

    On the last day I've spent my time on polishing GeoCouch a bit and getting it ready for the Couchhack in Vienna. I've backported all changes from Couchbase to the CouchDB 1.2.x branch and also ported the geometry search into an experimental branch. You can now search your GeoCouch with any Geometry GEOS supports.

    The event was fun as always and I also get to know some new people (hello B3Partners guys). Thanks Jeroen from GeoCat for organizing it, and thanks all other hackers that made it such a awesome event. Hope to see you all next year!

    by Volker Mische at June 16, 2012 08:27 PM

    June 11, 2012

    Cloudant

    Cloudant Data Layer Now Available on Windows Azure

    We are very excited to announce that the Cloudant Data Layer now runs on Azure. As a result, .NET, Node.js, PHP, Java, and Python developers can now enhance their fast-growing Azure apps with a scalable data layer that is co-located with their application servers in Azure data centers.

    It’s great to see Windows Azure strengthen support for open source development technologies and hybrid clouds. The addition of Linux-based virtual machines made it possible for us to offer the Cloudant Data Layer service on Azure

    We’re psyched to be active members of the Azure community now…the response to our announcement last week has been exciting. For those of you who are new to Cloudant, we offer a NOSQL database service (with an Apache CouchDB-compatible API.) A couple of things set Cloudant apart from other SQL and NOSQL database options available to Azure developers:

    • Cloudant is a service: we provision your data layer and then keep it running non-stop and scale it as you grow. Unlike scale-it-yourself DBs that you have to install and manage in the cloud, Cloudant’s back-end DB operations (provisioning, backup, scaling, re-partitioning, etc.) are taken care of for you by us, the experts.

    • Global data distribution: we push your data to the edge–to Azure data centers around the world–so it’s closer to users for low-latency data access.

    • Data Replication & Sync: enable offline data access & collection (especially to mobile devices) or use it to replicate data from the cloud to on-premise data centers, so you are never locked into Cloudant or Azure.

    Cloudant on Azure is still in “preview” mode as we continue to tune it for performance and expand it to additional Azure data centers in North America, Europe, and Asia. Please let us know how it is working for you and what we can do to improve it.

    To get a Cloudant Data Layer account running on Azure, please visit our signup page and choose “Lagoon” as your data layer location.

    If you already have a Cloudant account and you want to move it to Azure, just follow these directions:

    • Log into Cloudant
    • Choose the ‘Account’ tab
    • Change the placement of your cluster to ‘LAGOON’
    • Cloudant will notify you after your account has been relocated to Azure

    by Sam Bisbee at June 11, 2012 07:00 AM

    April 11, 2012

    Cloudant

    Cloudant Data Layer on Joyent Cloud

    We just added Joyent Cloud to the list of Cloudant Data Layer locations.

    Joyent Cloud is a high-performance environment for running apps and data layer services in the cloud. Backed by Joyent SmartOS - an operating system optimized for multi-tenant cloud environments - and the ZFS file system, Joyent Cloud enables Cloudant to take advantage of specific caching and data backup features in the ZFS file system and deliver outstanding data layer performance and reliability Aside from Cloudant, Joyent Cloud also powers a growing number of cloud platform and software services providers including Nodejitsu, StackMob, and GameSalad.

    Learn more about Joyent Cloud

    Create a Free Cloudant Data Layer running on Joyent Cloud

    Getting started is easy. Just create a free Cloudant account, and select the Joyent Cloud option “Paloma” from the list of locations, as shown below.

    Or Move Your Data Layer to Joyent Cloud

    If you already have a Cloudant account and you want to move it to Joyent Cloud, follow these instructions:

    1. Log into Cloudant
    2. Choose the ‘Account’ tab
    3. Change the placement of your cluster to ‘Paloma - Joyent, US Southwest’
    4. Cloudant will notify you after your account has been relocated to Joyent Cloud.

    by Alan Hoffman at April 11, 2012 07:00 AM

    IrisCouch

    Announcing paid cloud CouchDB hosting

    April 12 is Iris Couch’s one year birthday. Today, we offer paid cloud CouchDB hosting plans.

    How it Works

    Our pricing plan has the details. It is relaxing and RESTful. Your log file is your receipt.

    This month is completely free for everybody. May is the first month under the new policy.

    • People who don’t accrue $5/month of usage don't owe anything for that month
    • The vast majority of users don’t accrue $5/month in usage charges.
    • Thus, by the transitive property of service pricing, most of you won’t pay anything
    • Your account shows your historical usage since you signed up

    Below, co-founders Jeff and Jason reflect on our milestone.

    Jeff’s thoughts

    I’m really proud of how dialed-in this service has become with Jason’s expertise. I’m excited about this launch and new features we have in development. Mostly, though, I’m really looking forward to not feeling so sad every month signing huge checks to pay our hosting bills.

    We spent a lot of time mulling over how to price this so that beginners and tinkerers can still have a free sandbox, and I think what we’ve come up will protect that. Thus Iris Couch will remain both a platform and a tool to spread the word about how relaxing CouchDB is.

    Jason’s thoughts

    I am pleased with our first year. We have built technology I am proud of. Many people (Jeff in particular) have asked:

    Why the delay for paid stuff? Aren't you hungry? Are you lazy?

    We are both hungry and diligent. But despite being an Apache CouchDB developer, I am a sysadmin first. Before I take your money to steward your data, I have an ethical duty to produce a proven, reliable service.

    • We proved our mettle, in production, for two years (CouchOne plus Iris Couch)
    • We have had zero data loss incidents.
    • We have zero single points of failure.
    • We have a confirmed, tested backup process
    • We have a confirmed, tested off-site disaster recovery process

    Is it perfect? No, it is better than “perfect.” It is very good. We built something befitting a proper database company.

    by Jeff (jeff@iriscouch.com) at April 11, 2012 05:00 AM

    April 09, 2012

    Cloudant

    Cloudant CouchDB Summit Drinkup

    Cloudant is hosting the CouchDB Summit this week in Boston and we will be capping off the two-day affair with a public drinkup on Tuesday night. We invite everyone to come visit with the comitters and join us in wrapping up the summit. Just look for the folks in good lookin’ Cloudant swag. Details:

    • Tuesday April 10th
    • 6 - 9pm
    • Lucky’s Lounge
    • 355 Congress Street, Boston, MA.
    • All Beers on Cloudant’s Tab

    by Sam Bisbee at April 09, 2012 07:00 AM

    April 07, 2012

    IrisCouch

    Upgrade to Apache CouchDB 1.2.0

    Just now, the Apache CouchDB project released version 1.2.0.

    I am happy to announce that all Iris Couch accounts are now running this release. You needn’t do anything, the upgrade is automatic. Just relax on the Couch!

    Notable improvements:

    • Persistent cookies—users remain logged in through a browser restart
    • More secure _users database
    • Better, faster replication
    • Reduced (and faster) disk usage

    We also upgraded to the new Ocasta Labs Facebook Authentication plugin. It is now easy and relaxing to build a Couch app letting users log in via Facebook.

    Please email us if you have questions or issues, or visit our support forum. Thanks!

    by Jason (jhs@iriscouch.com) at April 07, 2012 12:49 AM

    March 28, 2012

    Cloudant

    The anti-sprint

    Every Monday morning the Cloudant engineering team jumps into a google hangout for our weekly “train” standup 1 where we swap stories from the weekend, I defend my hometown teams (MSU and Detroit), and decide what code to include in the next release – tomorrow. Yep, that’s right, we push new code every Tuesday (at the slowest 2.) This may not seem astounding to some, but we power applications for nearly 10,000 users and have an impressive amount of transactional data under management, spread globally across 11 data centers.

    In my past life I once labored for literally more than a year to get a new release cut, tested and pushed. In that light, I find our weekly/daily release rhythm pretty amazing. More importantly, I think it’s an example of Jay Krep’s compelling argument that, when it comes to distributed systems, the true value is not solely, or even primarly, in the underlying software, but in the service provided by those that run it. That ethos lies at the very heart of all things Cloudant. It drives our technology choices, product vision, roadmap, release schedule, operations, support, and (perhaps most importantly) business model. We write software to scale your data and we run it. Below I briefly expand on the tools we use to constantly push new code, and why that is important to our customers.

    Tools

    • Erlang. Building our core stack as a collection of Erlang OTP applications and using Chef for configuration management means that we upgrade our running database applications without ever taking them offline or even dropping active connections. If you haven’t ever witnessed the magic of hot code upgrades in Erlang, you haven’t lived. New and old code live side by side, processes are suspended, a new code path kicks in, state transformations are made on the fly, and voila, you’re running new code without ever skipping a beat. Don’t like it? Hit the roll-back button.

    • Github + CI + Tests. Hot code upgrades would be worthless if we didn’t have good code to push. Github’s tools have really matured over the last few years and it empowers a very efficient workflow (pull requests, code review, etc) for a distributed team like ours. Coupled with Jenkins for builds and automated testing (etap, etc) we can efficiently go from idea to tested implementation.

    Value

    • Cloud. This is the promise of Cloud – Isolating non-core elements of your technical stack and finding the best software and people to deliver them faster, cheaper, and more reliably you can do yourself. Making a quantitative ROI argument has always been challenging because it misses a key point, “it’s not just the software, but who runs it.”

    • Response. On stage in 2009, one of our first customers (who was piping the twitter firehose into Cloudant) referred to our software as young and developing. Ouch. But he followed up by calling us some of the best problem solvers he had encountered. The distributed systems used to handle Big Data problems are relatively new. Our team has decades of experience in big science and enterprise IT, building and operating some of the world’s biggest computing systems, and still we are able to find novel failure modes. Our ability to handle these failures gracefully stems from our deep working knowledge of the code and it’s operational characteristics.

    • Priorities. There’s little worse than investing substantial time and money into a feature/product that is irrelevant in the market. Thus, while we have a clear product roadmap and development plan, we take the just-in-time ethos to heart. “Somebody is trying to create 100 million databases (true story). We should probably re-write that LRU module.”

    Culture

    • Team. Perhaps most importantly, this perma-release / anti-sprint schedule suits us. We are lucky to have an expert team where developers push code, trouble shoot, and close tickets. Nobody on our team is above jumping into #cloudant on IRC and working through questions/issues with customers. If Kreps is right (hint: he is) and the only real meaningful metric is “continuous hours of successful production operations,” then any distributed data system is only as strong as the people that run it, that is, the people that ensure continuous operations. In Cloudant’s case, the builders (kocolosk: rnewson: bitdiddle: davisp: banjiewen: and others) are also the operators. We’ve been running these systems in production for our customers for more than three years.

    Since we began working on Cloudant we have seen some major public failures of distributed big data systems. In nearly all those cases, the failure was not one of software but of operations. This seems like a natural consequence of putting up a wall between those that build code and those that run it. At Cloudant, that wall doesn’t exist, and this is what allows our customers to grow into their data layer, not out of it.

    by Mike Miller at March 28, 2012 07:00 AM