Cloudant’s NYC meet-up will take place at our office this Wednesday (tomorrow)! Please RSVP using the following link:
http://cloudantimagineeasy.eventbrite.com/
The office’ address is 12 West 31 St:
If you can’t make the meet-up, make sure to check out the webinar on Thursday:
Slides from a talk I gave at the CouchDB conference in Berlin last month. As always, the slides are on speakerdeck:
https://speakerdeck.com/till/easybib-and-cloudant
— Till
We’re excited to announce our latest product—a Premium Server option for Iris Couch users. Premium Server gives serious users an option to increase the stability and response time of their couches. It costs $75/month (in addition to the regular, metered fees), and buys you oh-so-much-more peace of mind... and performance.
A Premium server is more responsive and reliable than a standard CouchDB server. Premium servers run on isolated, dedicated clusters.
Interested? Log in and enable Premium Server in our billing page.
Once you have a credit card on file, you will see a Premium Server option. If for any reason you’re dissatisfied with the service, you can cancel at any time.
What, exactly, is a Premium server?
Premium servers run permanently in specialized clusters, with dedicated CPU and memory. Premium servers run on a common physical server, with dedicated CPU, memory, and i/o resources. Nothing is “oversold.”
Premium servers never pause when idle. Typically, if a couch goes unused for too long, we pause it, freeing resources to share with others. Unfortunately, infrequent usage suffers high latency. Premium servers do not have this issue.
Premium servers are monitored. Our automated tools monitor everything. But people watch every premium server 24/7. If we see any server issues, we respond immediately.
Premium servers are isolated. Each cluster runs in a dedicated DNS zone and uses dedicated HTTP infrastructure. Issues in one cluster do not affect another.
Premium servers work. Thanks to many of our users for months of beta testing. Premium service already supports many couches, such as the Node.js npm registry, a mission-critical CouchDB service, processing over 100 million monthly requests.
Slides to go with the headings.
Hi, welcome to CouchDB Conf.
CouchDB means a lot to me and it makes me very happy that you are all here today.
My name is Jan Lehnardt and I am the elected project management committee chair of the Apache CouchDB project. If you don’t know what this means, don’t worry, it means I sign up to deal with politics and paperwork voluntarily.
I have been with CouchDB since 2006. My first contribution was the initial UNIX build system to CouchDB. I stuck around ever since.
My main job back then was PHP/MySQL consulting and Open Source work as my own 20% project. In 2007, I started giving talks about CouchDB at user groups and conferences and quickly became addicted. People called me the CouchDB Chief Evangelist, although I don’t really like the title. From 2008 I started getting contracting jobs specifically for CouchDB and things blew up quickly that it became clear soon, that an organisation of some sort was needed to handle this. I eventually joined forces with CouchDB’s original inventor and J Chris Anderson to form the company Couch.io. We initially weren’t keen on taking venture capital, but when the whole NoSQL thing blew up, it was very hard to say no and on January 1st 2010 we were in business.
Couch.io turned into CouchOne and was eventually merged with Membase into Couchbase. Unfortunately, at this point, there isn’t much Couch-y about Couchbase aside from some underlying technology, so the name can be confusing.
I parted ways with Couchbase a week ago in order to concentrate and focus more on Apache CouchDB and related technologies.
In late 2011 two major supporters of CouchDB withdrew their support: Canonical’s UbuntuOne service, on paper a poster-child use-case for CouchDB and Saucelab’s Selenium platform publicly made clear that they were moving to other technology. It wasn’t a big deal if you looked closely, the one project didn’t have enough resources to integrate with CouchDB the right way and the other diverged so far from the original system design, that CouchDB was no longer a good fit. We were and we remain good friends with the respective developers, so we didn’t feel like pointing that out specifically. The result though was that people who didn’t look closely saw that CouchDB had popular failures where it hadn’t. This wasn’t all bad, but set the stage for another incident a few weeks later.
CouchDB’s original inventor publicly announced that he’d move on from Apache CouchDB to Couchbase fully. That new technology would only go into Couchbase. If you are in the know, this wasn’t a big deal again, he hadn’t contributed to CouchDB in two years, Couchbase was a different thing and he wanted to rally support for that. The casual reader though took away one lesson: “CouchDB is dead”.
With the added confusion of Couchbase’s name, and continuing their CouchConf conferences, things looks dire.
With this in the wild, the active CouchDB committers got together and put out an excellent 1.2.0 release that was accompanied by a new, modern website design. We thought instead of writing counter-statements and start unnecessary he-said-she-said discussions, we’d rather concentrate on shipping good software. 1.2.0 was well received and the whole thing lead the team to come together as a whole in Boston and two weeks later in a smaller group in Dublin in April 2012.
We came up with a long list of of features that we wanted to get into CouchDB. Some work happened over the summer, but I personally took a bit of a break to avoid burn-out from CouchDB. In fall, things started moving faster again when more community members started pushing things along. New contributors came and we finally came up with a way to go forward.
We decided to not be insane. The definition of insanity is trying the same thing over and over again and expecting different results. While it wasn’t all bad in CouchDB land we could do better and to do better we had to change a few things.
We determined that CouchDB needs a fresh mission statement. I signed up to come up with a few drafts, but nothing happened yet, so this one is still out there cough.
We decided to release more often. With the upcoming 1.3.0 release we will switch to a quarterly release cycle. A new minor version will come out approximately every 90 days, even if it is just a few bug fixes that a small number of people direly need.
CouchDB 1.3.0 will ship with top-notch API documentation right inside of Futon, the built in admin interface.
We will start to set up teams where anyone can sign up to work on a specific thing: docs, our JavaScript components, the Erlang core, the admin interface (built in HTML+JS). We want to make sure anyone who has the right skills can contribute in their area of expertise and not be scared away by the Erlang.
We put some emphasis on QA, automated testing, building binaries and packages for releases as well as in-development versions, so people can try new features before they come out more easily and as a result report issues early.
A few of the core team will put some focus on encouraging contributions rather than coding themselves. We hope to get more people on board with CouchDB development.
We also started doing roughly quarterly in-person meetings (at least as smaller groups) and weekly status meetings on IRC to give everything a little more focus and transparency.
The next few slides show a long list a few features that we’ll be working on. If you think anything is missing, let us know!
The more prominent things are: Merging rcouch, BigCouch, the new Futon, the plugin system and v8/node integration.
rcouch is a cutting edge distribution of CouchDB with many experimental features. A lot of that work is useful for Apache CouchDB and we plan to fold those back into the project.
BigCouch is a industry-strength clustering solution for CouchDB to allow seamless scaling and storing of lots of data. BigCouch is going to be merged into mainline CouchDB as well.
The admin interface Futon is getting a complete refresh with modern web development technologies.
There are many features hidden in some developer’s git repository because, while useful, it wasn’t generally applicable to CouchDB. With the plugin system, we hope that we can fuel a repository of plugins to CouchDB that are easily installed (think browser extensions, right from Futon). For example, you could install GeoCouch with the click of a button.
Finally, we are currently discussing the move from SpiderMonkey and to V8 or even Node. The jury is still out on which one it is going to be, but we hope that this is going to open up a lot more opportunities for CouchDB in terms of features and new contributors.
CouchDB would be nothing without you, the community. People who know little about CouchDB might like it or not, but people who really get to know CouchDB are extremely passionate about it.
CouchDB’s core idea, the promise that data can be anywhere it is needed, to make applications easier, more approachable and effective for people in modern computing environments, that idea is a big one. It’s the one that brings this community together.
Thanks again for coming, you are all awesome. CouchDB means a lot to me and I get a little teary seeing you all here.
If you like any of the above todos, or have some of your own, please help out, get in touch about how.
The CouchDB ecosystem is full of exciting projects that support CouchDB in one way or another.
PouchDB is the newest project on the scene. PouchDB is a pure JavaScript implementation of CouchDB targeted at the browser.
Let me say that again: CouchDB in JavaScript, for the browser.
PouchDB is the oldest idea, but it took the longest to realise because browsers weren’t ready for it. Now is the time though that we can build and ship a CouchDB compatible data store and replication engine that lives entirely in your browser. If you want to build web apps that can seamlessly synchronise data between a laptop, a phone and a server and not treat one as a “master” and the other as “slaves”, PouchDB is your solution.
TouchDB-iOS and -Android are Objective-C and Java implementations of the same idea, but for mobile phone platforms. They natively integrate into your iOS and Android applications and allow synchronisation between themselves, multiple devices, CouchDB’s on the server and even PouchDB in a browser. Again, a milestone in fulfilling the CouchDB vision: data where it is needed, not on some server, far away over a slow and unreliable or even unavailable connection.
This might not be too obvious to people who don’t do Node, but CouchDB is big, if not essential here, too. As any programming language & platform, Node.js comes with a package manager called NPM. As far as package managers go (and I’ve used them all except I think ant) NPM is the one that sucks least, or rather, it is pretty good.
If you are invested in Node.js programming NPM isn’t just a tool you can use, it is essential to the development and deployment workflow of your applications. It is that good.
And here is the kicker: NPM’s (de)-central registry is build on CouchDB. NPM is effectively a CouchApp. The Node elders love CouchDB.
I suggest, and plan to turn this into an opportunity for CouchDB. Say, for example, we need benchmarking tools for CouchDB. The Node folks are great at that. If we were to go and say, “hey, do you want to help make NPM faster? Help us build benchmarking tools, so we can speed up CouchDB for you.” — And that’s just one idea, the potential collaboration is immense. Add to that that we maybe get native Node support inside CouchDB, we can do all sorts of crazy shit like writing core add-ons to CouchDB in JavaScript.
Finally, I’d like to mention Hoodie, a project that Gregor, Alex, I and a few friends have been working on on and of over the past year. It is frontend web app framework that uses Node and CouchDB. We are happy to show that to you on the weekend.
I am going to mention Node.js one more time. Some of you might know The Node Firm, but for those who don’t, think of it as a company made up of the core contributors to Node.js that come together as a collective of professionals to offer services around Node.js that are complementary to the open source project. Things like consulting, technical vetting, training, things that an open source project rarely can do itself.
The key part for me to like The Node Firm (I joined recently) is that, and I repeat, it is complementary to the open source project. It doesn’t try to extract any value from the project, but merely satisfies demand for services that the project can provide. Finally, proceeds are going right back into the community, in the form of conference support, swag, stickers, t-shirts, and so on.
I think this is a brilliant set up for an open source company and I hope many will fork this model.
With that as a preamble, I am happy to announce that together with Dave Cottlehuber and the preliminary support of a few community members, we are launching The Couch Firm, modelled after The Node Firm.
It is early days for us, but expect the same setup, a business complementary to the open source project.
Above all, Open Source is about transparency and community, and I hope you feel you have an accurate picture of where we’re at right now. We had a hard year, lost our traction, and we still came out on top.
With this, I’ll leave the stage for a day of exciting speakers with all sorts of crazy cool topics.
Thank you all for being here!
On January 25-27th 2013 we ran CouchDB Conf Berlin. A one-day conference and two-day CouchHack event dedicated to the Apache CouchDB community and technology.
CouchDB Conf was born out of the desire to run an Apache CouchDB Technology conference. Couchbase’s CouchConf used to be that, but has moved away, while sadly keeping the name.
The CouchDB community is a busy bunch and any excuse to get together is taken up with enthusiasm. CouchDB Conf was sold out with ~70 participants in a wintery Berlin.
Conferences in general are great for various reasons:
the core developers get some face time to have a good time, but also to get neck deep into discussions and hacking that require in-person communication. Lots of future plans can be hashed out quickly as well. We made good use of that.
the user community gets to hang out with the core developers, learn about how they work and most importantly, get to ask all the fun, hard, silly and important questions they never dared ask before. We also made good use of that.
Everybody gets to show of what they are working on and get everybody else excited about it. We made extensive use of that. Exictement was rampant.
Eventify captured a few good impressions.
The slides for the talks are all linked up on the conference site.
One common thread through all presentations that CouchDB’s replication feature is most the popular thing CouchDB has to offer. HTTP and JSON are nice and simple, but replication is the one thing all would go out of their way to point out that it is the thing that makes CouchDB most attractive.
CouchDB’s vision early on was to be able to run on small clients, big servers, with lots of data and very little data, mobile phones and really any device that one can carry around, and crazily enough also in the browser. For the longest time though this was a bit of a pipe dream, as the Erlang CouchDB implementation wasn’t really suited to run on low-power devices and definitely not in a browser.
This now changed. BigCouch makes the start, as it handles CouchDB’s BigData and clustering aspirations. It has been around for a while, but is now finally being integrated into Apache CouchDB proper.
Hot on the heels is TouchDB, which comes in two flavours: iOS and Android, or Objective-C and Java respectively. Both are native reimplementations of CouchDB for mobile devices. TouchDB-iOS is production ready (a few apps are already live), the Android version is a little behind and both are open for contributions.
The newest player on the scene is PouchDB, a pure JavaScript implementation of CouchDB targeted at modern desktop and mobile browsers. Albeit being the newest member of the CouchDB family, it has the highest activity and was recently selected to act as a sample project in the Stanford Computer Science course. It is on the verge of being production ready.
All of this finally brings CouchDB’s vision together. Lots of data can live on BigCouch clusters, regular data can live on regular servers, user-data can live on mobile handsets, tablets and even in the browser and data can float freely between all of them. Application developers no longer have to worry about how to get a user’s data in front of them, handle concurrent updates, sync with offline devices and online servers. Even peer-to-peer sync is possible.
Towards the end of the day, when the whole vision inadvertently came together, the energy levels were very high. And nothing fuels developers better than seeing users getting excited.
Shit is getting real, as the kids say.
The following weekend was dedicated to hands on hacking, some more cool show & tell type talks and a Skype-powered video talk from Bangkok.
Among the highlights are:
Use the Time Slider feature to see the topics for all days.
As the organiser, I couldn’t be more happy about how CouchDB Conf has turned out. Of course, I couldn’t have it all done by myself, so I’d like to thank the following people and companies:
Katrin Apel, Dale Harvey, David Zuelke, Martin Hilbig, Nuno Job, Volker Mische, Oliver Kurowski, Jason Smith, Robert Newson, Joan Touzet, Stephan Strohmeier, Till Klampaeckel and Randall Leeds.
Gregor Martynus, Dave Cottlehuber and Jan Friedensreich.
Co.Up for hosting us, Tito.io for the flawless ticketing, GitHub & Cloudant for the drinks, Nodejitsu, FreshX, Mozilla, EasyBib and IrisCouch for sending people or donating company time.
There are preliminary plans to set up a CouchDB Conf in Seattle later this year. We’ll keep you posted.
Members of the CouchDB Community will be attending the following conferences:
You are welcome to join us and nerd out about CouchDB :)
Thanks!
Jan
About two weeks ago I parted ways with Couchbase’s day to day business. Not that I was involved much in the end, but it was time to move on. I remain a co-founder and shareholder, but my regular involvement is over.
This wasn’t an easy decision, and as those go, it took me a good while to make it.
About a year ago I started noticing a change. Things I’ve always cared about intensified manyfold and my work situation didn’t leave me with much time and energy to pursue them.
Couchbase made every attempt to accommodate me, let me work on Apache CouchDB for most of the past year, and help with an arrangement where I could spend some of my time at my discretion (with no pay, but enough work to keep me afloat) and I am very grateful for that opportunity.
I spare you the details (ask me over a drink), but in the end I had to answer the question of whether I am a CouchDB person or a Couchbase person. I had to answer that for my working relationship with Couchbase and ultimately I had to answer that for myself.
My heart knew, of course, but leaving a company that you helped founding isn’t the easiest of things to consider. I carried that conflict through most of 2012 and I tried to make things work in a compromise, but ultimately failed, realising that I am, at heart, a CouchDB person.
2012 wasn’t the best of years for CouchDB. That helped me realise how much I believe in the idea and the community behind it.
I learned a lot about myself, turned 30 in December. Around that time, the introspective period of the year turned into action. I was done contemplating what to do with newfound revelations and had a vague plan of the direction things should go towards.
There are a number of distinct moments that helped me figure out what is important to me.
The Rails Girls movement teaches non-programmers web programming for free. Volunteers from the Rails community would spend their weekends teaching, regular week-nights coaching development groups. All stuff I thought was honourable, but I didn’t see how I could fit that into my schedule, along my regular work and have enough energy to do it all. Yet I saw others doing just that. This created an urgency for changes, so I could, while not teaching Rails Girls, do things I deem important. More on that later.
In the spirit of Rails Girls we started a JavaScript workshop JSFAB (JavaScript for Absolute Beginners) which, along with other similar initiatives eventually culminated in OpenTechSchool, a roof organisation for volunteer learning and teaching. The idea spreads like fire, still, into other tech communities, and other cities and countries. This is truly a tech education innovation. I am very proud to be a part of this.
Around the US Equal Pay Day Gina Trapani put up Narrow the Gapp. Anne suggested there should be something like that for Germany, so I spent a total of four or so hours to rip off the project’s UI (open source) and made it so Anne could put in the hard data into a JSON file on GitHub.
Gleicherlohn.de was a mini project with a large effect. The lesson for me though was that web technology, while incredibly accessible already, is still not easy enough to handle for regular people, even technically inclined non-programmers struggle. Why, I thought, would Anne not be able to put this up all by herself.
Then I realised what my original motivation for working on CouchDB and eventually founding Couch.io was: empowering people to use technology to their advantage. I wanted to get back to that.
While talking to friends last winter I made the realisation that we technology people are a rather privileged bunch. Not that there is anything wrong with that, but for me that means finding ways to give back to the communities that enable me to live such a privileged life. Teaching and making technology more accessible fit right in there, so it wasn’t hard to figure out what to do. Instead of working five or seven days a week, I tried to find a way to work as much as I need to live comfortably, while having enough time and energy to pursue pro-bono work. Again, Couchbase was very accommodating here.
One last topic that found its way to me is diversity in tech communities. The numbers don’t look particularly good, so I tried to find ways to get this improved. This lead to helping Tiffany to get We Are All Awesome set up, start our diversity-friendly Call for Papers for JSConf EU and the result that we were able to organically get 25% women speakers, an absolute novelty for hardcore tech conferences.
More importantly though is that we inspired many more events to follow in our footsteps, improve on what we did and share the results. I lost count of how many conferences took after us, but is easily 20+. The resulting cultural change is immense. I couldn’t be more proud of that.
I want to make sure I can continue to do this work and I hope I can inspire more people to do the same in the way that works best for them. In terms of work, I want to be able to work on things that matter and get paid fairly, but also not over-do it. Leave enough time to work outside of work.
To that end, I am doing three things:
The Node Firm is a complementary company to the Node.js open source project. A collaboration of professionals to support the open source projects in areas it can’t provide itself.
The Couch Firm is exactly that, but for CouchDB. It is early days, I’ll keep you posted.
Hoodie is a thing to be revealed at a later time, but it is not exactly a secret, if you know where to look.
2013 looks mighty exciting from where I am sitting.
I can’t wait to get started!
Slides to go with the headings.
Hi, welcome to CouchDB Conf.
CouchDB means a lot to me and it makes me very happy that you are all here today.
My name is Jan Lehnardt and I am the elected project management committee chair of the Apache CouchDB project. If you don’t know what this means, don’t worry, it means I sign up to deal with politics and paperwork voluntarily.
I have been with CouchDB since 2006. My first contribution was the initial UNIX build system to CouchDB. I stuck around ever since.
My main job back then was PHP/MySQL consulting and Open Source work as my own 20% project. In 2007, I started giving talks about CouchDB at user groups and conferences and quickly became addicted. People called me the CouchDB Chief Evangelist, although I don’t really like the title. From 2008 I started getting contracting jobs specifically for CouchDB and things blew up quickly that it became clear soon, that an organisation of some sort was needed to handle this. I eventually joined forces with CouchDB’s original inventor and J Chris Anderson to form the company Couch.io. We initially weren’t keen on taking venture capital, but when the whole NoSQL thing blew up, it was very hard to say no and on January 1st 2010 we were in business.
Couch.io turned into CouchOne and was eventually merged with Membase into Couchbase. Unfortunately, at this point, there isn’t much Couch-y about Couchbase aside from some underlying technology, so the name can be confusing.
I parted ways with Couchbase a week ago in order to concentrate and focus more on Apache CouchDB and related technologies.
In late 2011 two major supporters of CouchDB withdrew their support: Canonical’s UbuntuOne service, on paper a poster-child use-case for CouchDB and Saucelab’s Selenium platform publicly made clear that they were moving to other technology. It wasn’t a big deal if you looked closely, the one project didn’t have enough resources to integrate with CouchDB the right way and the other diverged so far from the original system design, that CouchDB was no longer a good fit. We were and we remain good friends with the respective developers, so we didn’t feel like pointing that out specifically. The result though was that people who didn’t look closely saw that CouchDB had popular failures where it hadn’t. This wasn’t all bad, but set the stage for another incident a few weeks later.
CouchDB’s original inventor publicly announced that he’d move on from Apache CouchDB to Couchbase fully. That new technology would only go into Couchbase. If you are in the know, this wasn’t a big deal again, he hadn’t contributed to CouchDB in two years, Couchbase was a different thing and he wanted to rally support for that. The casual reader though took away one lesson: “CouchDB is dead”.
With the added confusion of Couchbase’s name, and continuing their CouchConf conferences, things looks dire.
With this in the wild, the active CouchDB committers got together and put out an excellent 1.2.0 release that was accompanied by a new, modern website design. We thought instead of writing counter-statements and start unnecessary he-said-she-said discussions, we’d rather concentrate on shipping good software. 1.2.0 was well received and the whole thing lead the team to come together as a whole in Boston and two weeks later in a smaller group in Dublin in April 2012.
We came up with a long list of of features that we wanted to get into CouchDB. Some work happened over the summer, but I personally took a bit of a break to avoid burn-out from CouchDB. In fall, things started moving faster again when more community members started pushing things along. New contributors came and we finally came up with a way to go forward.
We decided to not be insane. The definition of insanity is trying the same thing over and over again and expecting different results. While it wasn’t all bad in CouchDB land we could do better and to do better we had to change a few things.
We determined that CouchDB needs a fresh mission statement. I signed up to come up with a few drafts, but nothing happened yet, so this one is still out there cough.
We decided to release more often. With the upcoming 1.3.0 release we will switch to a quarterly release cycle. A new minor version will come out approximately every 90 days, even if it is just a few bug fixes that a small number of people direly need.
CouchDB 1.3.0 will ship with top-notch API documentation right inside of Futon, the built in admin interface.
We will start to set up teams where anyone can sign up to work on a specific thing: docs, our JavaScript components, the Erlang core, the admin interface (built in HTML+JS). We want to make sure anyone who has the right skills can contribute in their area of expertise and not be scared away by the Erlang.
We put some emphasis on QA, automated testing, building binaries and packages for releases as well as in-development versions, so people can try new features before they come out more easily and as a result report issues early.
A few of the core team will put some focus on encouraging contributions rather than coding themselves. We hope to get more people on board with CouchDB development.
We also started doing roughly quarterly in-person meetings (at least as smaller groups) and weekly status meetings on IRC to give everything a little more focus and transparency.
The next few slides show a long list a few features that we’ll be working on. If you think anything is missing, let us know!
The more prominent things are: Merging rcouch, BigCouch, the new Futon, the plugin system and v8/node integration.
rcouch is a cutting edge distribution of CouchDB with many experimental features. A lot of that work is useful for Apache CouchDB and we plan to fold those back into the project.
BigCouch is a industry-strength clustering solution for CouchDB to allow seamless scaling and storing of lots of data. BigCouch is going to be merged into mainline CouchDB as well.
The admin interface Futon is getting a complete refresh with modern web development technologies.
There are many features hidden in some developer’s git repository because, while useful, it wasn’t generally applicable to CouchDB. With the plugin system, we hope that we can fuel a repository of plugins to CouchDB that are easily installed (think browser extensions, right from Futon). For example, you could install GeoCouch with the click of a button.
Finally, we are currently discussing the move from SpiderMonkey and to V8 or even Node. The jury is still out on which one it is going to be, but we hope that this is going to open up a lot more opportunities for CouchDB in terms of features and new contributors.
CouchDB would be nothing without you, the community. People who know little about CouchDB might like it or not, but people who really get to know CouchDB are extremely passionate about it.
CouchDB’s core idea, the promise that data can be anywhere it is needed, to make applications easier, more approachable and effective for people in modern computing environments, that idea is a big one. It’s the one that brings this community together.
Thanks again for coming, you are all awesome. CouchDB means a lot to me and I get a little teary seeing you all here.
If you like any of the above todos, or have some of your own, please help out, get in touch about how.
The CouchDB ecosystem is full of exciting projects that support CouchDB in one way or another.
PouchDB is the newest project on the scene. PouchDB is a pure JavaScript implementation of CouchDB targeted at the browser.
Let me say that again: CouchDB in JavaScript, for the browser.
PouchDB is the oldest idea, but it took the longest to realise because browsers weren’t ready for it. Now is the time though that we can build and ship a CouchDB compatible data store and replication engine that lives entirely in your browser. If you want to build web apps that can seamlessly synchronise data between a laptop, a phone and a server and not treat one as a “master” and the other as “slaves”, PouchDB is your solution.
TouchDB-iOS and -Android are Objective-C and Java implementations of the same idea, but for mobile phone platforms. They natively integrate into your iOS and Android applications and allow synchronisation between themselves, multiple devices, CouchDB’s on the server and even PouchDB in a browser. Again, a milestone in fulfilling the CouchDB vision: data where it is needed, not on some server, far away over a slow and unreliable or even unavailable connection.
This might not be too obvious to people who don’t do Node, but CouchDB is big, if not essential here, too. As any programming language & platform, Node.js comes with a package manager called NPM. As far as package managers go (and I’ve used them all except I think ant) NPM is the one that sucks least, or rather, it is pretty good.
If you are invested in Node.js programming NPM isn’t just a tool you can use, it is essential to the development and deployment workflow of your applications. It is that good.
And here is the kicker: NPM’s (de)-central registry is build on CouchDB. NPM is effectively a CouchApp. The Node elders love CouchDB.
I suggest, and plan to turn this into an opportunity for CouchDB. Say, for example, we need benchmarking tools for CouchDB. The Node folks are great at that. If we were to go and say, “hey, do you want to help make NPM faster? Help us build benchmarking tools, so we can speed up CouchDB for you.” — And that’s just one idea, the potential collaboration is immense. Add to that that we maybe get native Node support inside CouchDB, we can do all sorts of crazy shit like writing core add-ons to CouchDB in JavaScript.
Finally, I’d like to mention Hoodie, a project that Gregor, Alex, I and a few friends have been working on on and of over the past year. It is frontend web app framework that uses Node and CouchDB. We are happy to show that to you on the weekend.
I am going to mention Node.js one more time. Some of you might know The Node Firm, but for those who don’t, think of it as a company made up of the core contributors to Node.js that come together as a collective of professionals to offer services around Node.js that are complementary to the open source project. Things like consulting, technical vetting, training, things that an open source project rarely can do itself.
The key part for me to like The Node Firm (I joined recently) is that, and I repeat, it is complementary to the open source project. It doesn’t try to extract any value from the project, but merely satisfies demand for services that the project can provide. Finally, proceeds are going right back into the community, in the form of conference support, swag, stickers, t-shirts, and so on.
I think this is a brilliant set up for an open source company and I hope many will fork this model.
With that as a preamble, I am happy to announce that together with Dave Cottlehuber and the preliminary support of a few community members, we are launching The Couch Firm, modelled after The Node Firm.
It is early days for us, but expect the same setup, a business complementary to the open source project.
Above all, Open Source is about transparency and community, and I hope you feel you have an accurate picture of where we’re at right now. We had a hard year, lost our traction, and we still came out on top.
With this, I’ll leave the stage for a day of exciting speakers with all sorts of crazy cool topics.
Thank you all for being here!
Slides to go with the headings.
Hi, welcome to CouchDB Conf.
CouchDB means a lot to me and it makes me very happy that you are all here today.
My name is Jan Lehnardt and I am the elected project management committee chair of the Apache CouchDB project. If you don’t know what this means, don’t worry, it means I sign up to deal with politics and paperwork voluntarily.
I have been with CouchDB since 2006. My first contribution was the initial UNIX build system to CouchDB. I stuck around ever since.
My main job back then was PHP/MySQL consulting and Open Source work as my own 20% project. In 2007, I started giving talks about CouchDB at user groups and conferences and quickly became addicted. People called me the CouchDB Chief Evangelist, although I don’t really like the title. From 2008 I started getting contracting jobs specifically for CouchDB and things blew up quickly that it became clear soon, that an organisation of some sort was needed to handle this. I eventually joined forces with CouchDB’s original inventor and J Chris Anderson to form the company Couch.io. We initially weren’t keen on taking venture capital, but when the whole NoSQL thing blew up, it was very hard to say no and on January 1st 2010 we were in business.
Couch.io turned into CouchOne and was eventually merged with Membase into Couchbase. Unfortunately, at this point, there isn’t much Couch-y about Couchbase aside from some underlying technology, so the name can be confusing.
I parted ways with Couchbase a week ago in order to concentrate and focus more on Apache CouchDB and related technologies.
In late 2011 two major supporters of CouchDB withdrew their support: Canonical’s UbuntuOne service, on paper a poster-child use-case for CouchDB and Saucelab’s Selenium platform publicly made clear that they were moving to other technology. It wasn’t a big deal if you looked closely, the one project didn’t have enough resources to integrate with CouchDB the right way and the other diverged so far from the original system design, that CouchDB was no longer a good fit. We were and we remain good friends with the respective developers, so we didn’t feel like pointing that out specifically. The result though was that people who didn’t look closely saw that CouchDB had popular failures where it hadn’t. This wasn’t all bad, but set the stage for another incident a few weeks later.
CouchDB’s original inventor publicly announced that he’d move on from Apache CouchDB to Couchbase fully. That new technology would only go into Couchbase. If you are in the know, this wasn’t a big deal again, he hadn’t contributed to CouchDB in two years, Couchbase was a different thing and he wanted to rally support for that. The casual reader though took away one lesson: “CouchDB is dead”.
With the added confusion of Couchbase’s name, and continuing their CouchConf conferences, things looks dire.
With this in the wild, the active CouchDB committers got together and put out an excellent 1.2.0 release that was accompanied by a new, modern website design. We thought instead of writing counter-statements and start unnecessary he-said-she-said discussions, we’d rather concentrate on shipping good software. 1.2.0 was well received and the whole thing lead the team to come together as a whole in Boston and two weeks later in a smaller group in Dublin in April 2012.
We came up with a long list of of features that we wanted to get into CouchDB. Some work happened over the summer, but I personally took a bit of a break to avoid burn-out from CouchDB. In fall, things started moving faster again when more community members started pushing things along. New contributors came and we finally came up with a way to go forward.
We decided to not be insane. The definition of insanity is trying the same thing over and over again and expecting different results. While it wasn’t all bad in CouchDB land we could do better and to do better we had to change a few things.
We determined that CouchDB needs a fresh mission statement. I signed up to come up with a few drafts, but nothing happened yet, so this one is still out there cough.
We decided to release more often. With the upcoming 1.3.0 release we will switch to a quarterly release cycle. A new minor version will come out approximately every 90 days, even if it is just a few bug fixes that a small number of people direly need.
CouchDB 1.3.0 will ship with top-notch API documentation right inside of Futon, the built in admin interface.
We will start to set up teams where anyone can sign up to work on a specific thing: docs, our JavaScript components, the Erlang core, the admin interface (built in HTML+JS). We want to make sure anyone who has the right skills can contribute in their area of expertise and not be scared away by the Erlang.
We put some emphasis on QA, automated testing, building binaries and packages for releases as well as in-development versions, so people can try new features before they come out more easily and as a result report issues early.
A few of the core team will put some focus on encouraging contributions rather than coding themselves. We hope to get more people on board with CouchDB development.
We also started doing roughly quarterly in-person meetings (at least as smaller groups) and weekly status meetings on IRC to give everything a little more focus and transparency.
The next few slides show a long list a few features that we’ll be working on. If you think anything is missing, let us know!
The more prominent things are: Merging rcouch, BigCouch, the new Futon, the plugin system and v8/node integration.
rcouch is a cutting edge distribution of CouchDB with many experimental features. A lot of that work is useful for Apache CouchDB and we plan to fold those back into the project.
BigCouch is a industry-strength clustering solution for CouchDB to allow seamless scaling and storing of lots of data. BigCouch is going to be merged into mainline CouchDB as well.
The admin interface Futon is getting a complete refresh with modern web development technologies.
There are many features hidden in some developer’s git repository because, while useful, it wasn’t generally applicable to CouchDB. With the plugin system, we hope that we can fuel a repository of plugins to CouchDB that are easily installed (think browser extensions, right from Futon). For example, you could install GeoCouch with the click of a button.
Finally, we are currently discussing the move from SpiderMonkey and to V8 or even Node. The jury is still out on which one it is going to be, but we hope that this is going to open up a lot more opportunities for CouchDB in terms of features and new contributors.
CouchDB would be nothing without you, the community. People who know little about CouchDB might like it or not, but people who really get to know CouchDB are extremely passionate about it.
CouchDB’s core idea, the promise that data can be anywhere it is needed, to make applications easier, more approachable and effective for people in modern computing environments, that idea is a big one. It’s the one that brings this community together.
Thanks again for coming, you are all awesome. CouchDB means a lot to me and I get a little teary seeing you all here.
If you like any of the above todos, or have some of your own, please help out, get in touch about how.
The CouchDB ecosystem is full of exciting projects that support CouchDB in one way or another.
PouchDB is the newest project on the scene. PouchDB is a pure JavaScript implementation of CouchDB targeted at the browser.
Let me say that again: CouchDB in JavaScript, for the browser.
PouchDB is the oldest idea, but it took the longest to realise because browsers weren’t ready for it. Now is the time though that we can build and ship a CouchDB compatible data store and replication engine that lives entirely in your browser. If you want to build web apps that can seamlessly synchronise data between a laptop, a phone and a server and not treat one as a “master” and the other as “slaves”, PouchDB is your solution.
TouchDB-iOS and -Android are Objective-C and Java implementations of the same idea, but for mobile phone platforms. They natively integrate into your iOS and Android applications and allow synchronisation between themselves, multiple devices, CouchDB’s on the server and even PouchDB in a browser. Again, a milestone in fulfilling the CouchDB vision: data where it is needed, not on some server, far away over a slow and unreliable or even unavailable connection.
This might not be too obvious to people who don’t do Node, but CouchDB is big, if not essential here, too. As any programming language & platform, Node.js comes with a package manager called NPM. As far as package managers go (and I’ve used them all except I think ant) NPM is the one that sucks least, or rather, it is pretty good.
If you are invested in Node.js programming NPM isn’t just a tool you can use, it is essential to the development and deployment workflow of your applications. It is that good.
And here is the kicker: NPM’s (de)-central registry is build on CouchDB. NPM is effectively a CouchApp. The Node elders love CouchDB.
I suggest, and plan to turn this into an opportunity for CouchDB. Say, for example, we need benchmarking tools for CouchDB. The Node folks are great at that. If we were to go and say, “hey, do you want to help make NPM faster? Help us build benchmarking tools, so we can speed up CouchDB for you.” — And that’s just one idea, the potential collaboration is immense. Add to that that we maybe get native Node support inside CouchDB, we can do all sorts of crazy shit like writing core add-ons to CouchDB in JavaScript.
Finally, I’d like to mention Hoodie, a project that Gregor, Alex, I and a few friends have been working on on and of over the past year. It is frontend web app framework that uses Node and CouchDB. We are happy to show that to you on the weekend.
I am going to mention Node.js one more time. Some of you might know The Node Firm, but for those who don’t, think of it as a company made up of the core contributors to Node.js that come together as a collective of professionals to offer services around Node.js that are complementary to the open source project. Things like consulting, technical vetting, training, things that an open source project rarely can do itself.
The key part for me to like The Node Firm (I joined recently) is that, and I repeat, it is complementary to the open source project. It doesn’t try to extract any value from the project, but merely satisfies demand for services that the project can provide. Finally, proceeds are going right back into the community, in the form of conference support, swag, stickers, t-shirts, and so on.
I think this is a brilliant set up for an open source company and I hope many will fork this model.
With that as a preamble, I am happy to announce that together with Dave Cottlehuber and the preliminary support of a few community members, we are launching The Couch Firm, modelled after The Node Firm.
It is early days for us, but expect the same setup, a business complementary to the open source project.
Above all, Open Source is about transparency and community, and I hope you feel you have an accurate picture of where we’re at right now. We had a hard year, lost our traction, and we still came out on top.
With this, I’ll leave the stage for a day of exciting speakers with all sorts of crazy cool topics.
Thank you all for being here!
On January 25-27th 2013 we ran CouchDB Conf Berlin. A one-day conference and two-day CouchHack event dedicated to the Apache CouchDB community and technology.
CouchDB Conf was born out of the desire to run an Apache CouchDB Technology conference. Couchbase’s CouchConf used to be that, but has moved away, while sadly keeping the name.
The CouchDB community is a busy bunch and any excuse to get together is taken up with enthusiasm. CouchDB Conf was sold out with ~70 participants in a wintery Berlin.
Conferences in general are great for various reasons:
the core developers get some face time to have a good time, but also to get neck deep into discussions and hacking that require in-person communication. Lots of future plans can be hashed out quickly as well. We made good use of that.
the user community gets to hang out with the core developers, learn about how they work and most importantly, get to ask all the fun, hard, silly and important questions they never dared ask before. We also made good use of that.
Everybody gets to show of what they are working on and get everybody else excited about it. We made extensive use of that. Exictement was rampant.
Eventify captured a few good impressions.
The slides for the talks are all linked up on the conference site.
One common thread through all presentations that CouchDB’s replication feature is most the popular thing CouchDB has to offer. HTTP and JSON are nice and simple, but replication is the one thing all would go out of their way to point out that it is the thing that makes CouchDB most attractive.
CouchDB’s vision early on was to be able to run on small clients, big servers, with lots of data and very little data, mobile phones and really any device that one can carry around, and crazily enough also in the browser. For the longest time though this was a bit of a pipe dream, as the Erlang CouchDB implementation wasn’t really suited to run on low-power devices and definitely not in a browser.
This now changed. BigCouch makes the start, as it handles CouchDB’s BigData and clustering aspirations. It has been around for a while, but is now finally being integrated into Apache CouchDB proper.
Hot on the heels is TouchDB, which comes in two flavours: iOS and Android, or Objective-C and Java respectively. Both are native reimplementations of CouchDB for mobile devices. TouchDB-iOS is production ready (a few apps are already live), the Android version is a little behind and both are open for contributions.
The newest player on the scene is PouchDB, a pure JavaScript implementation of CouchDB targeted at modern desktop and mobile browsers. Albeit being the newest member of the CouchDB family, it has the highest activity and was recently selected to act as a sample project in the Stanford Computer Science course. It is on the verge of being production ready.
All of this finally brings CouchDB’s vision together. Lots of data can live on BigCouch clusters, regular data can live on regular servers, user-data can live on mobile handsets, tablets and even in the browser and data can float freely between all of them. Application developers no longer have to worry about how to get a user’s data in front of them, handle concurrent updates, sync with offline devices and online servers. Even peer-to-peer sync is possible.
Towards the end of the day, when the whole vision inadvertently came together, the energy levels were very high. And nothing fuels developers better than seeing users getting excited.
Shit is getting real, as the kids say.
The following weekend was dedicated to hands on hacking, some more cool show & tell type talks and a Skype-powered video talk from Bangkok.
Among the highlights are:
Use the Time Slider feature to see the topics for all days.
As the organiser, I couldn’t be more happy about how CouchDB Conf has turned out. Of course, I couldn’t have it all done by myself, so I’d like to thank the following people and companies:
Katrin Apel, Dale Harvey, David Zuelke, Martin Hilbig, Nuno Job, Volker Mische, Oliver Kurowski, Jason Smith, Robert Newson, Joan Touzet, Stephan Strohmeier, Till Klampaeckel and Randall Leeds.
Gregor Martynus, Dave Cottlehuber and Jan Friedensreich.
Co.Up for hosting us, Tito.io for the flawless ticketing, GitHub & Cloudant for the drinks, Nodejitsu, FreshX, Mozilla, EasyBib and IrisCouch for sending people or donating company time.
There are preliminary plans to set up a CouchDB Conf in Seattle later this year. We’ll keep you posted.
Members of the CouchDB Community will be attending the following conferences:
You are welcome to join us and nerd out about CouchDB :)
Thanks!
Jan
On January 25-27th 2013 we ran CouchDB Conf Berlin. A one-day conference and two-day CouchHack event dedicated to the Apache CouchDB community and technology.
CouchDB Conf was born out of the desire to run an Apache CouchDB Technology conference. Couchbase’s CouchConf used to be that, but has moved away, while sadly keeping the name.
The CouchDB community is a busy bunch and any excuse to get together is taken up with enthusiasm. CouchDB Conf was sold out with ~70 participants in a wintery Berlin.
Conferences in general are great for various reasons:
the core developers get some face time to have a good time, but also to get neck deep into discussions and hacking that require in-person communication. Lots of future plans can be hashed out quickly as well. We made good use of that.
the user community gets to hang out with the core developers, learn about how they work and most importantly, get to ask all the fun, hard, silly and important questions they never dared ask before. We also made good use of that.
Everybody gets to show of what they are working on and get everybody else excited about it. We made extensive use of that. Exictement was rampant.
Eventify captured a few good impressions.
The slides for the talks are all linked up on the conference site.
One common thread through all presentations that CouchDB’s replication feature is most the popular thing CouchDB has to offer. HTTP and JSON are nice and simple, but replication is the one thing all would go out of their way to point out that it is the thing that makes CouchDB most attractive.
CouchDB’s vision early on was to be able to run on small clients, big servers, with lots of data and very little data, mobile phones and really any device that one can carry around, and crazily enough also in the browser. For the longest time though this was a bit of a pipe dream, as the Erlang CouchDB implementation wasn’t really suited to run on low-power devices and definitely not in a browser.
This now changed. BigCouch makes the start, as it handles CouchDB’s BigData and clustering aspirations. It has been around for a while, but is now finally being integrated into Apache CouchDB proper.
Hot on the heels is TouchDB, which comes in two flavours: iOS and Android, or Objective-C and Java respectively. Both are native reimplementations of CouchDB for mobile devices. TouchDB-iOS is production ready (a few apps are already live), the Android version is a little behind and both are open for contributions.
The newest player on the scene is PouchDB, a pure JavaScript implementation of CouchDB targeted at modern desktop and mobile browsers. Albeit being the newest member of the CouchDB family, it has the highest activity and was recently selected to act as a sample project in the Stanford Computer Science course. It is on the verge of being production ready.
All of this finally brings CouchDB’s vision together. Lots of data can live on BigCouch clusters, regular data can live on regular servers, user-data can live on mobile handsets, tablets and even in the browser and data can float freely between all of them. Application developers no longer have to worry about how to get a user’s data in front of them, handle concurrent updates, sync with offline devices and online servers. Even peer-to-peer sync is possible.
Towards the end of the day, when the whole vision inadvertently came together, the energy levels were very high. And nothing fuels developers better than seeing users getting excited.
Shit is getting real, as the kids say.
The following weekend was dedicated to hands on hacking, some more cool show & tell type talks and a Skype-powered video talk from Bangkok.
Among the highlights are:
Use the Time Slider feature to see the topics for all days.
As the organiser, I couldn’t be more happy about how CouchDB Conf has turned out. Of course, I couldn’t have it all done by myself, so I’d like to thank the following people and companies:
Katrin Apel, Dale Harvey, David Zuelke, Martin Hilbig, Nuno Job, Volker Mische, Oliver Kurowski, Jason Smith, Robert Newson, Joan Touzet, Stephan Strohmeier, Till Klampaeckel and Randall Leeds.
Gregor Martynus, Dave Cottlehuber and Jan Friedensreich.
Co.Up for hosting us, Tito.io for the flawless ticketing, GitHub & Cloudant for the drinks, Nodejitsu, FreshX, Mozilla, EasyBib and IrisCouch for sending people or donating company time.
There are preliminary plans to set up a CouchDB Conf in Seattle later this year. We’ll keep you posted.
Members of the CouchDB Community will be attending the following conferences:
You are welcome to join us and nerd out about CouchDB :)
Thanks!
Jan
About two weeks ago I parted ways with Couchbase’s day to day business. Not that I was involved much in the end, but it was time to move on. I remain a co-founder and shareholder, but my regular involvement is over.
This wasn’t an easy decision, and as those go, it took me a good while to make it.
About a year ago I started noticing a change. Things I’ve always cared about intensified manyfold and my work situation didn’t leave me with much time and energy to pursue them.
Couchbase made every attempt to accommodate me, let me work on Apache CouchDB for most of the past year, and help with an arrangement where I could spend some of my time at my discretion (with no pay, but enough work to keep me afloat) and I am very grateful for that opportunity.
I spare you the details (ask me over a drink), but in the end I had to answer the question of whether I am a CouchDB person or a Couchbase person. I had to answer that for my working relationship with Couchbase and ultimately I had to answer that for myself.
My heart knew, of course, but leaving a company that you helped founding isn’t the easiest of things to consider. I carried that conflict through most of 2012 and I tried to make things work in a compromise, but ultimately failed, realising that I am, at heart, a CouchDB person.
2012 wasn’t the best of years for CouchDB. That helped me realise how much I believe in the idea and the community behind it.
I learned a lot about myself, turned 30 in December. Around that time, the introspective period of the year turned into action. I was done contemplating what to do with newfound revelations and had a vague plan of the direction things should go towards.
There are a number of distinct moments that helped me figure out what is important to me.
The Rails Girls movement teaches non-programmers web programming for free. Volunteers from the Rails community would spend their weekends teaching, regular week-nights coaching development groups. All stuff I thought was honourable, but I didn’t see how I could fit that into my schedule, along my regular work and have enough energy to do it all. Yet I saw others doing just that. This created an urgency for changes, so I could, while not teaching Rails Girls, do things I deem important. More on that later.
In the spirit of Rails Girls we started a JavaScript workshop JSFAB (JavaScript for Absolute Beginners) which, along with other similar initiatives eventually culminated in OpenTechSchool, a roof organisation for volunteer learning and teaching. The idea spreads like fire, still, into other tech communities, and other cities and countries. This is truly a tech education innovation. I am very proud to be a part of this.
Around the US Equal Pay Day Gina Trapani put up Narrow the Gapp. Anne suggested there should be something like that for Germany, so I spent a total of four or so hours to rip off the project’s UI (open source) and made it so Anne could put in the hard data into a JSON file on GitHub.
Gleicherlohn.de was a mini project with a large effect. The lesson for me though was that web technology, while incredibly accessible already, is still not easy enough to handle for regular people, even technically inclined non-programmers struggle. Why, I thought, would Anne not be able to put this up all by herself.
Then I realised what my original motivation for working on CouchDB and eventually founding Couch.io was: empowering people to use technology to their advantage. I wanted to get back to that.
While talking to friends last winter I made the realisation that we technology people are a rather privileged bunch. Not that there is anything wrong with that, but for me that means finding ways to give back to the communities that enable me to live such a privileged life. Teaching and making technology more accessible fit right in there, so it wasn’t hard to figure out what to do. Instead of working five or seven days a week, I tried to find a way to work as much as I need to live comfortably, while having enough time and energy to pursue pro-bono work. Again, Couchbase was very accommodating here.
One last topic that found its way to me is diversity in tech communities. The numbers don’t look particularly good, so I tried to find ways to get this improved. This lead to helping Tiffany to get We Are All Awesome set up, start our diversity-friendly Call for Papers for JSConf EU and the result that we were able to organically get 25% women speakers, an absolute novelty for hardcore tech conferences.
More importantly though is that we inspired many more events to follow in our footsteps, improve on what we did and share the results. I lost count of how many conferences took after us, but is easily 20+. The resulting cultural change is immense. I couldn’t be more proud of that.
I want to make sure I can continue to do this work and I hope I can inspire more people to do the same in the way that works best for them. In terms of work, I want to be able to work on things that matter and get paid fairly, but also not over-do it. Leave enough time to work outside of work.
To that end, I am doing three things:
The Node Firm is a complementary company to the Node.js open source project. A collaboration of professionals to support the open source projects in areas it can’t provide itself.
The Couch Firm is exactly that, but for CouchDB. It is early days, I’ll keep you posted.
Hoodie is a thing to be revealed at a later time, but it is not exactly a secret, if you know where to look.
2013 looks mighty exciting from where I am sitting.
I can’t wait to get started!
About two weeks ago I parted ways with Couchbase’s day to day business. Not that I was involved much in the end, but it was time to move on. I remain a co-founder and shareholder, but my regular involvement is over.
This wasn’t an easy decision, and as those go, it took me a good while to make it.
About a year ago I started noticing a change. Things I’ve always cared about intensified manyfold and my work situation didn’t leave me with much time and energy to pursue them.
Couchbase made every attempt to accommodate me, let me work on Apache CouchDB for most of the past year, and help with an arrangement where I could spend some of my time at my discretion (with no pay, but enough work to keep me afloat) and I am very grateful for that opportunity.
I spare you the details (ask me over a drink), but in the end I had to answer the question of whether I am a CouchDB person or a Couchbase person. I had to answer that for my working relationship with Couchbase and ultimately I had to answer that for myself.
My heart knew, of course, but leaving a company that you helped founding isn’t the easiest of things to consider. I carried that conflict through most of 2012 and I tried to make things work in a compromise, but ultimately failed, realising that I am, at heart, a CouchDB person.
2012 wasn’t the best of years for CouchDB. That helped me realise how much I believe in the idea and the community behind it.
I learned a lot about myself, turned 30 in December. Around that time, the introspective period of the year turned into action. I was done contemplating what to do with newfound revelations and had a vague plan of the direction things should go towards.
There are a number of distinct moments that helped me figure out what is important to me.
The Rails Girls movement teaches non-programmers web programming for free. Volunteers from the Rails community would spend their weekends teaching, regular week-nights coaching development groups. All stuff I thought was honourable, but I didn’t see how I could fit that into my schedule, along my regular work and have enough energy to do it all. Yet I saw others doing just that. This created an urgency for changes, so I could, while not teaching Rails Girls, do things I deem important. More on that later.
In the spirit of Rails Girls we started a JavaScript workshop JSFAB (JavaScript for Absolute Beginners) which, along with other similar initiatives eventually culminated in OpenTechSchool, a roof organisation for volunteer learning and teaching. The idea spreads like fire, still, into other tech communities, and other cities and countries. This is truly a tech education innovation. I am very proud to be a part of this.
Around the US Equal Pay Day Gina Trapani put up Narrow the Gapp. Anne suggested there should be something like that for Germany, so I spent a total of four or so hours to rip off the project’s UI (open source) and made it so Anne could put in the hard data into a JSON file on GitHub.
Gleicherlohn.de was a mini project with a large effect. The lesson for me though was that web technology, while incredibly accessible already, is still not easy enough to handle for regular people, even technically inclined non-programmers struggle. Why, I thought, would Anne not be able to put this up all by herself.
Then I realised what my original motivation for working on CouchDB and eventually founding Couch.io was: empowering people to use technology to their advantage. I wanted to get back to that.
While talking to friends last winter I made the realisation that we technology people are a rather privileged bunch. Not that there is anything wrong with that, but for me that means finding ways to give back to the communities that enable me to live such a privileged life. Teaching and making technology more accessible fit right in there, so it wasn’t hard to figure out what to do. Instead of working five or seven days a week, I tried to find a way to work as much as I need to live comfortably, while having enough time and energy to pursue pro-bono work. Again, Couchbase was very accommodating here.
One last topic that found its way to me is diversity in tech communities. The numbers don’t look particularly good, so I tried to find ways to get this improved. This lead to helping Tiffany to get We Are All Awesome set up, start our diversity-friendly Call for Papers for JSConf EU and the result that we were able to organically get 25% women speakers, an absolute novelty for hardcore tech conferences.
More importantly though is that we inspired many more events to follow in our footsteps, improve on what we did and share the results. I lost count of how many conferences took after us, but is easily 20+. The resulting cultural change is immense. I couldn’t be more proud of that.
I want to make sure I can continue to do this work and I hope I can inspire more people to do the same in the way that works best for them. In terms of work, I want to be able to work on things that matter and get paid fairly, but also not over-do it. Leave enough time to work outside of work.
To that end, I am doing three things:
The Node Firm is a complementary company to the Node.js open source project. A collaboration of professionals to support the open source projects in areas it can’t provide itself.
The Couch Firm is exactly that, but for CouchDB. It is early days, I’ll keep you posted.
Hoodie is a thing to be revealed at a later time, but it is not exactly a secret, if you know where to look.
2013 looks mighty exciting from where I am sitting.
I can’t wait to get started!
Before I joined Cloudant, I worked in what we now simply call “Enterprise”. Not, you understand, the one where you go into space and meet aliens (and maybe learn a little about the human condition) but the other one, where you go into meetings and meet managers (and learn a lot about the human condition). It was there that I first encountered Lucene.
This was before the days of Solr, and way before the days of Elastic Search, a time when a Lucene index was an island, entire of itself. I was involved in building a proprietary clustered search platform and saw up close just how difficult that can be. Later, I was searching for some foundational technology to build a distributed archival system, having failed to create one of my own (a few times). Happily, this was around the dawn of “NoSQL”, as we no longer call it, and I found CouchDB. This database made so many good decisions that I was able to overlook that it was written in some arcane language I’d never heard of. I had already come to value the crash-only mentality that CouchDB enforces (tragically, by building a complicated system with elaborate and brittle shutdown code) as well as the append-only nature of disk writes and the all-in-one file format. The degree to which CouchDB’s API anticipates clustering was considerable too (albeit incomplete).
I could use this to build what I needed. To cut a long and winding story short, I didn’t. I joined Cloudant instead, who already had.
Before that, however, I had become an active CouchDB contributor and, a little later, a committer and community member. Further, I felt there was a gap that I was well-positioned to fill; the ability to index CouchDB databases with Lucene. This project, with the remarkably bland name, is, of course, couchdb-lucene.
The first question to answer about Search 2.0 is “What was so wrong with Search 1.0?”. I could write a lot about this but much of it would be pedantry (not yet an Olympic sport, so my trophy shelf remains empty) but two important points remain. Firstly, the code base was large (relative to the feature set) and the original author had left. Secondly, and much more importantly, was the choice of on-disk format for the indexes.
For those familiar with Lucene, it suffices to say that Search 1.0 does not use the familiar index format you know. For those that don’t, or wouldn’t object to a reminder, Lucene’s on disk format is something splendid. It’s highly optimized for full-text search access patterns where you necessarily have to examine every document you have looking for the top-scoring matches. Specifically, it’s an inverted index where the keys are words (“cat”, for example) followed by a contiguous, ordered list of document id’s that contain that word (and where they contain it). A separate index enables you to efficiently find the index for a given word. There are many more things in there than I’ve said, and more optimizations, but this is the core of it.
Search 1.0 used the Lucene library for analysis but used clustered CouchDB views for the index. The idea, of course, being to reuse all of Cloudant’s considerable infrastructure around managing secondary indexes. It was anticipated that the less optimized data structure of the view would be compensated by the parallelism introduced from sharding.
This worked adequately but never spectacularly, and we wanted to do better.
Search 1.0 is written in Java and uses Jinterface to enable interprocess communication between the JVM and our Erlang nodes running everything else. Java is a verbose language in general, and profligately so from the Erlang perspective. Mindful that my coworkers have no Java history, I investigated other JVM languages for a better fit.
I didn’t choose Scala but I did choose Scalang (which happens to be written in Scala). Scalang provides a natural interaction with Erlang, notably because it follows the actor model for concurrency. Of course it was necessary to learn Scala in order to develop against Scalang and, indeed, to make some substantial contributions to the Scalang project (process monitoring being the main one).
Search 2.0 isn’t based on Apache Solr or Elastic Search. Why not? Well, Cloudant already has production quality solutions to the clustering problems we face; sharding, fail-over, read repair, and the like. Search 1.0 was right to use that infrastructure.
This seems a good time to note that I don’t call this “Search 2.0” at all. What we’re branding as Search 2.0 is, internally, two components, Dreyfus and Clouseau. Dreyfus is about 800 lines of Erlang and deals with plugging into existing Cloudant technologies like Chttpd, Fabric and Mem3. Clouseau is about 1000 lines of Scala which provides access to the Lucene library via Scalang. For (a not entirely fair) comparison, Search 1.0 was a little under 7000 lines of Java.
Lucene is an excellent library and to use it well is to largely stay out of its way and not be stupid. Lucene 3.6, the latest at the time, is a much more refined release than when I started, back in the 1.4 days. Search 2.0 deliberately exposes only a part of the Lucene feature set so that we can make the core bullet-proof but there are quite a few exciting things I have my eye on, which will have to wait for a future blog post.
Readers familiar with couchdb-lucene will know that it uses Rhino for its javascript processing. Search 2.0 does not, it uses the same SpiderMonkey as the rest of Cloudant. This ensures we evaluate Javascript the same way in views, update handlers, and index functions and leverages our tools for managing and monitoring couchjs processes. Those readers will also notice that the Search 2.0 API is not the same as couchdb-lucene’s. I took the opportunity to design a simpler API, I hope you like it (Some details here). There are some planned enhancements to that API that will be deployed when the features they enable are ready for production.
Today, Search 2.0 provides a rich query language into your clustered indexes, including a minor tweak to the standard Lucene query syntax, namely that a range query where both ends look like numbers are transformed into a numerical range query. So queries for price ranges, or latitude/longitude ranges will work as you’d expect. Simon Metson kindly produced a nice demo of this here.
In addition to rich searching is a feature I’ve called “Bookmarks” for reasons I hope become apparent. We’re all familiar with the notion of “pagination” if only because of the Gooooooogle links we never click. In Search 2.0, every search result returns a “bookmark” value in addition to the number of matches we found and the first “page” of matches. Passing this value back in will return results on the next page. There’s a caveat, though. It’s quite possible that, between two searches, a new document (or an update to an existing document, or removal of a document) will alter the scoring such that you will see results from an earlier page.
We will be posting a full description of the API and all the features in the For Developers of our website.
Cloudant isn’t like other databases. So it goes that the content we present to developers in our newly minted For Developers section isn’t like other tech documentation; i.e. it’s not static. More on that in a minute.
First, a bit about the world we live in…
Cloudant is a hosted, distributed database service. Apps write to and read from Cloudant using our RESTful API. The payload is lightweight JSON. For all of that, you may be thinking it’d take a lot time to get Cloudant set up in order to play with it, and kick the tires. Nope. All it takes is the five or so seconds to fill out the sign up form.
And so…
More than just docs, the For Developers section is a live demo of the data layer at work. Click a couple buttons to CRUD, get JSON via secondary indexes and search the sample database that you replicated into your account. It’s a breeze.
Technical references (including handy curl commands) run inline with each of the demos.
As a newb noob N00B who is developing his first application on Cloudant, it is no small comfort to me to play with live demonstrations alongside the reference material in here. Not only are the docs in-step with the product (whew), but also it’s clear right away that the thing just works. And fast.
Last night, I had the Olympics on in the background as I worked on my first Cloudant-driven app. When I picked up my head to look around, prime-time coverage was long over. Kayaking was on. It was 2am.
The hours that disappeared on me weren’t spent banging my simple brain against complex SQL statements, and writing and rewriting blocks of server-side code. The time was spent playing around with the data coming from my Cloudant database, moving it around the interface I’m working on. Which is a long way of saying that time flies when you’re having fun.
I went from setting up my database to playing with the interface in minutes. Here’s how I got there.
Step 1 I put my data (JSON) into Cloudant with a bulk insert.
Step 2 I built my search index inside my Cloudant design doc (the same place I’ll keep my secondary indexes).
{
"_id": "_design/people",
"_rev": "13-24d8a5edfd54d61dacb0bc4230eee28a",
"language": "javascript",
"indexes": {
"searchByName": {
"index": "function(doc) {
if(doc.docType=='person') {
index('default', doc.last);index('default', doc.first);
}
}"
}
}
}
(My search index, searchByName, indexes the values of the first and last names of the people in my database so I can search for them by either of those or both in my application.)
Step 3 A little JavaScript to deliver it to the interface.
// Dependencies: JQuery and sag.js (http://www.saggingcouch.com/)
$(function() {
// since I want to work locally (but use Cloudant data), I've set up a reverse proxy
// more on reverse proxy: http://wiki.apache.org/couchdb/Apache_As_a_Reverse_Proxy
// use sag.js to connect
var user_db = 'my-Cloudant-database-name';
var couch = sag.server('my-server'); // localhost
couch.setDatabase(user_db);
// capture the search form submission
$('form.search').submit(function(e){
// kill default action
e.preventDefault();
// get the value of the search input
var query = $('#last').val();
// get data via sag.js
couch.get({
// query my Cloudant search index (see above) and account for wildcards (partial string searches)
url: '/_design/people/_search/searchByName?include_docs=true&q=' + query + '*',
callback: function(resp, succ) {
// vars for iterating over the results
var rows = '', i;
// create the rows for my table
for(i in resp.body.rows) {
rows += '<tr><td>';
rows += '<a href="#" class="view-detail data-cloudant-id="' + resp.body.rows[i].id + '">';
rows += resp.body.rows[i].doc.first + ' ' + resp.body.rows[i].doc.last;
rows += '</a></td></tr>';
}
// drop the rows into a table and append it to the results container in my interface
$('#results').html('<table id="people" class="table table-striped"><tbody>' + rows + ' </tbody></table>');
}
});
});
});
The launch of the For Developers section is a small but important step for us. A first step toward a full-featured library of useful content for people like me and, we hope, people like you. To that end, if you’re a developer using Cloudant and have ideas about docs or demos you’d like to see, please reach out. We’d love to hear from you.
Simon Metson started working for Cloudant in March 2012 as an “ecology engineer” and became a full time employee in July. He is based in the UK.
Before I joined Cloudant, I worked on the Compact Muon Solenoid (CMS), one of the two Large Hadron Collider (LHC) detectors that announced their discovery of the Higgs Boson earlier this month. I was a member of this collaboration for 10 years, from the start of my PhD through a bunch of post doc positions (all based in the UK at Bristol Uni), so was really excited and pleased to see the result being announced. It is the result of an amazing endeavor, where everything (hardware, software, wetware) has been pushed beyond known limits and, more importantly, delivered.
Candidate Higgs event - copyright CERN/CMS See the original
For most of my time with CMS, I was involved in one way or another with data and workflow management (DMWM); designing and delivering software that would be able to process, catalog and transfer the tens of petabytes of data that the detector would deliver each year, and for the last four and a half years I ran the DMWM project with a colleague from Fermi Lab.
CMS has a tiered, distributed computing system. Initial processing happens in the Tier 0 (at CERN). The resulting data is shipped to 7 national labs (Tier 1’) located in US, EU and Asia for second pass processing and offsite tape backup. Once on tape the data is made available to the collaboration; any CMS collaborator can request the transfer of any dataset to their local Tier 2 cluster. Last time I check we’d moved over 100PB of data around and had about 50PB of data active in the system.
CMS 2011 transfer link map - copyright Paul Rossman See the original
The DMWM project writes the software that manages these transfers, records and manages the metadata for all the datasets and submits and tracks all the processing jobs through the various batch and grid systems for both organized central processing and more ad hoc user analysis.
Watching how the computing system of CMS evolved over the last ten years from something that required a lot of hand holding to something that could be relied upon to deliver, often well beyond specification, was a great experience; seeing the sites, organization, software and people mature has had a profound effect on how I think about the design, implementation and management of software projects.
The nature of these experiments is such that any and all tools are available for use. We have a complex problem to solve, which results in a complex solution. While we rationalized a lot of things (making sensible decisions can save you thousands of lines of code) we still ended up with quite a range of supported technologies. Our stack used (from memory and in no particular order):
Wrangling that stack sure was entertaining…
I think CMS was the first experiment to start using NoSQL tools in production; HDFS is used at some of the T2 sites for their storage system, MongoDB is used as a smart cache aggregating data from various data services and CouchDB is used for various state machine and monitoring tasks. CouchDB’s replication feature was especially useful in CMS’s distributed computing system. Being able to quickly build aggregated views of monitoring or state information was great, as was having everything served via a REST HTTP API. Having it all in one application was even better!
I met Mike Miller at CERN in 2007 (well, in a fondue restaurant, I think), had followed Cloudant from its inception and we’d kept in touch after he left CMS to go look at neutrinos and dark matter at University of Washington, build a company and a set of desks. He contacted me at the end of 2011 to see if I’d be interested in joining Cloudant.
Like a number of my former colleagues, I felt that it was time to move on. As Mike once said to me “we’re builders”, and I wanted to be building new things again. I’d experienced working in a large (CMS has about 4000 people involved, and is spread over nearly 120 institutions), collaborative environment with all the benefits and issues that they bring and decided that seeing what life is like in a startup would be interesting.
Am I sad that I left CMS before the discovery announcement? Not really. The nature of these collaborations is that everyone who makes a contribution is recognized; the papers are signed by “the CMS collaboration” and I am a member for the next two years. While I wasn’t directly involved in the analyses that were presented on the 4th, the software that I was responsible for enabled that research. That’s something I’m quietly proud of.
What’s great about joining Cloudant now is that I get to see that whole evolution and scale up cycle again. Everyone in the team knows that we’re just at the beginning of a very interesting journey and is really excited to see where the company goes over the coming years. I’m really looking forward to helping the company scale up; the software, the organization and, importantly, the customers.
As much as we’d like to say Cloudant athletes will be lighting up the 2012 London Olympic track, playing fields, or pool…instead, Cloudant technology will be used to help light up…the Ferris wheel.
Tonight at 9PM local time in London, Energy of the Nation, the world’s first social-media-driven light show will launch at the EDF Energy London Eye. The nightly light show, and accompanying realtime Energy of the Nation infographic and “Energy2012” mobile app will measure the mood of the UK during the Olympics and Paralympics by performing sentiment analysis on Olympics-related tweets sourced from the UK’s 10 million Twitter users.
(source: Sosolimited.com)
The campaign was conceived and developed by the creative geniuses down the street from us in Boston, Sosolimited in conjunction with UK-based brand experience consultancy, Ignite. The project was sponsored by EDF Energy, which is a sponsor of both the London Eye and the Olympics.
Sosolimited has written software that captures Olympics- and Paralympics-related tweets in the UK. It has been capturing tweets since May 1st of this year. The tweets, and a vast library of emotional words, punctuation and emoticons, are stored using the Cloudant Data Layer service (hosted on SoftLayer cloud infrastructure in Amsterdam).
The Sosolimited software runs sentiment analysis algorithms on the Tweets in Cloudant to determine their emotional content. The analysis output reflects the rising and falling mood of the nation with respect to the games and is charted throughout the day on a large LED wall next to the London Eye. Every night at 9pm, that data is to be played back in a spectacular, 20-minute, low-carbon light show featuring full-color architectural lighting fixtures around the Eye and large, ground-based search beams, all powered by EDF Energy. There’s more good info on how this all works located on the EDF Energy website.
We’re psyched for the Olympic games to begin, and even more psyched that Sosolimited chose Cloudant to play a role in the event. If you haven’t seen Sosolimited’s catalog of interactive and multisensory design work, check it out!
A fairly common question we get asked is “how do I manage my view code in insert your favorite version control system here?”. As people develop more complicated apps this question becomes increasingly important. Fortunately there are a set of tools that make transferring code from a version controlled directory into your database easy.
At the heart of Cloudant is Apache CouchDB. Applications that run out of CouchDB are known as “CouchApps”. At their most basic these applications are nothing more than a design document containing a view. There are a raft of tools that simplify the development of these applications that are served directly out of CouchDB. They each target slightly different use cases, from a minimal wrapper to upload code into the database to a fully featured development platform.
At the most basic level these tools enable developers to version control their views and other code, and “compile” and push them into the server. More advanced use enables development of “two-tier” applications, pulling in various libraries to build complex, dynamic applications; even ones available offline.
Below is a quick summary of the various tools that let you build and push code from your local system into the database. It’s by no means exhaustive, and I’d love any corrections/clarifications/additions.
CouchDB has supported running applications out of the database since 2009, and the associated tools are in a reasonable state of maturity. It’s nice to see other databases getting similar functionality now; it can only be a benefit to developers creating two-tier, dynamic applications.
For very simple tasks (e.g. pushing a view into the database) there isn’t much to differentiate between the various tools; it’s a very subjective choice. As you go “further down the rabbit hole” you may find one tool fits your application or workflow better than another.
Little known fact: about 20% of Cloudant Data Layer users are PHP developers. So, last week we were excited to announce that Cloudant is now available as an add-on at cloudControl PHP hosting platform, a cloud application platform for PHP developers (with Ruby and Python support coming soon).
The announcement follows several weeks of integration work and testing on both sides of the partnership, and we hope PHP developers can develop faster and scale their apps and databases easier as a result of the effort.
cloudControl account holders can add Cloudant to their service via the Cloudant Add-On page at cloudControl: https://www.cloudcontrol.com/add-ons/cloudant
Cloudant featured in a book on CouchDB & PHP Development Coincidentally, we also got a note from author Tim Juravich (@timjuravich) last week letting us know he had just published a book on developing PHP web applications on CouchDB. In the book, he walks developers through the creation and deployment of a simple social networking app using tools like PHP, CouchDB, Sag, Bootstrap, jQuery, PHPFog, and Cloudant! You can get the book here.
Cloudant’s growing, and we recently opened offices in Seattle and the UK (in addition to our headquarters in Boston, near Cloudant’s MIT birthplace). As a YCombinator company, Cloudant is conditioned to run lean, but we also have an exceptionally technical team with an appreciation for good design. This made finding and furnishing our new office particularly challenging. Here’s the story of how we did it in Seattle, aka “Cloudant West”.
In his seminal 2005 article “How to Start a Startup”, Paul Graham writes, “for a startup location is very important.” We couldn’t agree more, and it’s why we surveyed five different neighborhoods in Seattle: Downtown, Pioneer Square, Capital Hill, Fremont/Ballard, and South Lake Union. Our criteria were loosely based on:
Environment concerns quickly narrowed down the list. Downtown is rather soulless during and after business hours. Fremont/Ballard and Capital Hill are amazing places to live and hang out, but they don’t have the necessary concentration of other companies. I wanted to love South Lake Union, primarily because it’s close to the lake and Amazon and biotech are absolutely transforming the neighborhood. Everything is shiny, new, and most intersections have their own cop to direct traffic (AMZ makes things happen). Unfortunately, aside from Techstars it doesn’t have many companies in our space. The kicker was the small selection of food and the surprisingly high costs. That left us with Pioneer Square, which luckily maps well onto space, cost, connectivity and commute. There are a multitude of lofts with just enough brick, priced well, and close to the all-important fiber. To boot, we have several customers around the corner, Future Advisor (YC10) across the street, easy access from three directions, and are right next to Century Link and Safeco fields.
Next step – find a good space.
Ben Anderson took the lead at scouting locations. I should say that commercial real estate is due for disruption. The fact that agents still control the listings and cost is absurd and creates an inefficient market. That said, we followed advice from several advisors and were introduced to a trusted agent that put plenty of good properties in front of us. We found it challenging to envision how we would lay out an office, so we called in the experts – my fiancee LJ. I delivered a blueprint to LJ and before I knew it she gave us a Google Sketchup model for layout and a video to send back east for the final sign-off. Here’s the video from the winning space; thanks LJ.
We decided that we could nicely lay this space out for 10 people without feeling crowded. We could get desks (more on that below) that would be easily re-arrangeable, and we could have space for sales and bizdev, meetings, open discussions, and door-closed, heads-down coding.
We had quite a bit of fun making these desks. We got help from a longtime friend and advisor of Cloudant Ross Rosen and YCombinator's own Tony Wright (seen pulling quite a face below left). Tony, always the designer, augmented LJ's initial tapered leg design by rounding the corners (trying his best to make the table look like a big beveled "call to action" button). We ended up with a stack of gorgeously rounded legs, but they were a bit tricky to hold together, drill, and screw together without someone holding them in place with inventive yoga moves.
These desks are huge; you can comfortably fit 4 laptops if you use both sides, but we give one to each employee. It’s highly conducive to teamwork, as it’s easy to pull up a chair and work with somebody to solve a problem, review a design or test an interface. The rest of the office we furnished primarily from Costco business, and we’ve been particularly happy with the ProGrid chairs (highly adjustable, robust, comfortable) for under a third of their list value.
The OSGeo Code Sprint in Bolsena/Italy was great. Many interesting people sitting the whole day in front of their laptops surrounded by a beautiful scenery. This year I spent most of my time on GeoCouch, CouchDB, Couchbase, deegree, Imposm and GeoNetwork.
Already on the first hacking day we had a result, a Couchbase backend for the deegree blob storage. This means that you can now store your rich features (e.g. from INSPIRE) in Couchbase and serve them up as a WFS with deegree. In case you wonder what rich features are, it's the new, shiny and more descriptive name for complex features.
In the following days I worked together with Oliver Tonnhofer on a CouchDB/GeoCouch backend for Imposm. You are now able to store your OpenStreetMap data in CouchDB and make queries on it through GeoCouch. I've created a small demo that displays the some data import from Andorra with directly with MapQuery, without the need of any additional server/service. The CouchDB backend should be easily adaptable to Couchbase, if you want to help, let me know.
I've then spent some time on the GeoNetwork project and helped translating the language file to German. I cleaned it up a bit and fixed major wrong translations. It's not perfect yet, as I've only spent little time on it, but at least it should be way more understandable (and sadly less funny) than the original version which was generated by Google Translate.
When it was time for presentations, I give a quick overview over the Couch* ecosystem. From CouchDB to GeoCouch, BigCouch, PouchDB, TouchDB (TouchDB-iOS, TouchDB-Android), Couchbase Syncpoint and Couchbase. You can find the slides as PDF here.
On the last day I've spent my time on polishing GeoCouch a bit and getting it ready for the Couchhack in Vienna. I've backported all changes from Couchbase to the CouchDB 1.2.x branch and also ported the geometry search into an experimental branch. You can now search your GeoCouch with any Geometry GEOS supports.
The event was fun as always and I also get to know some new people (hello B3Partners guys). Thanks Jeroen from GeoCat for organizing it, and thanks all other hackers that made it such a awesome event. Hope to see you all next year!
We are very excited to announce that the Cloudant Data Layer now runs on Azure. As a result, .NET, Node.js, PHP, Java, and Python developers can now enhance their fast-growing Azure apps with a scalable data layer that is co-located with their application servers in Azure data centers.
It’s great to see Windows Azure strengthen support for open source development technologies and hybrid clouds. The addition of Linux-based virtual machines made it possible for us to offer the Cloudant Data Layer service on Azure
We’re psyched to be active members of the Azure community now…the response to our announcement last week has been exciting. For those of you who are new to Cloudant, we offer a NOSQL database service (with an Apache CouchDB-compatible API.) A couple of things set Cloudant apart from other SQL and NOSQL database options available to Azure developers:
Cloudant is a service: we provision your data layer and then keep it running non-stop and scale it as you grow. Unlike scale-it-yourself DBs that you have to install and manage in the cloud, Cloudant’s back-end DB operations (provisioning, backup, scaling, re-partitioning, etc.) are taken care of for you by us, the experts.
Global data distribution: we push your data to the edge–to Azure data centers around the world–so it’s closer to users for low-latency data access.
Data Replication & Sync: enable offline data access & collection (especially to mobile devices) or use it to replicate data from the cloud to on-premise data centers, so you are never locked into Cloudant or Azure.
Cloudant on Azure is still in “preview” mode as we continue to tune it for performance and expand it to additional Azure data centers in North America, Europe, and Asia. Please let us know how it is working for you and what we can do to improve it.
To get a Cloudant Data Layer account running on Azure, please visit our signup page and choose “Lagoon” as your data layer location.
If you already have a Cloudant account and you want to move it to Azure, just follow these directions:
We just added Joyent Cloud to the list of Cloudant Data Layer locations.
Joyent Cloud is a high-performance environment for running apps and data layer services in the cloud. Backed by Joyent SmartOS - an operating system optimized for multi-tenant cloud environments - and the ZFS file system, Joyent Cloud enables Cloudant to take advantage of specific caching and data backup features in the ZFS file system and deliver outstanding data layer performance and reliability Aside from Cloudant, Joyent Cloud also powers a growing number of cloud platform and software services providers including Nodejitsu, StackMob, and GameSalad.
Create a Free Cloudant Data Layer running on Joyent Cloud
Getting started is easy. Just create a free Cloudant account, and select the Joyent Cloud option “Paloma” from the list of locations, as shown below.
Or Move Your Data Layer to Joyent Cloud
If you already have a Cloudant account and you want to move it to Joyent Cloud, follow these instructions:
April 12 is Iris Couch’s one year birthday. Today, we offer paid cloud CouchDB hosting plans.
Our pricing plan has the details. It is relaxing and RESTful. Your log file is your receipt.
This month is completely free for everybody. May is the first month under the new policy.
Below, co-founders Jeff and Jason reflect on our milestone.
I’m really proud of how dialed-in this service has become with Jason’s expertise. I’m excited about this launch and new features we have in development. Mostly, though, I’m really looking forward to not feeling so sad every month signing huge checks to pay our hosting bills.
We spent a lot of time mulling over how to price this so that beginners and tinkerers can still have a free sandbox, and I think what we’ve come up will protect that. Thus Iris Couch will remain both a platform and a tool to spread the word about how relaxing CouchDB is.
I am pleased with our first year. We have built technology I am proud of. Many people (Jeff in particular) have asked:
Why the delay for paid stuff? Aren't you hungry? Are you lazy?
We are both hungry and diligent. But despite being an Apache CouchDB developer, I am a sysadmin first. Before I take your money to steward your data, I have an ethical duty to produce a proven, reliable service.
Is it perfect? No, it is better than “perfect.” It is very good. We built something befitting a proper database company.
Cloudant is hosting the CouchDB Summit this week in Boston and we will be capping off the two-day affair with a public drinkup on Tuesday night. We invite everyone to come visit with the comitters and join us in wrapping up the summit. Just look for the folks in good lookin’ Cloudant swag. Details:
Just now, the Apache CouchDB project released version 1.2.0.
I am happy to announce that all Iris Couch accounts are now running this release. You needn’t do anything, the upgrade is automatic. Just relax on the Couch!
Notable improvements:
_users databaseWe also upgraded to the new Ocasta Labs Facebook Authentication plugin. It is now easy and relaxing to build a Couch app letting users log in via Facebook.
Please email us if you have questions or issues, or visit our support forum. Thanks!
Every Monday morning the Cloudant engineering team jumps into a google hangout for our weekly “train” standup 1 where we swap stories from the weekend, I defend my hometown teams (MSU and Detroit), and decide what code to include in the next release – tomorrow. Yep, that’s right, we push new code every Tuesday (at the slowest 2.) This may not seem astounding to some, but we power applications for nearly 10,000 users and have an impressive amount of transactional data under management, spread globally across 11 data centers.
In my past life I once labored for literally more than a year to get a new release cut, tested and pushed. In that light, I find our weekly/daily release rhythm pretty amazing. More importantly, I think it’s an example of Jay Krep’s compelling argument that, when it comes to distributed systems, the true value is not solely, or even primarly, in the underlying software, but in the service provided by those that run it. That ethos lies at the very heart of all things Cloudant. It drives our technology choices, product vision, roadmap, release schedule, operations, support, and (perhaps most importantly) business model. We write software to scale your data and we run it. Below I briefly expand on the tools we use to constantly push new code, and why that is important to our customers.
Erlang. Building our core stack as a collection of Erlang OTP applications and using Chef for configuration management means that we upgrade our running database applications without ever taking them offline or even dropping active connections. If you haven’t ever witnessed the magic of hot code upgrades in Erlang, you haven’t lived. New and old code live side by side, processes are suspended, a new code path kicks in, state transformations are made on the fly, and voila, you’re running new code without ever skipping a beat. Don’t like it? Hit the roll-back button.
Github + CI + Tests. Hot code upgrades would be worthless if we didn’t have good code to push. Github’s tools have really matured over the last few years and it empowers a very efficient workflow (pull requests, code review, etc) for a distributed team like ours. Coupled with Jenkins for builds and automated testing (etap, etc) we can efficiently go from idea to tested implementation.
Cloud. This is the promise of Cloud – Isolating non-core elements of your technical stack and finding the best software and people to deliver them faster, cheaper, and more reliably you can do yourself. Making a quantitative ROI argument has always been challenging because it misses a key point, “it’s not just the software, but who runs it.”
Response. On stage in 2009, one of our first customers (who was piping the twitter firehose into Cloudant) referred to our software as young and developing. Ouch. But he followed up by calling us some of the best problem solvers he had encountered. The distributed systems used to handle Big Data problems are relatively new. Our team has decades of experience in big science and enterprise IT, building and operating some of the world’s biggest computing systems, and still we are able to find novel failure modes. Our ability to handle these failures gracefully stems from our deep working knowledge of the code and it’s operational characteristics.
Priorities. There’s little worse than investing substantial time and money into a feature/product that is irrelevant in the market. Thus, while we have a clear product roadmap and development plan, we take the just-in-time ethos to heart. “Somebody is trying to create 100 million databases (true story). We should probably re-write that LRU module.”
Since we began working on Cloudant we have seen some major public failures of distributed big data systems. In nearly all those cases, the failure was not one of software but of operations. This seems like a natural consequence of putting up a wall between those that build code and those that run it. At Cloudant, that wall doesn’t exist, and this is what allows our customers to grow into their data layer, not out of it.
On March 8th we hosted a webinar featuring presentations from Meteor Solutions and SoftLayer. Ben Straley, CEO of Meteor Solutions, gave a presentation on how his company developed their social audience marketing platform on the Cloudant data layer (powered by Softlayer). Afterward, Nathan Day, Chief Scientist at SoftLayer, elaborated on SoftLayer’s inner workings and high-performance network, which allows it to provide a global platform for web applications. Kicking things off I described some of the pertinent technology behind and features of the Cloudant Data Layer, and how, together with SoftLayer, we helped Meteor Solutions keep up with their ever growing data volumes and customer demands.
For those that missed it (or just want to view it again), we recorded the event, which you can view below. If you are interested in a copy of the slides, you will find those at the link below the video.
Download the slides here
We are happy to announce that Andy Palmer, serial entrepreneur and Boston startup luminary, is joining the Cloudant Board of Directors. Andy has a remarkable track record of starting and building amazing companies in Boston. His experience with and deep knowledge of Big Data systems (which he recently wrote about for GigaOm) will be invaluable as we build out the Cloudant Data Layer. Having previous worked with Andy, I know first hand how committed he is to building great companies in New England. His energy and passion are second to none and we at Cloudant are privileged to have him join our board.
For more information about Andy and Cloudant see our press page here
We are happy to announce today that Cloudant is hosting CouchDB Summit: Boston on April 9th and 10th. It promises to be a great, productive time and we are excited to introduce everyone to The @Cloudanator in person!
Here is the announcement e-mail that was sent to the Apache CouchDB user and developer mailing lists today.
Hello community,
For a few months now we at Cloudant have been talking about hosting a CouchDB Summit in Boston for the Apache CouchDB committers and some community members. This got pushed to the front burner due to recent events in the community, chief among them the purchase of a kegerator.
The goal is to give the CouchDB community the chance to be in the same room to discuss the direction of CouchDB over a two day period. Some admirable goals could include what CouchDB 2.0 should look like, how to get there, talking about auth, etc. Plus plenty of hacking time.
The results of the discussions will be posted to the community for decision making. Think RFC.
This is an invite only event with a max capacity of 20 to 30 people. We want to make sure that there is progress and flooding a room would inhibit that. The Apache CouchDB Committers are of course invited, plus some community members.
Apache CouchDB Commiters, please let me know if you would need financial assistance to attend. Cloudant is open to providing financial assistance for flights.
The particulars…
When: April 9 & 10
Where: Boston, MA, USA (venue to be announced)
$$$?: Cloudant is NOT charging for the event
Party?: Yes, there will be a PUBLIC drinkup for the community. Details to be posted separately.
RSVP: Invites are going out this and next week.
Let me know if you have any questions or concerns. We hope to make this a fun, positive experience for the community.
Cheers,
Sam Bisbee
Congratulations and a hearty round of applause are in order. Noah Slater, speaking on behalf of the Apache CouchDB Project Management Committee, recently announced that our very own Paul Davis has been appointed to that very PMC. According to Noah’s email to the CouchDB user and dev lists:
Paul Joseph Davis, a long time contributor to the CouchDB project, has been appointed to the PMC. He’s an asset to this community, and we’re delighted to have him on board. Please take a moment to congratulate him and show your appreciation for all his hard work so far.
Paul is an outstanding engineer and a pillar of the CouchDB community. We consider ourselves extremely lucky to have lured him to Cloudant last fall. He continues to be a staggering force within Cloudant, working on all aspects of our product, and still has the energy to contribute heavily to CouchDB. It’s a pleasure to see him rewarded for his efforts.
I have no doubt that he will make a fantastic member of the CouchDB PMC. With Paul helping to steer the project, and with version 2.0 coming up this year, I believe CouchDB has a very bright 2012 ahead.
I have a confession to make. We have been neglecting our friends in Europe. More than a year ago we launched our multi-tenant data layer with clusters on the east and west coasts of the US. Since then we have added shared clusters in Singapore and Japan and private clusters in numerous sites around the world. But, despite the requests from developers in Berlin, Amsterdam, Dublin, and more, we resisted opening up a shared cluster in Europe.
This was folly and it ends today. We are happy to announce the immediate availability of our brand-spanking-new EU-based cluster, “Jenever.” Jenever lives in SoftLayer’s new data center in Amsterdam just a few short milliseconds round trip from almost anywhere in the EU and UK. No longer will our European developer friends be forced to send requests or users across the Atlantic to hit the DB.
BUT WAIT THERE’S MORE! You can now choose the location of your data when you sign up or at any time. We maintain four shared clusters around the world, “Julep” in Northern Virginia, “Meritage” in San Jose, “Sling” in Singapore, and “Jenever” in in Amsterdam. When you sign up for a new account you will be asked where you want your data to live. If you ever want to change clusters, you can switch via your ‘accounts’ page. Note that switching clusters happens seamlessly but asynchronously. It takes some time to move an account so don’t expect this to happen instantly.
We at Cloudant believe that your data should live where your application lives, and your application should live where your users live. Our goal is for the Cloudant data layer to cover the globe, providing developers all around the world low-latency access to their data. With Jenever in Amsterdam, we take one step closer to this goal.
It is a big day here at Cloudant HQ; we are announcing the release of BigCouch 0.4! This release, which brings BigCouch into API equivalence with Apache CouchDB 1.1.1, has been baking for a while, and we are excited that it is now ready for public consumption. Instructions for installing and using BigCouch 0.4 can be found on the BigCouch page. Users running Debian Squeeze, Ubuntu (LTS or newer) or RedHat / CentOS / Amazon Linux are welcome and encouraged to use our prebuilt distributions based on Erlang/OTP R14B01 and SpiderMonkey 1.8.5.
There are a whole slew of new features and performance improvements in this release. You can read the entire list here, but I wanted to highlight a few of our favorite features. These should convince you to upgrade if you haven’t already.
For my money, the killer addition to BigCouch 0.4 is support for ‘zones,’ which allows you control over where you place the individual copies of your data. Cloudant’s hosted service makes extensive use of this feature to protect against isolated failures in individual data centers. As an example, our west coast cluster “Meritage” is actually spread over two different data centers in two distinct geographic regions. We use zone-support to ensure that at least one copy of all of our customers’ data exists in both locations. That way, if one location falls into the ocean or is subject to nuclear attack, our customers’ data is still safe and available. With this feature now in BigCouch, you can extend clusters across multiple locations.
Another important improvement we added, taken from Apache CouchDB 1.1, is support for the replicator DB to help manage multiple replications. The BigCouch replicator DB works the same as the Apache CouchDB version. A nice writeup of usage can be found here.
We’d like to thank everyone in the CouchDB community that has contributed bug reports, patches, feature requests, etc. for BigCouch and to all the brave souls willing to try out the bleeding-edge releases. This release would not be nearly the upgrade it is without all of you. Special shout out goes to Benoit for his help with FreeBSD and native SSL support.
Finally, for those following along with recent events in the land of CouchDB, we’re dedicated to donating and integrating BigCouch to Apache CouchDB. This sort of integration won’t happen overnight, so we are still planning on making releases to BigCouch while we iron out the divergences in the two projects. We very much appreciate the support and enthusiasm we have recieved from the CouchDB community to date, and we look forward to working with that community to strengthen Apache CouchDB.
Cloudant is happy to be sponsoring Node Summit next week in San Francisco. To keep the party going an extra day we will be hosting a CouchDB drinkup Thursday night to touch base with our west coast crew. This is instead of Benoit and Randall's meetup which they kindly merged with us - community powers unite!
Just look for the folks in good lookin' Cloudant swag.
Damien Katz’s blog post, The Future of CouchDB deserves a response.
Damien tackles difficult problems, thinking clearly and executing well. If he is focusing on Couchbase, then Couchbase will be outstanding.
Both CouchDB and Couchbase are domain-specific databases. Moreover, these domains intersect minimally. The projects represent non-overlapping magisteria; and one might say instead that the future of awesome datatabases is Couchbase.
We are interested in CouchDB’s domain, web applications modeled after the web itself: standard, decentralized, and skewed toward empowering regular people.
… is not Damien Katz.
The blog post Damien Katz wrote earlier today, doesn't mean much or anything for the Apache CouchDB project (or memcache project for that matter). If anything it's a public note that Damien Katz acknowledged that he moved (on) from CouchDB to Couchbase.
I'm not a contributor to CouchDB by means of code, (but) I blog a lot, I maintain the FreeBSD port, wrote a book and have an opinion on many things CouchDB. I've been a CouchDB-user for something like four years (since pre-0.8 times) and a BigCouch-user (and Cloudant customer) of about 1.5-two years.
I am not sure what Damien Katz tried to achieve when he posted his message to the community and while I personally find it ignorant (to say the least), it worries me how it is perceived by the general public.
Of course it may sound like the end of the world when the creater of CouchDB quits his own project, but truth to be told, Damien Katz left CouchDB a long time ago. Couchbase moved past CouchDB long before they announced it. Basically when they started integrating Membase, though there are all kinds of notable contributions from Couchbase employees (e.g. Filipe and Jan).
Damien himself hasn't (actually) commited in over a year to CouchDB. Which makes his move no real surprise, just the way he decided to communicate surprised me. Especially since he said to have no regrets, I find the tone and statements in his blog post rather questionable. That is both from a personal and professional perspective.
I just attended a Couchbase event in Berlin last year where talks about CouchDB were given along with newer Couchbase developments. So personally, while I welcome clarity, all too sudden changes in strategy don't make me happy. If I was new to CouchDB and/or Couchbase, this would look like a headless chicken (excuse the image) and way too much drama to get into.
And on a professional level, Damien's posts invalidates the efforts of many people who both contribute and work with Apache CouchDB on a daily basis.
Then later today Damien shared this:
TIL, if you create an open source project, you should stick with it forever and ever. Family can live off unicorns and stardust. — Damien Katz
First off, the most people in this discussion (excluding HN, of course ;-)) are actually active Open Source contributors one way or the other. Many of us have other projects (plural) besides CouchDB. It's not that we troll about something we have no idea about.
Secondly, it's not the fact that Damien left, it's how he left.
No one blames people for moving on: it happens all the time. I do it all the time — write code, push it out, move on.
If code is good enough it'll be picked up, if not, it'll rott on Github for forever. It happened to other projects and it happened to CouchDB. But why would anyone pronounce a project dead where he is not anymore invested in?
Anyway: I wish Damien good luck in the future.
I wrote a blog post about the current state of CouchDB last year (2011) in early December.
A few things have changed since then:
Overall, I still see contributions from notable community members all around. Not sure if it's my own perception, but there has been more activity as of late. A new release of Apache CouchDB is just around the corner. Overall, good times for Apache CouchDB indeed.
The other thing that happened after my blog post is that Couchbase said in their 2011 review (which was published in late December) that it would officially step off Apache CouchDB and contribute documentation and OSX builds (aka CouchDBX or Couchbase Single Server) to the Apache CouchDB project.
This is great news and announcing that they will step off the turf is fine too since it clears up a couple misconceptions people may have about Apache CouchDB and the former Couchbase Single Server.
And all in, this makes Damien's blog post even more unnecessary and confusing to many users out there. Especially confusing for those who are not neck-deep in Apache CouchDB — and by that I mean: they are neither subscribed to a mailing list, take part on IRC, read the CouchDB planet or know any of the contributors directly.
In the end his blog post confuses people because it contains absolutely nothing but fear, uncertainty and doubt (FUD).
I can't reveal too much because it's not my business to announce anything — let me just say that there are good times ahead for Apache CouchDB.
I'd like to get past all the drama and re-focus on what is important: CouchDB and our data. I don't care for the rest, I want to see exciting things from Apache CouchDB in 2012.
by Till Klampaeckel (till@php.net) at January 09, 2012 10:51 PM
The CouchDB world is currently full of “The future of CouchDB” blog posts. It started with the blog post from Damien Katz the creator of CouchDB. Of course people were also concerned about the future of GeoCouch. No worries, it will be good.
The reactions were quite different. People who are not deeply involved with the CouchDB community think that this means the end of Apache CouchDB. My reaction was positive, I tweeted:
“It’s good to see the Damien is so open to [the] world”
The reason was, that for me it was pretty clear that it would happen, and I was just happy that Damien officially made the cut.
The reactions from CouchDB community members where pretty much what Till Klampäckel describes in his blog post. You could see it comming after Couchbase announced that they are not the CouchDB company and that their product won’t be Apache CouchDB compatible.
I agree with Till here, the way Damien wrote his blog post, isn’t the best imaginable. For outsiders, it really seems to be the end of Apache CouchDB, but it is not. For me it just shows, why foundations like the Apache Foundation are such a great idea. Even if the original creator leaves the project, it still lives on.
Apache CouchDB has a lot of contributers and the mailing lists and IRC channel is busy as always. That CouchDB has a future is also shown by the blog post from Cloudant. They will keep supporting Apache CouchDB.
After this quick recap what happened so far, it’s time to talk about the future of GeoCouch. As you may know, I work for Couchbase on the integration of spatial functionality into their product.
Currently the overlap between Apache CouchDB and the version Couchbase uses internally is still quite huge, but it will diverge more and more in the future. Thus it will get harder and harder to maintain a single version that supports Apache CouchDB and Couchbase.
The good news is, that GeoCouch is pretty much a data structure only. It's an R-tree that stores JSON documents. This can easily be used by CouchDB and Couchbase. Perhaps small wrappers will be needed, but those should be minimal.
The easiest way to understand how the future looks like is in a small illustration:
GeoCouch's core is the R-tree, it's the same code for CouchDB and Couchbase. On top of it there will be code that is specific to either CouchDB or Couchbase.
This means that the majority of the devlopment I do for Couchbase will also improve the GeoCouch you can use for CouchDB.
The future of all three, Apache CouchDB, Couchbase and GeoCouch looks bright.
CouchDB and it's users have been in a state of flux all year, with declarations of the project's death by outsiders and the rest of community scrambling to fall in-line behind their chosen implementation.
Let's relax and ignore the marching drums and nay-sayers for a moment. For me, CouchDB was never about a specific platform or Mongo vs. Couch, but about a more fundamental view of the world. CouchDB offered a refuge from the aggressive centralisation of the web. Its about decentralisation, smart clients and sharing data.
The cloud promised a new era of flexibility and reduced cost, but it's in opposition to one of the most important principles of the 'net. An open network without intermediaries, a network of decentralised services and truly public spaces. The cloud means centralisation. That may not chime with our delusion of the startup dream where anyone can change the world, but in reality Amazon and Facebook are not easily displaced. Centralisation risks control, these intermediaries are far easier to censor and lock-down than the edges of the network.
The most powerful feature CouchDB offers is a protocol for moving data around using HTTP. Master-master replication really shines in decentralised systems, and makes possible the idea of smart clients and services at the edge of the network, without completely throwing out all the benefits of the cloud. Or as Chris Anderson puts it 'Ground Computing'.
There are a number of implementations now working following this protocol, not just Apache CouchDB. There's PouchDB for the browser, the upcoming lightweight TouchDB from CouchBase and of course BigCouch from Cloudant. Rather than seeing this as fragmentation, I see it as a vibrant ecosystem of projects sharing the CouchDB protocol. That means more opportunities to share data between devices without locked down platforms sitting between them.
If replication addresses the difficulties of decentralised data, CouchApps address the difficulties of running web applications at the edges. CouchApps are a structured way of writing a data-powered application so it can be run in multiple places and on multiple platforms. Max Ogden and Mikeal are even exploring running them as Google Chrome extensions. I believe the combination of easy to install web apps and easy to sync data has the potential to unlock nascent demand for data ownership.
This is why I spend my time working on Kanso and CouchApps. The future of CouchDB is not Apache CouchDB, it's all of us working together. The fact that CouchBase Server does not follow the HTTP replication protocol and does not support CouchApps means it has stepped away from the rest of the community and failed to re-kindle the ideas that defined my earliest encounters of the internet. A massively decentralised network of plural participation.
Forget trying to be the next Mongo and get back to changing the world.
The future of CouchDB is Apache CouchDB.
Cloudant over the past 3 years has built and refined BigCouch, a fault-tolerant, horizontally scalable clustering framework purpose-built for CouchDB. Today we announce our intent to contribute this work back to the community. Working with the ASF and the CouchDB community, we hope to integrate the core capabilities of BigCouch into Apache CouchDB. Hopefully this will put to rest the tired (and false) “CouchDB doesn’t scale” meme. BigCouch forms the bedrock of a globally distributed, sophisticated technology stack that we’ve had in production operation for over two years at scale.
Those of you building applications with CouchDB know that its performance has been improving by leaps and bounds. We’ve achieved this through low-level implementations of performance-critical routines and smart refactoring of the Erlang codebase, but without sacrificing the rock-solid durability and stable REST API people have come to expect from CouchDB. The growing community of active committers includes passionate, stellar developers. More and more people are using CouchDB for projects, big and small, every day.
We, along with a host of other companies, strongly support the open source community in building CouchDB and we do not plan on stopping. We have been fortunate in our ability to attract outstanding engineers, investors, and customers. We intend to continue devoting resources to Apache CouchDB and offer our help in any way the community desires. The future of CouchDB is CouchDB.
Damien Katz has been an excellent founder and steward of CouchDB. It’s been an honor to work with him, and we wish him well in his new location and mission.
Update, 2011-12-21: Couchbase posted their review of 2011 (the other day) — TL;DR: Couchbase Single Server (their Apache CouchDB distribution) is discontinued and its documentation (and its buildtools) will be contributed to Apache CouchDB.
When Ubuntu1 dropped CouchDB two weeks ago, there were a couple things which annoy (present tense) me a lot. Add to that the general echo from various media outlets blogs which pronounced CouchDB dead and a general misconception how this situation or CouchDB in general is dealt with.
Some people said I am caremad about CouchDB and that is probably true. Let me try to work through these things without offending more people.
What annoy[ed,s] me about this situation is that I wrote a chapter about Ubuntu1 in my CouchDB book. And while I realize that as soon as a book is published the information is outdated, I also want to say that I could have used the space for another project.
I talked to a couple of people about CouchDB at Ubuntu1 on IRC and no one made it sound like they are having huge or for that matter any issues.
Of course I neither work for Canonical or Couchbase. I haven't signed any NDAs etc. — but looking back a week or two my well-educated guess is that not even the people at Couchbase knew there were fundamental issues with CouchDB and Ubuntu1.
The NDA-part is of course an assumption: don't quote me on it.
Scumbag Ubuntu1 drops CouchDB and doesn't say why. — myself on Twitter
First off: I'm not really sorry. I was abusing a meme and if you read my Twitter bio, you should not take things personal.
I also should have known better since it's not like I expect anything transparent from Canonical. (Just said it.)
When people are compelled to write a press release and put it out like that, they should expect a backlash. The reason why I reacted harsh is that Canonical didn't share any valuable information on why they discontinued using CouchDB except for: it doesn't scale.
And I'm not aware of anything concious to date.
Please take a look at the following email: https://lists.launchpad.net/u1db-discuss/msg00043.html
This email contains a lot of criticism. And it's all valid as well.
Other examples:
These are great emails because they contain extremely valuable feedback.
In my (humble) opinion, these kind of emails are exactly what is necessary in CouchDB-land, and many other open source projects: criticism and a little time to reflect on not so awesome features. And then moving on to make it better. If the feedback cycle doesn't happen, there's no development or evolution — just stagnation.
And in retrospect I wish more people would share their opinion on CouchDB and this situation more often. Since I'm personally invested in CouchDB, it's hard to say certain things. Honesty is sometimes brutal, but it's necessary.
In summary, a CouchDB user like Ubuntu1 (or Canonical) doesn't have the civic duty to give feedback, but to desert a project while pretending to be an Open Source vendor, and not talking to the community of the project or sharing your issues in public, that is extremely unhelpful.
Overall it strikes me that the only thing to date known about Canonical's collaboration with CouchDB is the support for OAuth in CouchDB. And most people don't even know about that (or wouldn't know how to use it). It worries me personally to not know the kind of problems Canonical ran into because they seem so messed up that they couldn't be discussed in public.
One thing I was able to extract is: CouchDB doesn't scale.
Thanks! But no thanks.
I wrote a book on CouchDB and I pretty much used it all, or at least looked at it very, very closely. I also get plenty of experience with CouchDB due to my job. Indeed, there are many situations where CouchDB doesn't scale or where it becomes extremely hard to make it scale. Situations where the user is better of putting data somewhere else.
Myself (and I'm assuming others) enjoy to learn the reasons why things break, so we can take this experience and use it going forward. If this doesn't happen we might as well all subscribe to the koolaid of a closed source vendor and purchase update subscriptions, install security packs and happily live ever after.
Another piece of information I gathered from the various emails written is that Canonical maintained CouchDB-specific patches for Ubuntu1. However, it's unknown what the purpose of these patches were. For example, if these patches made CouchDB scale (magically) for Ubuntu1 or if the patchset added a new feature.
What I'd really like to know is why these patches were not discussed in the open and why no one worked with the project on incorporating them into upstream. The upstream is the Apache CouchDB project.
This is another example of where communication went horribly wrong or just didn't happen.
I'm a little torn here and I don't want to offend anyone (further) especially since I know a couple Couchbase'rs or original CouchOne'rs (Hello to Jan, JChris and Mikeal) in person, but seriously: a lot of people realized that CouchOne stopped being The CouchDB company a long time ago.
This is not to say that the CouchDB project members who are employed by CouchOne/Couchbase are not dedicated to CouchDB. But if I take a look at the mobile strategy and the more or less recent integration of CouchDB with Membase/Memcache, I must notice that these strategies are far away from Apache CouchDB. Big data (whatever that means), to mobile and back.
The conclusion is that the majority of work done will not be merged into Apache CouchDB and this is one of the reasons why the Apache CouchDB project hasn't evolved much in a long time.
I realize that when a company has a different strategy, not everything they do can be send upstream. After all, most if not all companies operate in a world where money is to be made and goals are to be met. Nothing wrong there.
But let's take a look at the one project which could have been dedicated to Apache CouchDB: the documentation project.
CouchOne hired an ex-MySQL'er to write really great documentation for CouchDB. The documentation made sense, it was up to date with releases, contained lots examples and what not. But it was never contributed to the open source project. The documentation is still online today, though it's now the documentation of the Couchbase Server HTTP API.
So in my opinion the biggest news is not that Canonical stopped using CouchDB and it's also not outrageous to think that there can be one CouchDB company. The biggest news is that Couchbase officially said: "It's not us!".
Having said that and also not knowing much about Canonical's setup and scale, I still fail to even remotely understand why they didn't work with Cloudant who spezialize in making CouchDB scale all along.
Of course it is unfair to single them (Couchbase employees) out like that. For the record, there are pretty vivid projects such as GeoCouch which are also funded by Couchbase and while being devoted to the project, these guys also have to meet goals for their company.
Add to that, that other CouchDB contributors involved have not driven sustantial user-facing changes in Apache CouchDB either. CouchDB is still a very technical project with a lot of technical issues to solve. The upside to this situation is that while other NoSQL vendors add new buzzwords to each and every CHANGELOG, CouchDB is very conservative and stability driven. I appreciate that a lot.
User-facing changes on the other side are just as important for the health of a project. Subtle changes aside, but today's talks on for example querying CouchDB are extremely similar to those talks given a year or two ago. Whatever happens in this regard is not visible to users at all.
Take URL rewriting, virtualhosts and range queries as examples for features. I question:
Users need to have the ability to grasp what's on the roadmap for CouchDB. There needs to be a way for not so technical users to provide feedback which is actually incorporated into the project. All of these things aside from opening issues in a monster like Jira.
Since no one bothers currently, this is not going to happen soon.
Pretty candid stuff.
In terms of marketing and with a lack of an official CouchDB company, the CouchDB project has taken a PostgreSQL-attitude in the last two years.
In a nutshell:
We don't give a damn if you don't realize that our database is better than this other database.
This is a little dangerous for the project itself because when I look at the cash other NoSQL vendors pour into marketing for their NoSQL database, I realized quickly that with the lack of support this project can go away pretty soon.
CouchDB being an Apache project doesn't save me or anyone either: clean intellectual property, deserted, for forever.
The various larger companies (let's say Cloudant and Meebo) are basically employed with their own forks with maybe too little reason to merge anything back to upstream yet. There are independent contributors Enki Multimedia who contribute to core but also sub projects like CouchApp.
And then, there's Couchbase which is trying to tie CouchDB behind Memcached. And from what I can tell pretty much abondens HTTP and other slower CouchDB principals in the process.
You saw it coming: it depends!
Dear Jan, I'm still thinking about the email you wrote while I write my own blog entry. And honestly, that email and the general response raised more questions for myself and others than it answered.
I'd like to emphasize a difference I see (thanks, Lukas):
Is the core of Apache CouchDB alive? — It's not dead.
There is a lot of innovation going on in CouchDB's ecosystem.
Most notable, the following projects come to mind:
Need more? Check out CouchDB in the wild which I think is more or less up to date.
Hate it or love it — there is plenty of innovating going on. And many (if not all) CouchDB committers are a part of it.
The innovation just doesn't happen in CouchDB's core.
My closing words are that I don't plan on migrating anywhere else. If anything, we have mostly migrated to BigCouch.
For Apache CouchDB, I think it's important that someone fills that void. That can be either a company, a BDFL or more engaging project leaders (plural). I think this is required so the project continues vividly.
Because I would really like to see the project survive.
by Till Klampaeckel (till@php.net) at December 21, 2011 04:15 PM
2011 has been quite an eventful year for Cloudant. In the spring we outgrew our tiny basement offices in Davis Square and moved Cloudant HQ into Boston proper. We added a number of fantastic engineers to the team and we said goodbye to some of our earliest hires.
Going into the new year, we’ve expanded our platform to 5 geographic zones in the US and 3 more around the globe. Most importantly we more than doubled the number of developers, applications, small businesses, and enterprises using our hosted service – and found quite a few independent BigCouch users out there along the way. We continue to be surprised and delighted by the passion our users show for our products. It is that passion that drives us to continuously improve BigCouch and our hosted service.
With all that has been going on here, the founders decided it was time to expand the team. So today we are happy to announce that Derek Schoettle has joined Cloudant as our new CEO. Yes, Derek has an MBA – don’t hold that against him. His passion is working within early-stage companies bringing great technology to market, most recently at Vertica, and numerous startups before that. Derek has been an advisor to Cloudant for most of 2011 and was a natural choice when the time came. We are very excited to have him on board helping Cloudant grow.
With Derek on board, Adam, Mike and I will have more time to focus on the things we are passionate about . For me, that means working closely with our customers and users, focusing on building a great product that solves their data problems. Mike will continue to focus on customer success and outreach, and will hopefully have more time ‘on the road,’ evangelizing BigCouch to a wider audience. Adam will continue to work deep in the guts of Apache CouchDB, BigCouch, and the Cloudant platform, making all of our crazy thought experiments into real features.
2011 was a big year for Cloudant, but we have even bigger goals for 2012 (assuming the world doesn’t end.) Stay tuned to this channel as we announce some of the big features and plans we have.
Alan (for Adam and Mike)
Co-Founder, Cloudant
If you've ever wondered whether there's a tool that will automatically compress JavaScript files when you're pushing your CouchApp somewhere, the answer is that there exists at least one: couchapp-compress.
Basically, couchapp-compress is a small Ruby script that wraps the couchapp command line tool. It compresses a CouchApp's JavaScript files and puts them altogether into one file, and temporarily changes the CouchApp so that instead of all the single uncompressed JavaScript files the compressed one is used. It pushes the CouchApp and then restores the previous state, so that again everything looks like before couchapp-compress was executed.
Check out the README for more details if you're curious and want to give it a try.
On December 14th we hosted a webinar with our friends at 2600hz, the open-source VOIP platform. Darren Schreiber, CEO of 2600hz, gave a presentation on how his company uses the BigCouch data platform to better serve its customers. Darren gave some great examples of how moving to a scalable document store has helped his company move faster, save money, and avoid operational headaches.
In case you missed it, we recorded the webinar and you can view it below. If you just want the slides, follow the link below the video.
Download the slides here
parse() method: Invitees = Backbone.Collection.extend({
model: Models.Invitee,
url: function( models ) {
return '/invitees?' + ( models ? 'ids=' + _.pluck( models, 'id' ).join(',') : '' );
},
parse: function(response) {
debugger;
return _(response.rows).map(function(row) { return row.doc; });
}
});parse() method, the CouchDB response contains a "rows" attribute with the actual invitee documents / attributes. The "rows" in that response contain several attributes. The only one that is needed to create an Invitee Backbone model is the "doc" attribute, which contains the entire JSON representation of a Person/Invitee:{
"_id": "6bb06bde80925d1a058448ac4d006f6e",
"_rev": "3-231654f6914afe2e20eb57a41ec8497a",
"firstName": "Black",
"lastName": "Francis",
"type": "Person"
}
parse method should work fine. And it does. The problem is that the existing models in the collection look like:
attributes: Object
id: "6bb06bde80925d1a058448ac4d006f6e"doc: ObjectDo you see the problem there? I did not. Not for quite some time. The problem is that the existing placeholder has an "id" attribute, but the replacement from CouchDB has an "_id" attribute. CouchDB puts an underscore in front of the ID to indicate that it is meta data. Far be it for me to argue the wisdom of doing so, but it sure screws things up for me here.
_id: "6bb06bde80925d1a058448ac4d006f6e"
_rev: "3-231654f6914afe2e20eb57a41ec8497a"
firstName: "Black"
lastName: "Francis"
type: "Person"
parse() I copy the "_id" attribute to "id": Invitees = Backbone.Collection.extend({
// ...
parse: function(response) {
return _(response.rows).map(function(row) {
var doc = row.doc;
doc['id'] = doc['_id'];
return doc;
});
}
});
by Chris Strom (noreply@blogger.com) at November 11, 2011 05:31 AM
{
"_id": "6bb06bde80925d1a058448ac4d004fb9",
"_rev": "2-7fb2e6109fa93284c19696dc89753102",
"title": "Test #7",
"description": "asdf",
"startDate": "2011-11-17",
"invitees": [
"6bb06bde80925d1a058448ac4d0062d6",
"6bb06bde80925d1a058448ac4d006758",
"6bb06bde80925d1a058448ac4d006f6e"
]
}And, to get that invitees attribute loading actual Invitee models, I had to add a relations attribute to my Appointment relational-model: Appointment = Backbone.RelationalModel.extend({
// ...
relations: [
{
type: Backbone.HasMany,
key: 'invitees',
relatedModel: 'Invitee',
collectionType: 'Invitees'
}
],
// ...
});loadInvitees method call from my pre-backbone-relational days with the fetchRelated() method from backbone-relational: Appointment = Backbone.RelationalModel.extend({
// ...
initialize: function(attributes) {
if (!this.id)
this.id = attributes['_id'];
this.fetchRelated("invitees");
// this.loadInvitees();
},
// ...
}); var AppointmentEdit = new (Backbone.View.extend({
// ...
showInvitees: function() {
$('.invitees', this.el).remove();
$('#edit-dialog').append('<div class="invitees"></div>');
if (this.model.get("invitees").length == 0) return this;
var view = new Invitees({collection: this.model.get("invitees")});
$('.invitees').append(view.render().el);
return this;
}
}));fetch() model that I wrote a few nights back. Specifically, the collection overrides fetch to individually retrieve the models specified by the list of IDs, manually triggering the "reset" event when complete.app.get('/invitees', function(req, res){
var options = {
method: 'POST',
host: 'localhost',
port: 5984,
path: '/calendar/_all_docs?include_docs=true'
};
var couch_req = http.request(options, function(couch_response) {
console.log("Got response: %s %s:%d%s", couch_response.statusCode, options.host, options.port, options.path);
couch_response.pipe(res);
}).on('error', function(e) { /* ... */ });
var ids = req.param('ids').split(/,/);
couch_req.write(JSON.stringify({"keys":ids}));
couch_req.end();
});/appointments resource with a query parameter of a comma separated list of strings.url() method of my collection (url can be either property or method in Backbone): Invitees = Backbone.Collection.extend({
model: Models.Invitee,
url: function( models ) {
return '/invitees' + ( models ? '?ids=' + _.pluck( models, 'id' ).join(',') : '' );
},
parse: function(response) {
return _(response.rows).map(function(row) { return row.doc;});
}
});parse() the results coming back from CouchDB to ensure that they map into an array of Model attributes).
by Chris Strom (noreply@blogger.com) at November 10, 2011 05:08 AM
Big day for Cloudant — and big news in the near future for all BigCouch users. We are happy to announce that biotech heavyweight and perennial Fortune 500 company Monsanto has chosen Cloudant to be the core of their new genome analysis platform.
We’ve been working with them for a few weeks now and we couldn’t be more thrilled with the partnership. We are no strangers to the analysis of the big bytes generated by the smallest of things. Our founders first devised our open-source platform BigCouch while at MIT, working on Large Hadron Collider experiments generating millions of data points per second from the collision of atomic particles. Turns out, Biotechnology & genome analysis is no different in this regard. As GigaOM summarized it: “innovation is rampant, but data growth is outpacing the ability to analyze it, making faster, cheaper and more scalable data systems integral to leveling the playing field”.
This is new ground for us — and for NOSQL as a whole. This Monsanto platform is not destined to toil away in one of their (many) R&D labs, only used by scientists with lab coats & beakers. We’re not powering the CMS of some administrative division or a minor part of their website. Cloudant’s BigCouch will be the core, for both storage and analysis of a new, company-wide platform powering a fundamental aspect of a Fortune 500 business: the analysis & identification of new traits & genomic combinations in agricultural crops. The data & reporting interfaces will be used across Monsanto and should be instrumental in the making of key business decisions.
Now, this also means big news for our users. Monsanto has been very eager to let us open-source many of the core enhancements we make to BigCouch for them. So while this won’t quite make it into the impending BigCouch 0.4 release (which is days away now), or on our cloudant.com clusters for another few months, you can definitely expect to see many Monsanto-driven improvements folded back into our offerings, such as support for more languages on view servers, new tools to model data workflows, as well as a flurry of improvements to our storage & retrieval algorithms.
At Cloudant, we’re all excited to see NOSQL platforms, including BigCouch, being used to power the next generation of business technology. So when announcements like this also translate to a more stable, performant & productive experience, for everybody else that uses a Cloudant product, we find it to be a big day indeed.
You can always contact us if you have any questions. There is also a press release available.
draw_calendar(), which takes an ISO 8601 string that can draw the basic calendar (without the Backbone application adding appointments): function draw_calendar(year_and_month) {
$('.year-and-month', 'h1').html(' (' + year_and_month + ') ');
reset_calendar();
add_dates_to_calendar(year_and_month);
};
draw_calendar('2011-09');Then I am treated to a blank September calendar:
startDate:
2011-10:➜ calendar git:(pagination) curl http://localhost:5984/calendar/_design/appointments/_view/by_month\?key\='"2011-10"'Yay!
{"total_rows":18,"offset":12,"rows":[
{"id":"4a5600f5b2e36fc99d24fe9b8700037d",
"key":"2011-10",
"value":{
"_id":"4a5600f5b2e36fc99d24fe9b8700037d",
"_rev":"5-181418fffeacdc08b5fdda91525978b6",
"title":"Update dialog errors",
"description":"asdf1",
"startDate":"2011-10-05"}},
{"id":"8b5c80c0211068428272af4784000451",
"key":"2011-10",
"value":{
"_id":"8b5c80c0211068428272af4784000451",
"_rev":"1-87f5be84687eada2cd178c8dab7aa34c",
"title":"Finish Beta",
"description":"Book important",
"startDate":"2011-10-31"}},
{"id":"8b5c80c0211068428272af478400f7bb",
"key":"2011-10",
"value":{
"_id":"8b5c80c0211068428272af478400f7bb",
"_rev":"2-b1ce8c6c815a11f7b78b7db412988451",
"title":"Go to bed early",
"description":"asdf",
"startDate":"2011-10-22"}},
{"id":"8b5c80c0211068428272af478400fc93",
"key":"2011-10",
"value":{
"_id":"8b5c80c0211068428272af478400fc93",
"_rev":"1-b19359520433df557ad2fa0d56165f24",
"title":"Validations",
"description":"description should be required",
"startDate":"2011-10-03"}},
{"id":"956a5c19fd866a6a024bbb4c39002e3b",
"key":"2011-10",
"value":{
"_id":"956a5c19fd866a6a024bbb4c39002e3b",
"_rev":"2-0bbb8660f7f116cc813e0fd9093cec6a",
"title":"Has Description",
"description":"asdf",
"startDate":"2011-10-13"}},
{"id":"956a5c19fd866a6a024bbb4c390031a2",
"key":"2011-10",
"value":{
"_id":"956a5c19fd866a6a024bbb4c390031a2",
"_rev":"2-0568aded9df520a0c1c8bb2ee8961156",
"title":"In-dialog errors","description":"asdf",
"startDate":"2011-10-04"}}
]}
_all_doc resource it had been using.Apppointments Backbone collection how to interact with this new resource requires almost no changes at all. Only the parse() method needs to be updated to return appointments from the value attribute of my query: var Collections = (function() {
var Appointments = Backbone.Collection.extend({
model: Models.Appointment,
url: '/appointments',
parse: function(response) {
return _(response.rows).map(function(row) { return row.value ;});
}
});
return {Appointments: Appointments};
})();date query parameter to the Appointments URL. This is done by passing a data option to the collection's fetch() method (just like with jQuery's ajax() call):// ...
// Initialize the app
var appointments = new Collections.Appointments();
new Views.Application({collection: appointments});
var today = new Date(),
year = today.getFullYear(),
month = today.getMonth() + 1,
year_and_month = year + '-' + pad(month);
appointments.fetch({data: {date: year_and_month}});september to last month's date, then draw_calendar(september) (to get the calendar itself drawn correctly). Lastly, I tell the collection to load in September's appointments via a calendar.appointments.fetch({data: {date: september}}):
by Chris Strom (noreply@blogger.com) at October 09, 2011 04:01 AM
Got response: 400 localhost:5984/calendar/undefinedHrm... Checking the CouchDB logs, I see that I have a least got the PUT part of the update working:
{ error: 'bad_request', reason: 'Invalid rev format' }
[info] 127.0.0.1 - - 'PUT' /calendar/undefined 400The PUT is correct, but the "undefined" is a clear indication that whatever I am updating does not have an
"id" attribute.Backbone.sync() is not sending the JSON representation of the model, but rather it is sending the model itself: var faye = new Faye.Client('/faye');
Backbone.sync = function(method, model, options) {
faye.publish("/calendars/" + method, model);
}toJSON() ought to fix the problem: var faye = new Faye.Client('/faye');
Backbone.sync = function(method, model, options) {
if (model.toJSON) model = model.toJSON();
faye.publish("/calendars/" + method, model);
}Backbone.sync() method, then something must already be calling toJSON() on my models. Since Backbone.sync() is sending directly to faye, it must be faye that is calling toJSON(). And in fact it is. Looking through the faye source, for each of the various transports, I see Faye.toJSON:Faye.Transport.WebSocket = Faye.extend(Faye.Class(Faye.Transport, {
// ...
request: function(messages, timeout) {
this._timeout = this._timeout || timeout;
this._messages = this._messages || {};
Faye.each(messages, function(message) {
this._messages[message.id] = message;
}, this);
this.withSocket(function(socket) { socket.send(Faye.toJSON(messages)) });
},
// ...
});toJSON() wrapper. That or faye is just a fabulous choice for a Backbone transport. At the very least, it is not the cause of my woes."_id" or "_rev" attributes needed on the server:client.subscribe('/calendars/update', function(message) {
// HTTP request options
var options = {
method: 'PUT',
host: 'localhost',
port: 5984,
path: '/calendar/' + message._id,
headers: {
'content-type': 'application/json',
'if-match': message._rev
}
};
// ...
});So I finally take the advice of Recipes with Backbone co-author and put mapping of "_id" and "_rev" attributes into the Backbone.sync() method: Backbone.sync = function(method, model, options) {
var message = model.toJSON();
if (!message._id && message.id) message._id = message.id
if (!message._rev && message.rev) message._rev = message.rev
faye.publish("/calendars/" + method, message);
}message variable so that setting attributes does not affect the real model.
Got response: 201 localhost:5984/calendar/8b5c80c0211068428272af478400df1eThat is a fine stopping point for tonight. I think that I might have resolved all of my faye transport issues (and issues that I noticed after better testing with faye than in the original). I will do some more monkey testing tomorrow and then possibly move on to other areas to explore.
{ ok: true,
id: '8b5c80c0211068428272af478400df1e',
rev: '4-2929cf0e4933a4e32474145eb3e79f02' }
by Chris Strom (noreply@blogger.com) at October 04, 2011 04:09 AM
Backbone.sync(), in my Backbone application, I already have delete requests being sent to the /caledars/delete faye channel: var faye = new Faye.Client('/faye');
Backbone.sync = function(method, model, options) {
faye.publish("/calendars/" + method, model);
} _(['create', 'update', 'delete', 'read', 'changes']).each(function(method) {
faye.subscribe('/calendars/' + method, function(message) {
console.log('[/calendars/' + method + ']');
console.log(message);
});
});
/calendars/delete faye channel in my express.js server. I already have this working for adding appointments, so I can adapt the overall structure of the /calendars/add listener for /calendars/delete:client.subscribe('/calendars/delete', function(message) {
// HTTP request options
var options = {...};
// The request object
var req = http.request(options, function(response) {...});
// Rudimentary connection error handling
req.on('error', function(e) {...});
// Send the request
req.end();
});The HTTP options for the DELETE request are standard node.js HTTP request parameters: // HTTP request options
var options = {
method: 'DELETE',
host: 'localhost',
port: 5984,
path: '/calendar/' + message._id,
headers: {
'content-type': 'application/json',
'if-match': message._rev
}
};if-match HTTP header ought to work. The _rev attribute on the record/message sent to /calendars/delete holds the latest revision that the client holds. Similarly, I can delete the correct object from CouchDB by specifying the object ID in the path attribute—the ID coming from the _id attribute of the record/message./calendars/remove channel.http.request(). This accumulator callback accumulates chunks of the reply into a local data variable. When all of the data has been received, the response is parsed as JSON (of course it's JSON—this is a CouchDB data store), and the JSON object is sent back to the client: var req = http.request(options, function(response) {
console.log("Got response: %s %s:%d%s", response.statusCode, options.host, options.port, options.path);
// Accumulate the response and publish when done
var data = '';
response.on('data', function(chunk) { data += chunk; });
response.on('end', function() {
var couch_response = JSON.parse(data);
console.log(couch_response)
client.publish('/calendars/remove', couch_response);
});
});/calendars/remove channel, I supply a simple logging tracer bullet: faye.subscribe('/calendars/remove', function(message) {
console.log('[/calendars/remove]');
console.log(message);
});/calendars/delete channel (which we have already seen working above). The faye subscription on the server should be able to use this to remove the object from the CouchDB database. Finally, this should result in another message being broadcast on the /calendars/remove channel. This /calendars/remove message should be logged in the browser. So let's see what actually happens...
remove() method of the collection? faye.subscribe('/calendars/remove', function(message) {
console.log('[/calendars/remove]');
console.log(message);
calendar.appointments.remove(message);
});remove() method looks up models to be deleted via get(): _remove : function(model, options) {
options || (options = {});
model = this.getByCid(model) || this.get(model);
// ...
}get() method looks up models by the id attribute: // Get a model from the set by id.
get : function(id) {
if (id == null) return null;
return this._byId[id.id != null ? id.id : id];
},id attribute was set on the message/record received on the /calendars/remove channel: var Appointment = Backbone.View.extend({
initialize: function(options) {
// ....
options.model.bind('destroy', this.remove, this);
options.model.bind('error', this.deleteError, this);
options.model.bind('change', this.render, this);
},
// ...
});destroy event is only emitted if the default Backbone.sync is in place. If the XHR DELETE is successful, a success callback is fired that emits destroy. Since I have replaced Backbone.sync, that event never fires.remove event. So all I need to change in order to make this work is to replace destroy with remove: var Appointment = Backbone.View.extend({
initialize: function(options) {
// ....
options.model.bind('remove', this.remove, this);
options.model.bind('error', this.deleteError, this);
options.model.bind('change', this.render, this);
},
// ...
});
by Chris Strom (noreply@blogger.com) at October 02, 2011 09:19 PM
sync() method in my Backbone.js model to achieve an added layer of persistence. In addition to the normal REST persistence, my model also persists newly created appointments on a Faye pub-sub channel: var Appointment = Backbone.Model.extend({
urlRoot : '/appointments',
initialize: function(attributes) {
// ...
this.faye = new Faye.Client('/faye');
},
// ...
sync: function(method, model, options) {
if (method == "create") {
this.faye.publish("/calendars/public", model);
}
Backbone.sync.call(this, method, this, options);
}
});Backbone.sync() to send any sync requests for create, update, delete or read to a faye channel named accordingly: var faye = new Faye.Client('/faye');
Backbone.sync = function(method, model, options) {
faye.publish("/calendars/" + method, model);
}
// Simple logging of Backbone sync messages
_(['create', 'update', 'delete', 'read']).each(function(method) {
faye.subscribe('/calendars/' + method, function(message) {
console.log('[/calendars/' + method + ']');
console.log(message);
});
});
fetch() of the collection: // Initialize the app
var appointments = new Collections.Appointments;
new Views.Application({collection: appointments});
appointments.fetch();reset() call. Personally, I prefer serving up a static file that shows something almost immediately followed by quick requests to populate the page with useful, actionable information./calendars/read channel. Doing faye things on the server is relatively easy. I already have the faye node.js adapter hooked in to my express.js application. I can then call to getClient() to gain access to client actions like subscribe():// Faye server
var bayeux = new faye.NodeAdapter({mount: '/faye', timeout: 45});
bayeux.attach(app);
// Faye clients
var client = bayeux.getClient();
client.subscribe('/calendars/read', function() {
// do awesome stuff here
});client.subscribe('/calendars/read', function() {
// CouchDB connection options
var options = {
host: 'localhost',
port: 5984,
path: '/calendar/_all_docs?include_docs=true'
};
// Send a GET request to CouchDB
var req = http.get(options, function(couch_response) {
console.log("Got response: %s %s:%d%s", couch_response.statusCode, options.host, options.port, options.path);
// Accumulate the response and publish when done
var data = '';
couch_response.on('data', function(chunk) { data += chunk; });
couch_response.on('end', function() {
var all_docs = JSON.parse(data);
client.publish('/calendars/reset', all_docs);
});
});
// If anything goes wrong, log it (TODO: publish to the /errors ?)
req.on('error', function(e) {
console.log("Got error: " + e.message);
});
});pipe() the response back to the client. Backbone (more accurately jQuery) itself takes care of parsing the JSON. Here, I have to accumulate the data response from CouchDB and parse it into a JSON object to be published on a Faye channel. I could send back a JSON string, requiring the client to parse, but that feels like bad form. Faye channels can transmit actual data structures, so that is what I ought to do./calendars/reset channel because that is what the client will do with this information—reset the the currently empty appointments collection: window.calendar = new Cal();
faye.subscribe('/calendars/reset', function(all_docs) {
console.log('[/calendars/reset]');
console.log(all_docs);
calendar.appointments.reset(all_docs);
});/calendar/read message is still going out. I also see that I am getting a response back that includes the ten appointments already scheduled for this month:
/calendars/reset channel as expected. It is the CouchDB query results as expected, but something is going wrong in the call to reset() on the appointments collection. Probably, something related to the "Uncaught ReferenceError: description is not defined" error message at the bottom of the Javascript console.reset() or add() need to be run through parse() first. Well, not always, just when parse() does something with the data. Something like I had to do with the CouchDB results: var Appointments = Backbone.Collection.extend({
model: Models.Appointment,
parse: function(response) {
return _(response.rows).map(function(row) { return row.doc ;});
}
});parse(): faye.subscribe('/calendars/reset', function(message) {
console.log('[/calendars/reset]');
console.log(message);
var all_docs = calendar.appointments.parse(message);
calendar.appointments.reset(all_docs);
});
sync() function.Backbone.sync() did the trick. On the minus side, I had to do a little more work to convert CouchDB responses into real Javascript data structures. That is not a huge negative (and one that I can easily push into a helper function). Still outstanding it how this will affect the entire application. Also, I have the feeling that I could choose faye channel names better. Questions for another day...by Chris Strom (noreply@blogger.com) at October 02, 2011 08:42 PM
/calendars/udpate faye channel on my backend. My debug subscription in the client verifies that the message is being published to that channel:
client.subscribe('/calendars/update', function(message) {
// HTTP request options
var options = {...};
// The request object
var req = http.request(options, function(response) {...});
// Rudimentary connection error handling
req.on('error', function(e) {...});
// Write the PUT body and send the request
req.write(JSON.stringify(message));
req.end();
}); // HTTP request options
var options = {
method: 'PUT',
host: 'localhost',
port: 5984,
path: '/calendar/' + message._id,
headers: {
'content-type': 'application/json',
'if-match': message._rev
}
};if-match is a CouchDB optimistic locking thing)/calendars/changes channel: // The request object
var req = http.request(options, function(response) {
console.log("Got response: %s %s:%d%s", response.statusCode, options.host, options.port, options.path);
// Accumulate the response and publish when done
var data = '';
response.on('data', function(chunk) { data += chunk; });
response.on('end', function() {
var couch_response = JSON.parse(data);
client.publish('/calendars/changes', couch_response);
});
}); // Write the PUT body and send the request
req.write(JSON.stringify(message));
req.end();/calendars/update will be seen by my server-side subscription, which will tell CouchDB to update the record and finally the browser will see the update on the /calendars/changes channel.Got response: 201 localhost:5984/calendar/66543e3457df7597f0e41764e500067cAnd I even see that response back in the browser:
{ ok: true,
id: '66543e3457df7597f0e41764e500067c',
rev: '3-243c4d7084fcdcb7c728917a73e94b97' }
Got response: 409 localhost:5984/calendar/66543e3457df7597f0e41764e500067c
{ error: 'conflict',
reason: 'Document update conflict.' }by Chris Strom (noreply@blogger.com) at October 02, 2011 08:40 PM
/calendars/update faye channel. The updated information is sent back from the server on the /calendars/changes faye channel:
rev attribute. If a subsequent update is sent with the old rev, CouchDB's optimistic locking will kick in and reject the update:Got response: 409 localhost:5984/calendar/cd823d4aaaf358069f9a800410000b95So, in my client, I subscribe to the
{ error: 'conflict',
reason: 'Document update conflict.' }
/calendars/changes channel. In the subscription's callback, I use the revision published by the server to update the Backbone model: faye.subscribe('/calendars/changes', function(message) {
console.log('[/calendars/changes]');
console.log(message);
var model = calendar.appointments.get(message);
model.set({rev: message.rev});
model.set({_rev: message.rev});
});Got response: 201 localhost:5984/calendar/cd823d4aaaf358069f9a800410000b95Yay! Except... immediately after I see the successful update, I see:
{ ok: true,
id: 'cd823d4aaaf358069f9a800410000b95',
rev: '7-55b3b95b6996072aa1519b6070cf12b6' }
Got response: 201 localhost:5984/calendar/cd823d4aaaf358069f9a800410000b95
{ ok: true,
id: 'cd823d4aaaf358069f9a800410000b95',
rev: '8-a06160ea8bef50bd71d92a73db61c14b' }
Got response: 201 localhost:5984/calendar/cd823d4aaaf358069f9a800410000b95When I update again, I successfully update the record, which is followed immediately by two conflicts:
{ ok: true,
id: 'cd823d4aaaf358069f9a800410000b95',
rev: '8-a06160ea8bef50bd71d92a73db61c14b' }
Got response: 409 localhost:5984/calendar/cd823d4aaaf358069f9a800410000b95
{ error: 'conflict',
reason: 'Document update conflict.' }
Got response: 201 localhost:5984/calendar/cd823d4aaaf358069f9a800410000b95Ah. I know what that is. And I will fix it. But first, I need to finish proof reading the recipes that taught me how to solve it.
{ ok: true,
id: 'cd823d4aaaf358069f9a800410000b95',
rev: '9-c09e4a0a8bbd5cc25a9e67b19b6a254b' }
Got response: 409 localhost:5984/calendar/cd823d4aaaf358069f9a800410000b95
{ error: 'conflict',
reason: 'Document update conflict.' }
Got response: 409 localhost:5984/calendar/cd823d4aaaf358069f9a800410000b95
{ error: 'conflict',
reason: 'Document update conflict.' }
by Chris Strom (noreply@blogger.com) at October 02, 2011 08:37 PM
Earlier this month, I gave a presentation at GoGaRuCo about using CouchDB, and how many people do it in inappropriate or wasteful ways.
The centerpiece of my talk was a step-by-step example of best-practices for CouchDB interaction, leading to an improvement of 99% in the median time it takes to do a simple insert to CouchDB (or Cloudant). Admittedly, I start from a worst-case scenario, but the scary part is that it is not a totally unrealistic one; in fact, I’ve seen it replicated many times over.
After factoring my advice in, we have a performance improvement of 96% (if you stick to conversing with CouchDB through JSON, which is recommended for now), or 99% with my experimental patches to support MessagePack in CouchDB & CouchRest.
What’s more, none of the advice I give leads to a relaxation in consistency or durability. It’s plain optimization, no strings attached, just there for the taking. Many of the points I touch on are also valid no matter what language or library you are using (or are easily translatable to other ecosystems), so I would strongly encourage anybody using CouchDB and interested in performance optimization to look at the video.
Head on over over to Confreaks to see the video (the meat of the talk starts at the 5:10 mark).
The FOSS4G 2011 is over now. Time for a small report. The crowd was amazing and it was again the ultimate gathering of the Free and Open Source for Geospatial developer tribe. Solid presentations and great evenings.
I'm really happy how my talk went, I really enjoyed it. The were lots of people (although there was a talk from Frank Warmerdam at the same time) asking interesting questions at the end.
The talk is not only about GeoCouch but also gives you an overview of some of the features it leverages from Apache CouchDB. In the end you should have an overview why you might want to use GeoCouch for your next project.
You can get the slides right here.
I was happy to see that there was another talk about GeoCouch. Other talks I really enjoyed were:
And of course there were also great talks from in the plenary sessions from Paul Ramsey about Why do you do that? An exploration of open source business models and Schuyler Erle's so funny lightning talk about Pivoting to Monetize Mobile Hyperlocal Social Gamification by Going Viral
At the code sprint I was working on MapQuery together with Steven Ottens and Justin Penka. Steven was working on TMS support, Justin on a 6 minutes tutorial and I on making manual adding of features possible.
The OpenLayers developers did the migration from Subversion to Git for their development. OpenLayers is now available on Github.
And luckily there was a fire alarm in between to take a group photograph.
I really hope there won't be a yearly FOSS4G conference for the whole of the US. There should be regional events, as I think one big one would draw the attention away from the international conference. Why should you fly to Beijing for the FOSS4G 2012 if you can meet the majority of the developers in the US as well?
The FOSS4G was great. It was organized well and people were always out in the evenings. The only minor nitpick is that many people working remote had the city of their company in the name badge and not the one they live in. It seems that the original for you had to fill was confusing. So for next year it should perhaps say “Location where you live”. Hence I still don't believe that there were more Dutch than German people at the conference (Tik hem aan, ouwe! ;)
window.Appointment = Backbone.Model.extend({
// ...
destroy: function() {
Backbone.Model.prototype.destroy.call(this, {
headers: {'If-Match': this.get("_rev")}
});
}
});{"ok":true,"id":"7acf98778a669f4d6fc33d6b340106de","rev":"1-21662a1368aa1592d1e5d1df710f6d8c"}But the actual data is stored as:{
"_id": "7acf98778a669f4d6fc33d6b340106de",
"_rev": "1-21662a1368aa1592d1e5d1df710f6d8c",
"title": "Delete me #2",
"description": "asdf",
"startDate": "2011-09-15"
}CouchDB normally represents meta data with a leading underscore ("_id", "_rev"). In the POST / create response, however, the ID and revision returned are not meta-data. Rather they are the actual data returned describing the newly created record.appointment.get("_rev") will not work but appointment.get("rev") will._rev or rev: destroy: function() {
Backbone.Model.prototype.destroy.call(this, {
headers: {'If-Match': this.get("_rev") || this.get("rev")}
});
}this.get("rev") and it just work—regardless of update, create or delete.
window.Appointment = Backbone.Model.extend({
get: function(attribute) {
return Backbone.Model.prototype.get.call(this, attribute) ||
Backbone.Model.prototype.get.call(this, "_" + attribute);
},
// ...
});get() method, I call the get() method directly on the Backbone.Model.prototype. Since I am invoking it directly, as not as method on an instantiated object, I have to supply the object context to be used inside the method. After all, the get() method expects to be called on an object and, as such, it expects the this variable to refer to that object. call() method does just this—it sets the this variable inside the function to the first argument supplied. In this case, I supply the this variable from my Appointment model. So, in the end, the original get() is called with the same this variable with which it would have otherwise been called. destroy() method to work with both CouchDB updates and deletes:
window.Appointment = Backbone.Model.extend({
get: function(attribute) {
return Backbone.Model.prototype.get.call(this, attribute) ||
Backbone.Model.prototype.get.call(this, "_" + attribute);
},
destroy: function() {
Backbone.Model.prototype.destroy.call(this, {
headers: {'If-Match': this.get("rev")
});
},
// ...
});Backbone.Model.prototype.get.call() in my get() will return undefined but the second one ("_rev") will return the current revision number for the records.Backbone.Model.prototype.get.call() will return the revision ID.by Chris Strom (noreply@blogger.com) at September 08, 2011 04:12 AM
<span> with a class of "delete" to my calendar event template:<script type="text/template" id="calendar-event-template">On the page, the "X" displays like:
<span class="event" title="<%= description %>">
<%= title %>
<span class="delete">X</span>
</span>
</script>
window.EventView = Backbone.View.extend({
// ...
events: {
'click .delete': 'test'
},
test: function() {console.log("delete")},
});
So, when a click event is received in this view for an element with the delete class, the "test" function is called. Sure enough, clicking on the X inside the delete <span> logs "delete" messages to the console:
test function with a handler for the "click .delete" event:window.EventView = Backbone.View.extend({
// ...
events: {
'click .delete': 'deleteClick'
},
deleteClick: function() {
this.model.destroy();
},
remove: function() {
$(this.el).find('.event').remove();
}
});
It feels a little awkward having a remove() method and a deleteClick(). The former removes the UI element from the page. The latter handles clicks that should signal the model to delete itself, which will, in turn, tell the view to remove itself from the page. I will worry about the odd feeling another day. For now, I am not quite done with my delete.e.destroy({headers: {'If-Match':e.model.get("_rev")} })
It seems really wrong to me that the View should be responsible for knowing about this. But how to get the model to do this? I could create a new destroyWithRevision method on the model, but the view would still need to know to call this instead of the conventional destroy() method.
window.Event = Backbone.Model.extend({
// ...
destroy: function() {
Backbone.Model.prototype.destroy.call(this, {
headers: {'If-Match': this.get("_rev")}
});
}
});
destroy function that resides on the Backbone.Model prototype. Since I am invoking that function directly, I need to supply an object instance so that the method has a this (or self if you're a Rubyist) to which it can refer. That is the first argument to a Javascript call method. Then I can supply the arguments that inform CouchDB of the revision being deleted.destroy() method—completely unaware of this complexity. As an added bonus, I am not losing any of the benefits of optimistic locking—if the loaded model was superseded before the user clicked the "X", the delete would fail. And yes, when I click the little "X", the calendar event goes away from the UI:by Chris Strom (noreply@blogger.com) at September 05, 2011 07:32 PM
!!!Next, I create a very simple dialog in the Jade template:
html
head
title= title
link(rel='stylesheet', href='/stylesheets/style.css')
link(rel='stylesheet', href='/stylesheets/blitzer/jquery-ui.css')
script(src='/javascripts/jquery.min.js')
script(src='/javascripts/jquery-ui.min.js')
script(src='/javascripts/underscore.js')
script(src='/javascripts/backbone.js')
body!= body
#dialog(title="Add calendar event")I have the intention of eventually grabbing the values for appointments from the two dialog fields and from the
#calendar-event-start-date
p title
p
input#calendar-event-title(type="text", name="title")
p description
p
input#calendar-event-description(type="text", name="description")
#calendar-event-start-date <div> (which I will populate from the date clicked). But before I reach that point, I need to make this a jQuery-ui dialog:script
$(function() {
$('#dialog').dialog({
autoOpen: false,
modal: true,
buttons: [
{ text: "OK",
click: function() { $(this).dialog("close"); } },
{ text: "Cancel",
click: function() { $(this).dialog("close"); } }
]
});
});AppView Backbone View class:
window.AppView = Backbone.View.extend({
el: $("#dialog"),
events: {
'click .ok': 'create'
},
create: function() {
console.log("here");
Events.create({
title: "foo",
description: "bar",
startDate: "2011-09-01"});
}
});
window.AppView = new AppView;$('#dialog').dialog('open')And I am greeted with a right proper jQuery-ui dialog:
dialog() invocation. But a new Event is not create. Even the console.log() statement is not reached.el attribute to the dialog's parent: window.AppView = Backbone.View.extend({
el: $("#dialog").parent(),
// ...
});
script
$(function() {
$('#dialog').dialog({
autoOpen: false,
modal: true,
buttons: [
{ text: "OK",
class: "ok",
click: function() { $(this).dialog("close"); } },
{ text: "Cancel",
click: function() { $(this).dialog("close"); } }
]
});
});
app.post('/events', function(req, res){
var options = {
method: 'POST',
host: 'localhost',
port: 5984,
path: '/calendar',
headers: {'content-type': 'application/json'}
};
var couch_req = http.request(options, function(couch_response) {
console.log("Got response: %s %s:%d%s", couch_response.statusCode, options.host, options.port, options.path);
couch_response.pipe(res);
}).on('error', function(e) {
console.log("Got error: " + e.message);
});
couch_req.write(JSON.stringify(req.body));
couch_req.end();
});Aside from the headers and the write() of the JSON data to the CouchDB request, the remainder of this route looks very similar to stuff that I have been writing for GETs and DELETEs over the past few days. It may be time to investigate adding an abstraction layer in my express app. Another day, perhaps.by Chris Strom (noreply@blogger.com) at September 05, 2011 07:31 PM
Got response: 201 localhost:5984/calendarThere are, in fact two bugs here. The first is that my Backbone app is not sending the revision number of the newly created appointment when it comes time to delete the record. That is a somewhat understandable oversight on my part. What is not so OK is the lack of error handling that I have built. The frontend responded to the 409 as if nothing went wrong—the appointment was removed from the calendar as if nothing went wrong.
Got response: 409 localhost:5984/calendar/7acf98778a669f4d6fc33d6b3400e480
Got response: 200 localhost:5984/calendar/_all_docs?include_docs=true
app.delete('/appointments/:id', function(req, res){
var options = { /* Connection Options */ };
var couch_req = http.request(options, function(couch_response) {
console.log("Got response: %s %s:%d%s", couch_response.statusCode, options.host, options.port, options.path);
couch_response.pipe(res);
}).on('error', function(e) {
console.log("Got error: " + e.message);
});
couch_req.end();
});Interesting. I had expected the 409 response from CouchDB to be considered an error by node.js's http.request(). But I am not seeing the "Got error" message logged. I am seeing the "Got response" message:Got response: 409 localhost:5984/calendar/7acf98778a669f4d6fc33d6b3400e480Ah, looking at the http.request documentation, I see that:
If any error is encountered during the request (be that with DNS resolution, TCP level errors, or actual HTTP parse errors) an 'error' event is emitted on the returned request object.The failure here is not a connection error and not technically a parse error, so I suppose that the error event should not be fired after all.
HTTP/1.1 200 OKLooking at the actual body of the response, however, there clearly was an error:
X-Powered-By: Express
Connection: keep-alive
Transfer-Encoding: chunked
"error":"conflict","reason":"Document update conflict."}Well, I have the correct 409
statusCode in the couch_response already. It seems that the solution here is simple enough. I set the HTTP response from my express.js app to be that of the CouchDB response that I am proxying: // ...
var couch_req = http.request(options, function(couch_response) {
console.log("Got response: %s %s:%d%s", couch_response.statusCode, options.host, options.port, options.path);
res.statusCode = couch_response.statusCode;
couch_response.pipe(res);
}). // ...
error event from the model? window.AppointmentView = Backbone.View.extend({
initialize: function(options) {
this.container = $('#' + this.model.get('startDate'));
options.model.bind('destroy', this.remove, this);
options.model.bind('error', this.deleteError, this);
},
deleteError: function(model, error) {
// TODO: blame the user instead of the programmer...
if (error.status == 409) {
alert("This site does not understand CouchDB revisions.");
}
else {
alert("This site was made by an idiot.");
}
},
// ...
});
by Chris Strom (noreply@blogger.com) at September 05, 2011 07:29 PM
...The resultant HTML is then:
tr#week4
td.sunday
td.monday 22
td.tuesday
td.wednesday
td#2011-08-25.thursday
td#2011-08-26.friday
td.saturday
...
/events resource in my app returns:{"total_rows":2,"offset":0,"rows":[
{"id":"fdbed27594feb433c74e82eb910015e0",
"key":"fdbed27594feb433c74e82eb910015e0",
"value":{"rev":"2-b7c22d428e648a6cdd2978c213f79ec0"},
"doc":{"_id":"fdbed27594feb433c74e82eb910015e0",
"_rev":"2-b7c22d428e648a6cdd2978c213f79ec0",
"startDate":"2011-08-25",
"title":"create blog post",
"description":"talk about node and CouchDB"}},
{"id":"fdbed27594feb433c74e82eb91001f45",
"key":"fdbed27594feb433c74e82eb91001f45",
"value":{"rev":"1-2b18432cf6e63b82c6507ff28af9724c"},
"doc":{"_id":"fdbed27594feb433c74e82eb91001f45",
"_rev":"1-2b18432cf6e63b82c6507ff28af9724c",
"startDate":"2011-08-26",
"title":"blog again",
"description":"add backbone into the node + couch mix"}}
]}(this is just a pass-thru to CouchDB's _all_docs?include_docs=true)getJSON call inside a document-ready: $(function() {
$.getJSON('/events', function(data) {
$.each(data.rows, function(i, rec) { add_event(rec.doc) });
});
});add_event().add_event() function then exploits the fact that the cells in my calendar are identified with an ISO 8601 date: function add_event(event) {
var date = event.startDate,
title = event.title,
description = event.description;
$('#' + date).html(
'<span title="' + description + '">' +
title +
'</span>'
);
}startDate from CouchDB is 2011-08-26, then this function finds the correct cell via a jQuery $('#2011-08-26') selector. If that selector is found, then the inner HTML is replaced with the event's title (and the description in a <span> title attribute). If the calendar event is for a date not currently displayed, no worries, the selector returns an empty wrapped set, in which case the html() has nothing to do.
by Chris Strom (noreply@blogger.com) at September 01, 2011 01:14 PM
delete route in my express.js app. Nothing too fancy ought to be required. I just need to make an HTTP request with a method of "DELETE" to my CouchDB backend. The response from CouchDB can then be piped directly to my Backbone app. Something like this ought to do:app.delete('/events/:id', function(req, res){
var options = {
method: 'DELETE',
host: 'localhost',
port: 5984,
path: '/calendar/' + req.params.id
};
// Send the HTTP request with the DELETE options
var couch_req = http.request(options, function(couch_response) {
console.log("Got response: %s %s:%d%s", couch_response.statusCode, options.host, options.port, options.path);
// Pipe the response from CouchDB to the browser
couch_response.pipe(res);
}).on('error', function(e) {
console.log("Got error: " + e.message);
});
// Send the complete request.
couch_req.end();
});Now to my Backbone application. As usual when I am exploring, I use Chrome's Javascript console for interacting with page elements and Javascript objects. In this case, I would like to delete the Backbone model responsible for the "foo" calendar event on the first of next month:
Event model, all I need do is call its destroy method and the offending event should be stricken from existence:> e.destroy()Hrm... Dunno what I expected. I suppose it has to be chainable, so maybe it worked..? Actually, no, it did not. Examining the express.js app's log, I see no log entries. Reloading the page, I see that the calendar event is still there. So what gives?
=> child
Event model itself:window.Event = Backbone.Model.extend({});Say, that looks a bit spartan. Perhaps more is needed for Backbone to know how to delete a thing from the database./events) and a record ID. In retrospect, both make all kinds of sense. How else is Backbone supposed to infer the resource to be DELETEd?Event model looks like: window.Event = Backbone.Model.extend({
urlRoot : '/events',
initialize: function(attributes) { this.id = attributes['_id']; }
});urlRoot property is fairly self-explanatory. The initialize method for setting the model's id is less so. CouchDB stores document IDs in the "_id" attribute:{
"_id": "a38f51509190f265959bbb2b5d001128",
"_rev": "1-174f31204df52e79a92c1ac875ac09a2",
"startDate": "2011-09-01",
"title": "foo",
"description": "bar"
}This is available in the model's attributes (I code call event.get("_id") to retrieve it), but Backbone has no way to tie it to the special id property of a Backbone model. So I link the two manually in the model's initializer.
Got response: 409 localhost:5984/calendar/a38f51509190f265959bbb2b5d001128That is certainly progress, but 409?!
DELETE /calendar/a38f51509190f265959bbb2b5d001128?rev=1-174f31204df52e79a92c1ac875ac09a2 HTTP/1.0Or as an If-Match header:
DELETE /calendar/a38f51509190f265959bbb2b5d001128 HTTP/1.0Hrm... I tend to think it would be easier to transmit the revision via the
If-Match: "1-174f31204df52e79a92c1ac875ac09a2"
If-Match header. If I could set that in my Backbone application, then I ought to be able to pass that directly through to CouchDB:app.delete('/events/:id', function(req, res){
var options = {
method: 'DELETE',
host: 'localhost',
port: 5984,
path: '/calendar/' + req.params.id,
headers: req.headers
};
var couch_req = http.request(options, function(couch_response) {
// ...
couch_req.end();
});I rather like that one line change. No futzing with query parameters just feels cleaner.destroy (or update or create) method is sent along to jQuery as an option. Since jQuery AJAX requests recognize a headers attribute, something like this ought to work:> e.destroy({headers: {'If-Match':'1-174f31204df52e79a92c1ac875ac09a2'} })And, finally, I see a change in the CouchDB response:Got response: 200 localhost:5984/calendar/a38f51509190f265959bbb2b5d001128?rev=1-174f31204df52e79a92c1ac875ac09a2Most importantly, I no longer have a bogus entry on the first:
by Chris Strom (noreply@blogger.com) at September 01, 2011 01:11 PM
➜ repos express calendarIn the new "calendar" app directory, I need to install the express and jade packages from npm:
create : calendar
create : calendar/package.json
create : calendar/app.js
create : calendar/public/stylesheets
create : calendar/public/stylesheets/style.css
create : calendar/public/javascripts
create : calendar/public/images
create : calendar/views
create : calendar/views/layout.jade
create : calendar/views/index.jade
➜ calendar git:(master) ✗ npm install express jadeMy next step is to use the Futon admin interface to create a database. I fancy a calendar app, so I name my database accordingly:
jade@0.14.2 ./node_modules/jade
express-unstable@2.4.3 ./node_modules/express
├── mime@1.2.2
├── connect@1.6.0
└── qs@0.3.1
_all_docs resource in CouchDB to pull back all records in the calendar DB (I prolly would not do that in a larger DB). Once the Couch DB response comes back, I write the data back to the browser:app.get('/events', function(req, res){
var options = {
host: 'localhost',
port: 5984,
path: '/calendar/_all_docs'
};
http.get(options, function(couch_response) {
console.log("Got response: %s %s:%d%s", couch_response.statusCode, options.host, options.port, options.path);
res.contentType('json');
// Send all couch data to the client
couch_response.on('data', function (chunk) {
res.write(chunk);
});
// When couch is done, so is this request
couch_response.on('end', function (chunk) {
res.end();
});
}).on('error', function(e) {
console.log("Got error: " + e.message);
});
});That's a bit of work establishing 'data' and 'end' listeners. Fortunately, node has an answer for this case in the pipe method for all stream objects: http.get(options, function(couch_response) {
console.log("Got response: %s %s:%d%s", couch_response.statusCode, options.host, options.port, options.path);
couch_response.pipe(res)
})Any events emitted by couch_response will be sent to the original Response object. The result of calling accessing the /events resource is:
app.get('/events/:id', function(req, res){
var options = {
host: 'localhost',
port: 5984,
path: '/calendar/' + req.params.id
};
http.get(options, function(couch_response) {
console.log("Got response: %s %s:%d%s", couch_response.statusCode, options.host, options.port, options.path);
couch_response.pipe(res);
}).on('error', function(e) {
console.log("Got error: " + e.message);
});
});This route makes use of some nifty parameter assignments in express.js. Named parameters in the route (the :id in /events/:id) are made available in the request params object (e.g. req.params.id). That bit of coolness aside, this is nearly identical to the all-events route from above./events route and add include_docs=true to the URL. This includes the documents (which are relatively small) along with the meta data:
by Chris Strom (noreply@blogger.com) at September 01, 2011 12:52 PM
Aimee Daniells
Chris Strom
Christopher Lenz
Cloudant
Daniel Tsadok
EasyBib
Helder Ribeiro
IrisCouch
Jan Lehnardt
John Wood
Kanso
Klaus Trainer
Mark Headd
Maximillian Dornseif
Paul Joseph Davis
Randall Leeds
Rinat Abdullin
Rodrigo Moya
Stuart Langridge
Till Klampäckel
Volker Mische
Will Hartung
Last updated:
May 23, 2013 11:20 PM
All times are UTC.
Powered by:
