Couchapps 1

‘
I’ve been trying to understand couchapps and how to write them.  The documentation is not entirely clear for someone like me who needs the big picture before getting into details.

So, we have an application, the couchapp, which is a single .json document called _design/[name].  That’s potentially quite a large document and writing it directly would be a chore.  Enter couchapp helper applications (couchapp.org).  These basically do one thing: they take all the various javascript, HTML and css files, build the _design/[name].json document and push it onto the couchdb.

I hadn’t quite got this at first.  I was looking at existing couchapp document trees trying to find the _design/[name] file.  The point is, there isn’t one, as such.  Now, it can be confusing to think of the application on the couchdb as a couchapp and the applications that create the _design/[name] document as couchapps too!

Even worse, there are a few couchapp helper applications and they don’t talk to each other.  Neither do they use the same directory structure of javascript, HTML and CSS files.  I’m not sure whether it is possible to install more than one on a device at one time, since they all use the word couchapp to run a command.

There is a couchapp helper application written in python (couchapp.py).  All of the Couchdb: Definitive Guide refers to that tool.  The sofa application referred to in the book is one that has to be pushed to the couchdb using couchapp.py.

There is also a couchapp helper application written in javascript (node.couchapp.js).  That is the one I’m using because it looks simpler, and I like simple.

The _design/[name] file structure is like this:

{ "_id" : "_design/myapp",
"_rev": "revision number created by couchdb"
"_attachments": { "images/logo.png": { "content_type": "image/png","revpos":1, 
"digest":"md5-GDPL+eLwE7kzEDWY7X4KdQ==","length":886,"stub":true } }, 
"lists": { "xml": "function..." }, 
"rewrites": "function...", 
"shows": { "preview": "function...", "xml": "function..." }, 
"updates": { "in-place": "function..." }, 
"views": { "foobar": { "map": "function...", "reduce": "function..." } }, 
"validate_doc_update": "function...", } 

I think this is the same structure whichever couchapp helper application is used, but documentation isn’t a strong point of couchdb projects.
The file tree structure for couchapp.py and node.couchapp.js are quite different.  I haven’t installed couchapp.py, so I’m not sure exactly what the boiler-plate framework looks like.  For node.couchapp.js this is what you get:

myapp/
    attachments/
    app.js

Couchdb definitive guide chapter 6 companion

.

I’m working through Chris Anderson’s Couchdb Definitive Guide.  I thought I would expand a little on the content of chapter 6.

The chapter starts with 3 documents: “biking”, “bought-a-cat”, “hello-world”.

First, I had to create the database:

curl -X PUT http://127.0.0.1:5984/basic

Then I copied and pasted document content from the guide into 3 .json documents (eg1.json, eg2.json, eg3.json) that I PUT to the database.  The content of the “biking” document looks like this.

{
  "title":"Biking",
  "body":"My biggest hobby is mountainbiking. The other day...",
  "date":"2009/01/30 18:04:11"
}

I asked couchdb for a UUID:

curl -X GET http://127.0.0.1:5984/_uuids

And received a new UUID as .json

{"uuids":["381795b3958bbdd47f1879e8ff000e84"]}

This allowed me to construct a PUT for each egX.json as follows (a new UUID was needed for each document):

curl -X PUT http://127.0.0.1:5984/basic/bd26423be3847a6b7b14e50f36048563 \
-d @egX.json -H 'Content-Type: application/json'

Or, you can let couchdb assign the UUID by using POST:

curl -X POST http://127.0.0.1:5984/basic -d @./CouchApps/egX.json \
-H 'Content-Type: application/json'

Next came the _design/example document.  I initially saved a version 1 and then updated to a version 2 _design/example document.  This is version 1:

{
  "_id" : "_design/example",
  "views" : {
    "foo" : {
      "map" : "function(doc){ emit(doc._id, doc._rev)}"
    }
  }
}

This was PUT to the database:

curl -X PUT http://127.0.0.1:5984/basic/_design/example --data-binary @mydesign.json

Couchdb automatically assigns a revision number to the document that appears within the .json.

And the function foo can be called using:

curl -X GET http://127.0.0.1:5984/basic/_design/example/_view/foo

The next step was to update _design/example with a new function called bar.  The revision number of _design/example was needed:

curl -X GET http://127.0.0.1:5984/basic/_design/example

And this was entered into the  _design/example document

{
  "_id" : "_design/example",
  "_rev" : "1-230141dfa7e07c3dbfef0789bf11773a",
  "views" : {
    "foo" : {
      "map" : "function(doc){ emit(doc._id, doc._rev)}"
    },
    "bar" : {
      "map" : "function(doc) {if(doc.date && doc.title){emit(doc.date, doc.title);}}"
    }
  }
}

_design/example was then updated

curl -vX PUT http://127.0.0.1:5984/basic/_design/example \
-d @./CouchApps/mydesign.json -H 'Content-Type: application/json'

In this example I have used the -d switch, which I think is synonymous with –data-binary, but if an error gets thrown, use –data-binary.  The Content-Type is also required.

Call the bar function like foo before:

curl -X GET http://127.0.0.1:5984/basic/_design/example/_view/bar

Installing couchapp

.

Having installed couchdb and geocouch the time was right to install couchapp!

First, I had to install node.js.  I cloned from github into /usr/local/src:

git clone https://github.com/joyent/node.git

Then entered the node directory and checked out a version that would install with make:

cd node
git checkout v0.6.3

I then configured and installed.  I installed into /usr/local:

./configure --prefix=/usr/local
make
sudo checkinstall

This installed node.js files throughout a number of the existing directories in the /usr/local directory.

Second was to install npm.  I cloned from github into /usr/local/src:

git clone https://github.com/isaacs/npm.git

then entered the npm directory and installed:

cd npm
sudo make install

npm files were installed within directories of the /usr/local directory.

Couchapp was installed globally:

sudo npm install couchapp -g

Installing to:

/usr/local/bin/couchapp
/usr/local/lib/node_modules/couchapp

I received this error:

The "sys" module is now called "util". It should have a similar interface

This was fixed by modifying 2 files:

./lib/node_modules/couchapp/main.js
./lib/node_modules/couchapp/node_modules/watch/main.js

By changing

sys=require('sys')

to

sys=require('util')

in both files.

I received this error when running couchapp to push for the first time:

Error: Cannot find module 'couchapp'

This was fixed by setting the following environment variable:

export NODE_PATH=/usr/local/lib/node_modules

Which I’ve also saved into the /etc/environment file.

Installing geocouch

Right, having installed couchdb everything went wrong when I installed geocouch.  But I have that fixed now, so I thought I would again make a note of what worked.

OS Ubuntu 11.10

couchdb 1.1.1

geocouch 1.1.x

First, I cloned geocouch from github:

git clone -b couchdb1.1.x  https://github.com/couchbase/geocouch.git

Note: select the correct branch for the version of couchdb running.  I think the default is master which is configured to work with couchbase.  So, selecting the branch (-b) is important.

Now, I originally cloned geocouch into the ~/Downloads directory.  Big mistake, since it subsequently became clear that Erlang couldn’t use the directory even when it was on its path.  So, I moved geocouch to /usr/local/src:

mv ~/Downloads/geocouch /usr/local/src/

Now comes the tricky bit.  In the ReadMe they use a <vanilla-couch> variable to designate the path to the couchdb source.  But, they also use it for the path to the /default.d directory of the installed couchdb, a different place – now that is confusing.

So, first set a local environment variable COUCH_SRC in the terminal.  The ‘make’ file uses this variable.  You don’t need to make this variable permanent, since it is only used when making geocouch.

export COUCH_SRC=/usr/local/src/apache-couchdb-1.1.1/src/couchdb

Go into the geocouch directory

cd /usr/local/src/geocouch

And run the make command

make

This creates a new directory called /build with lots of .beam files.

Now, Erlang needs to have these .beam files on its path.  Therefore, set the following environment variable:

ERL_FLAGS="-pa /usr/local/src/geocouch/build"

I edited the /etc/environment file to persist the ERL_FLAGS environment variable.  I think you need to login again to make the change.

Finally, geocouch has a geocouch.ini file here: … /geocouch/etc/couchdb/local.d/  Copy this into the /local.d directory of the installed couchdb (not the source).  I happen to have the couchdb installed at /opt/couchdb/.

sudo cp /usr/local/src/geocouch/etc/couchdb/local.d/geocouch.ini /opt/couchdb/etc/couchdb/local.d/

That’s it.

Geocouch comes with some tests to check that the install has worked.  First copy the test scripts into your installed couchdb:

cp /usr/local/src/geocouch/share/www/script/test/* /opt/couchdb/share/couchdb/www/script/test/

Then edit the couch_tests.js file to include these new tests.

sudo gedit /opt/couchdb/share/couchdb/www/script/couch_tests.js

Paste in:

loadTest("spatial.js"); 
loadTest("list_spatial.js"); 
loadTest("etags_spatial.js"); 
loadTest("multiple_spatial_rows.js"); 
loadTest("spatial_compaction.js"); 
loadTest("spatial_design_docs.js"); 
loadTest("spatial_bugfixes.js");

Start couchdb

sudo /etc/init.d/couchdb start

Then check

curl localhost:5984

Installing couchdb

OK, so I’ve spent most of the morning installing couchdb on my Ubuntu oneiric ocelot server.  Thought I would note down the steps so that I could remember how to do it next time!

The resources I’ve used include:

guide.couchdb.org/draft/source.html

till.klampaeckel.de/blog/archives/55-CouchDB-on-Ubuntu-on-AWS.html

1.  Download from couchdb.apache.org/downloads.html.  I used apache-couchdb-1.1.1.tar.gz.  With my Ubuntu the download is saved to Downloads directory in my home directory [~/Downloads].

2.  I then unpacked into the same Downloads directory:

tar -zxvf ~/Downloads/apache-couchdb-1.1.1.tar.gz

3.  CD into the newly extracted folder, in my case, apache-couchdb-1.1.1

4.  Couchdb installs 5 new directories: bin; etc; lib; share; var.  I wanted an obvious named directory to hold all of these.  So,

./configure --prefix=/opt/couchdb
make

5.  I used checkinstall to help management of the application.  There was an error creating sub-directories which was fixed by using the –fstrans option.

sudo checkinstall -y --fstrans=0 --pkgname=apache-couchdb --pkgversion=1.1.1
--maintainer=[name]@[address] --pakdir=/opt --pkglicense=Apache

6.  I created a user named ‘couchdb’.  But I ran into a problem when switching to the new user in the terminal, I didn’t know what the password was, and it wouldn’t accept an empty string.  So I created a password for the couchdb user.

sudo adduser --system --home /opt/couchdb/var/lib/couchdb --no-create-home
--shell /bin/bash --group --gecos "CouchDB" couchdb
sudo passwd couchdb

7.  I created 2 symlinks:

sudo ln -s /opt/couchdb/etc/init.d/couchdb /etc/init.d/couchdb
sudo ln -s /opt/couchdb/etc/logrotate.d/couchdb /etc/logrotate.d/couchdb

8.  I changed ownership of the new directories:

chown -R couchdb:couchdb /opt/couchdb/etc/couchdb
chown -R couchdb:couchdb /opt/couchdb/var/lib/couchdb
chown -R couchdb:couchdb /opt/couchdb/var/log/couchdb
chown -R couchdb:couchdb /opt/couchdb/var/run/couchdb

8.  I changed permission on the directories:

chmod -R 0770 /opt/couchdb/etc/couchdb
chmod -R 0770 /opt/couchdb/var/lib/couchdb
chmod -R 0770 /opt/couchdb/var/log/couchdb
chmod -R 0770 /opt/couchdb/var/run/couchdb

9.   Switching to the couchdb user on the terminal I started the newly minted couchdb database:

su couchdb
service couchdb start

10.  Running the test scripts in the futon administration app, I had four failures:

attachments

auth_cache

list_views

rev_stemming

The cause of this might have been using the localhost:5984/_utils URL.  These pass when using 127.0.0.1:5984/_utils

11.  However, in the meantime I decided to download a standalone version of couchdb and see if that would boot without errors in the tests!

So, over to https://github.com/iriscouch/build-couchdb

12.  I followed the clear instructions.  I had to install git, this was the only additional step.

sudo apt-get install git

13.  The rake step took ages, about 30 minutes, and now my standalone couchdb is a mammoth 500Mb directory.  The advantage is that this can be deleted to remove the install.  Starting the db isn’t so sweet:

build/bin/couchdb

Since control isn’t passed back to the terminal.  And I couldn’t figure how to stop it without closing the terminal!

14.  The same errors occurred on the tests in futon with this install; smelling a red-herring this is what caused me to investigate and conclude that these were not important.

Social BPM

A new book on Social BPM by the WfMC. I did enjoy Keith Swenson’s Mastering the Unpredictable last year, before I loaned (gave) it to my then boss.
Keith has recently blogged around the topic of his contribution to Social BPM.  The price of this new book is too hefty for me; I’ll wait and see if a few 2nd-hand copies begin appearing …

The team at Activiti are integrating ACM into their product. Activiti is now integrated with Alfresco and there is a link-up with Jive: social content management has been given some airplay. Looks like a good choice for those of us not working in a corporate to download and play with.

Situational Apps

I’ve been working as a contractor for the past 3 years or so.  I work as an interim manager in an information rich discipline unrelated to IT.  No business with whom I have had the pleasure of working has ever provided support for managing the copious information my function generates except for ubiquitous MS Office products.  These are big businesses and I usually operate as the functional head.

Over the course of my career I’ve spent countless days writing Access and Excel solutions.  Now that I spend my time contracting this situation hasn’t changed.   Whilst my poll of current UK plcs isn’t scientific I suspect my experience is multiplied across most of the UK.  I was at one of Europe’s largest and most productive plants of its type earlier this week and again the data for the function was stored on Excel spreadsheets.

I’d like to be able to simply and quickly develop the situational apps I need, but take them with me as I pass from contract to contract.  I think PaaS is getting to the point at which this might have become a possibility.  I can imagine developing the functionality I need with little or no heavy coding.  Leaving a solution with the client – a much better outcome than a spreadsheet on a local drive – and taking a clean solution for possible future use with new clients. I’ll build a library of my own situational apps.  Not SaaS, not an ISV, just developing the tools I use for the job.

I’m currently researching the various services to see if anything yet comes close to meeting my needs.

Start-ups and failure

I closed my software start-up in 2008.  Just as the storm of recession was hitting and I looked into the tea-leaves and saw PaaS, mobile, social, situational apps etc.  All of which make traditional applications, well, ‘traditional’.

Anyway, just cam across this post about why not to start a business.  I found myself nodding in agreement.

I would add that it is also very difficult to get back into mainstream employment.  I’ve been trying since 2008 … so I may have to ‘go again’ just because I’ve got Hobson’s choice and have nowhere else to go …

Google’s Wave

Took a peek at Google’s video presentation launching wave at I/O last week.

The engineering is hugely impressive – they demonstrated so many advanced features, all delivered through an HTML 5.0 compliant browser (Chrome, Firefox, Safari) (not IE),  that it reminded me of my visit to the Louvre or Musee d’Orsay last week when one has a room full of so many Van Gogh’s or Gaugin’s that one becomes ambivalent.

Available soon as open-source I think it certain that a rich ecosystem of extensions, wave servers and wave apps will appear and build on some of the radical ideas already built into the product.  Apparently the work of about 50 engineers over the past 2-years ‘down under’ just the name ‘wave’ deserves to make this a big success.

I spend some of my time building workflows with BPM.  Lots of forms, lots of email services etc.  These services, as well as polls, are already designed into the wave product.  There is roll-back/history, collaborative preparation of docs, amazing context specific spell check, real time translation, hooks for version control, in fact wave looks like a darn good entry point for ECM.  And this is in addition to the way email / IM has been recast with loads of great features.

For myself I’m interested in how I could leverage wave services within a workflow.  There is a platform here from which to build apps specific to particular functional need (the situational apps) since lots of the drudge communication glue is available out of the box.  Functional ‘waves’ available on separate wave servers dealing with specific needs interacting with individual / personal waves etc.  A wave field in other words.

However, in the UK, will Wave be enough to ween UK-plc off its addiction to Sharepoint/Outlook?  I’m not so sure.  Neither is there an appetite for open-source nor is there an effective counterpoint to the argument that MS Office users cannot learn to use much else.  Having said that, I hope the sheer quality of the creative engineering of Wave will begin to cause MS users to reflect the next time they are asked to cough up for that years fees.

BPM Training

Roberto Pasti has reminded me of forthcoming Intalio training in London in late April.  I’ve previously attended a couple of Intalio’s sessions and the intro 2-day session is recommended for those new to BPM – it’s not too vendor specific too quickly.

Here’s the link.