Berkeley DB Reference Guide: Transaction Protected Applications
Description: Berkeley DB: An embedded database programmatic toolkit.
Keywords: embedded,database,programmatic,toolkit,b+tree,btree,hash,hashing,transaction,transactions,locking,logging,access method,access methods,java,C,C++

### Berkeley DB Reference Guide: Transaction Protected Applications |

# Building transaction protected applications

Creating transaction-protected applications using the Berkeley DB access methods is quite easy. In almost all cases, applications use DBENV->open to perform initialization of all of the Berkeley DB subsystems. As transaction support requires all five Berkeley DB subsystems, the DB_INIT_MPOOL, DB_INIT_LOCK, DB_INIT_LOG and DB_INIT_TXN flags should be specified.

Once the application has called DBENV->open, it should open the databases it will use. Once the databases are opened, the application can make changes to the databases inside of transactions. Each set of changes that entail a unit of work should be surrounded by the appropriate txn_begin, txn_commit and txn_abort calls. Operations not surrounded by these calls will not be recoverable.

The Berkeley DB access methods will make the appropriate calls into the lock, log and memory pool subsystems in order to guarantee transaction semantics. When the application is ready to exit, all outstanding transactions should have been committed or aborted.

Databases accessed by a transaction must not be closed during the transaction. Once all outstanding transactions are finished, all open Berkeley DB files should be closed. When the Berkeley DB database files have been closed, the environment should be closed by calling DBENV->close.

It is not necessary to transaction-protect read-only operations, unless those transactions require repeatable reads. However, if there are update transactions present in the application, the reading transactions must still use locking, and should be prepared to repeat any operation (possibly closing and reopening a cursor) which fails with a return value of DB_LOCK_DEADLOCK.

Consider an application suite where multiple threads of control (multiple processes or threads in one or more processes) are changing the values associated with a key in one or more databases. Specifically, they are taking the current value, incrementing it, and then storing it back into the database. There are two reasons that such an application needs transactions (other than the obvious requirement of recoverability after application or system failure):

The first reason for transactions is that the application needs atomicity. Since we want to change a value in the database, we must make sure that once we read it, no other thread of control modifies it. For example, assume that both thread #1 and thread #2 are doing similar operations in the database, where thread #1 is incrementing records by 3, and thread #2 is incrementing records by 5. We want to increment the record by a total of 8. If the operations interleave in the right (well, wrong) order, that is not what will happen:

>     thread #1  read record: the value is 2
>     thread #2  read record: the value is 2
>     thread #2  write record + 5 back into the database (new value 7)
>     thread #1  write record + 3 back into the database (new value 5)

As you can see, instead of incrementing the record by a total of 8, we've only incremented it by 3, because thread #1 overwrote thread #2's change. By wrapping the operations in transactions, we ensure that this cannot happen. In a transaction, when the first thread reads the record, locks are acquired that will not be released until the transaction finishes, guaranteeing that all other readers and writers will block, waiting for the first thread's transaction to complete (or to be aborted).

The second reason for transactions is that when multiple processes or threads of control are modifying the database, there is the potential for deadlock. In Berkeley DB, deadlock is signified by an error return from the Berkeley DB function of DB_LOCK_DEADLOCK.

Here is an example function that does transaction-protected increments on database records.

>     void
>     increment(dbenv, dbp, key, increment)
>     	DB_ENV *dbenv;
>     	DB *dbp;
>     	DBT *key;
>     	int increment;
>     {
>     	DBT data;
>     	DB_TXN *tid;
>     	int ret;
>     	char buf[64];

>     	/* Initialization. */
>     	memset(&data, 0, sizeof(data));

>     	/* Abort and retry the modification. */
>     	if (0) {
>     retry:		if ((ret = txn_abort(tid)) != 0)
>     			db_err(dbenv, ret, "txn_abort");
>     		/* FALLTHROUGH */
>     	}

>     	/* Begin the transaction. */
>     	if ((ret = txn_begin(dbenv, NULL, &tid)) != 0)
>     		db_err(dbenv, ret, "txn_begin");

>     	/* Get the key. */
>     	switch (ret = dbp->get(dbp, tid, key, &data, 0)) {
>     	case DB_LOCK_DEADLOCK:
>     		goto retry;
>     	case 0:
>     		break;
>     	case DB_NOTFOUND:
>     		if (txn_commit(tid))
>     			db_err(dbenv, ret, "txn_commit");
>     		return (0);
>     	}

>     	/* Get the current value, add the increment. */
>     	(void)sprintf(buf, "%d", atoi(data.data) + increment);
>     	data.data = buf;
>     	data.size = strlen(buf) + 1;

>     	/* Store the new value. */
>     	switch (ret = dbp->put(dbp, tid, key, &data, 0)) {
>     	case DB_LOCK_DEADLOCK:
>     		goto retry;
>     	case 0:
>     		break;
>     	default:
>     		(void)txn_abort(tid);
>     		db_err(dbenv, ret, "dbp->put");
>     	}

>     	/* The transaction finished, commit it. */
>     	if ((ret = txn_commit(tid)) != 0)
>     		db_err(dbenv, ret, "txn_commit");

>     	return (0);
>     }

In applications supporting transactions, all Berkeley DB functions have an additional possible error return: DB_LOCK_DEADLOCK. In the sample code, any time the Berkeley DB function returns DB_LOCK_DEADLOCK, the transaction is aborted (by calling txn_abort, which releases all of its resources), and then the transaction is retried from the beginning. There is no requirement that the transaction be attempted again, but that is the common course of action for an application.

There is one additional error that transaction-protected applications need to handle, and which is not shown in the above example. There exists a class of errors that Berkeley DB considers fatal to an entire Berkeley DB environment. An example of this type of error is a log write failure due to the disk being out of free space. The only way to recover from these failures is for the application to exit, run recovery of the Berkeley DB environment, and re-enter Berkeley DB. It is not strictly necessary that the application exit, but that is the only way to recover system resources such as file descriptors and memory.

When this type of error is encountered, the error value DB_RUNRECOVERY is returned. This error can be returned by any Berkeley DB interface. If a fatal error occurs, DB_RUNRECOVERY will be returned from all subsequent Berkeley DB calls made by any threads or processes participating in the environment.

Optionally, applications may also specify a fatal-error callback function using the DBENV->set_paniccall function. This callback function will be called with two arguments: a reference to the DB_ENV structure associated with the environment, and the errno value associated with the underlying error that caused the problem.

Applications can handle such fatal errors in one of two ways: by checking for DB_RUNRECOVERY as part of their normal Berkeley DB error return checking, similarly to DB_LOCK_DEADLOCK in the above example, or, in applications that have no cleanup processing of their own, by simply exiting the application when the callback function is called.

Copyright Sleepycat Software