Autobee

Autobee gives a Hyperbee many writers. Each peer appends to its own local Hypercore; Autobee linearizes those cores into one causal order and replays it through your apply handler, which writes into a shared B-tree view. Every peer that replicates the same nodes derives the same view.

It sits alongside Autobase rather than replacing it. Autobase linearizes into a view of any shape you open; Autobee is the case where that view is always a Hyperbee, with the B-tree write path built in.

The apply handler

Treat apply like a pure reducer: given an ordered batch of nodes and a writable view, derive the next view state deterministically. Mutate only the view passed into apply. Do not:

• read or write external globals,
• open network connections, or
• assume ordering that Autobee has not yet committed. Side effects belong outside the linearization path.

Autobee reorders previously seen nodes when new causal information arrives, undoing and reapplying the view. A non-deterministic apply makes peers diverge, and the undo path cannot roll back effects it never saw.

Node values are raw bytes. Autobee has no valueEncoding, so apply decodes whatever your writers encoded:

async function apply (nodes, view, host) {
  for (const node of nodes) {
    const op = JSON.parse(b4a.toString(node.value))

    if (op.addWriter) {
      await host.addWriter(op.addWriter)
      continue
    }

    const batch = view.write()
    batch.tryPut(b4a.from(op.key), b4a.from(op.value))
    await batch.flush()
  }
}

Install

npm i autobee

Quickstart

import Corestore from 'corestore'
import Autobee from 'autobee'
import b4a from 'b4a'

const store = new Corestore('./autobee-demo')
const db = new Autobee(store, null, {
  async apply (nodes, view) {
    for (const node of nodes) {
      const op = JSON.parse(b4a.toString(node.value))
      const batch = view.write()
      batch.tryPut(b4a.from(op.key), b4a.from(op.value))
      await batch.flush()
    }
  }
})

await db.ready()
await db.append(b4a.from(JSON.stringify({ key: 'greeting', value: 'hello' })))
await db.update()

const entry = await db.view.get(b4a.from('greeting'))
console.log(b4a.toString(entry.value))
// hello

API Reference

Constructor and lifecycle

const db = new Autobee(store, key, options)

src

Create an Autobee. Passing key joins an existing database; omitting it or passing null creates a new one. options may be passed as the second argument when there is no key.

await db.ready()

Resolves once the database and its view are open.

• Returns: Promise<void>

await db.close()

Closes the database, its view, and the sessions it opened on store.

• Returns: Promise<void>

Identity

db.key

src

The public key of this Autobee. Share it so other peers can join.

• Returns: Buffer

db.discoveryKey

src

The discovery key, used as the swarm topic.

• Returns: Buffer

db.id

src

db.key as a z-base-32 string — the printable form to hand to peers.

• Returns: string

db.local

src

This peer's own writer Hypercore. db.local.key or db.local.id is what an existing writer passes to host.addWriter to grant write access.

• Returns: Hypercore

State

db.writable

src

Whether this instance has been added as a writer. Listen for db.on('writable', ...) to be notified when this flips, rather than polling it.

• Returns: boolean

db.isIndexer

src

Whether this writer is an indexer.

• Returns: boolean

db.view

src

The readable view, updated after each apply cycle. Without an open handler this is the Hyperbee snapshot itself, so read it with the Hyperbee API — view.get(key), view.createReadStream(), view.peek(range).

db.bee

src

The underlying Hyperbee snapshot. This is what open receives, so it is the same object as db.view only when no open handler is set — with one, db.view is whatever that handler returned and db.bee is still the raw B-tree.

db.stats

src

Counters for the current session: { undos, fastForwards, drains, applies, appends }. Useful for confirming that apply cycles and reorders are happening.

• Returns: object

db.optimistic

src

Whether optimistic appends are accepted.

• Returns: boolean

db.encrypted

src

Whether writer cores and the view are encrypted at rest.

• Returns: boolean

Writes and linearization

await db.append(value, options)

src

Append one value, or an array of values as one batch, to the local writer and trigger an apply cycle. Strings are converted to buffers; anything else must already be a buffer, so encode structured values yourself.

• Returns: Promise<void>
• Throws:
  • Autobee closed if the database is closing.
  • Not writable if the local writer has been removed and the append is not optimistic.

await db.update()

src

Run an apply cycle over everything currently available. Call it after replicating to process what arrived.

• Returns: Promise<void>

await db.updated()

src

Resolves once the in-flight apply cycle has finished, or immediately if none is running.

• Returns: Promise<void>

await db.flush()

src

Resolves once the database has finished booting online — that is, once known writers have been indexed.

• Returns: Promise<void>

Replication

const stream = db.replicate(isInitiatorOrStream, options)

src

Create a replication stream. Arguments match store.replicate(), and the stream is also registered with the wakeup protocol.

const swarm = new Hyperswarm()
swarm.join(db.discoveryKey)
swarm.on('connection', (conn) => db.replicate(conn))

await db.wakeup({ key, length })

src

Tell the database that the writer core key has at least length entries, then run a cycle. Use it when you learn about a writer out of band.

• Returns: Promise<void>

db.hintWakeup(wakeup)

src

Queue a wakeup hint, or an array of them, without awaiting a cycle.

const cores = await db.cores(options)

src

Added in v2.2.0. Report which cores a mirror should pin so it can replicate this database without knowing its internal structure — the local writer, this peer's views if it is trusted, and (with wait) the most recently trusted head, so a mirror that pins only what this call returns still lands on a valid entrypoint.

• Returns: Promise<object> — resolves to { key, views, writers }: key is db.key, writers is an array of writer core keys, views is an array of view/system core keys.

Fast-forward

Fast-forward lets a peer that is far behind jump onto a head another peer vouches for, instead of replaying everything. It deals only in oplog heads — { key, length } of a writer's core.

await db.moveTo(head)

src

Fast-forward onto head, skipping the usual distance and conservative checks.

• Returns: Promise<object> — resolves to { to, from }.

db.getMostRecentHead()

src

The oplog head this peer most recently vouched for.

Local writer

await db.setLocal(key, options)

src

Rotate the local writer onto a different key; the new core takes over as the active oplog. Pass { keyPair } to supply its signing keys. Emits rotate-local-writer.

• Returns: Promise<void>

Static methods

Autobee.isAutobee(value)

src

Whether value is an Autobee instance.

• Returns: boolean

const buf = Autobee.encodeValue(value, opts)

src

Encode a value into an Autobee block with its metadata.

• Returns: Buffer

const value = Autobee.decodeValue(buf, opts)

src

Decode an Autobee block back to its value.

Autobee.GENESIS

src

{ key: null, length: 0 } — the empty head representing the genesis state.

Events

db.on('update', () => { ... })

src

Emitted after an interrupt, alongside interrupt, and separately whenever the local writer's writable state flips (see db.on('writable', ...) below) — src. It is not a general view-changed signal for an ordinary apply cycle; use the update(view, changes) handler option for that.

db.on('writable', () => { ... })

src

Emitted when this instance transitions from not-writable to writable — that is, when db.writable flips to true. Fired from ActiveWriters._updateLocalState(), which runs as part of ordinary writer-set processing during an apply cycle, not only from an interrupt.

db.on('unwritable', () => { ... })

src

Emitted when this instance transitions from writable to not-writable — the counterpart to 'writable', fired by the same _updateLocalState() check.

db.on('interrupt', (reason) => { ... })

src

Emitted when host.interrupt(reason) is called inside apply.

db.on('error', (err) => { ... })

src

Emitted when applying fails. With no listener attached the process crashes deliberately, so attach one.

db.on('rotate-local-writer', () => { ... })

src

Emitted after db.setLocal() swaps the local writer.

db.on('move-to', (to, from) => { ... })

src

Emitted after a fast-forward, with the heads landed on and left behind.

AutobeeHostCalls

The host argument passed to apply. Its calls change the writer set, and because they run from inside apply every peer reaches the same result from the same nodes.

host.addWriter(key, options)

src

Add a writer. key may be a buffer, a hex string, or a z-base-32 id.

host.removeWriter(key)

src

Remove a writer.

host.ackWriter(key)

src

Acknowledge a writer without changing its permissions. Required for an optimistic block to be applied.

host.interrupt(reason)

src

Stop the current apply cycle and emit interrupt with reason. An escape hatch for when apply meets a node it cannot handle — a newer block type, say — so the peer stops rather than diverging.

• Throws: only callable inside apply; asserts otherwise.

const anchor = await host.createAnchor(key, length)

src

Create a verifiable checkpoint for the node at key/length, which future writers can use to prove causal ordering.

• Returns: Promise<object> — resolves to { key, length } of the anchor core.
• Throws: Anchor node is not in system if that node is not in the batch being applied, or not yet in the system core.

host.genesis

src

Whether the system has processed no nodes yet. Use it to bootstrap the first writer.

• Returns: boolean

host.clock

src

The system flush count — a monotonic counter of committed apply cycles.

• Returns: number

host also mirrors id, key, discoveryKey, and name from the database.

Types

AppendOptions

Options for db.append().

FastForwardOptions

The fastForward option. Pass false instead of an object to disable fast-forwarding entirely.

UpdateChanges

The changes argument passed to the update handler. changes.get(name) returns a record for 'view' or 'system':

See also

• Build a multiwriter database with Autobee—two-peer walkthrough covering apply, the update hook, and granting write access.
• Autobase—the general multiwriter linearizer, for views that are not a Hyperbee.
• Hyperbee—the single-writer B-tree whose read API db.view exposes.
• Corestore—storage and replication manager for the system, writer, and view cores.
• Hypercore—append-only log primitive that each writer appends to.