Build a multiwriter app with Autobase

Let several peers write to one shared, eventually consistent view with Autobase.

A Hypercore has exactly one writer: the peer that holds its key pair. Autobase lifts that restriction. Several writers each append to their own local core, Autobase linearizes those cores into one causal order, and an apply handler replays that order into a view—a derived Hypercore every peer rebuilds identically.

This guide uses Corestore to hold the cores and Hyperswarm to replicate them; see Work with many Hypercores using Corestore if those concepts are unfamiliar.

Pear-end logic—start from a boilerplate. The code in this guide lives in the Bare worker: peer-to-peer logic, no UI. The same worker runs unchanged on desktop, terminal, and mobile—only the shell differs (see Runtime and languages). Start from a boilerplate in Start from a template—desktop (hello-pear-electron) or terminal (hello-pear-bare)—and add this capability on top.

This guide consists of two applications:

base-writer-app - creates the Autobase, appends messages, and grants write access to other peers.
base-peer-app - joins the same Autobase by key, reads the view, and appends once it has been made a writer.

How Autobase differs from a plain Hypercore

Three ideas carry the rest of this guide:

Writers, not one writer. Every peer appends to its own local core. base.local.key identifies that core, and an existing writer grants access by naming it.
Linearization. Nodes reference the nodes they saw, forming a causal graph. Autobase orders that graph so no node precedes a node it references, and so all peers converge on the same order.
The view is derived, never written directly. You never append to base.view. You append to the base, and apply decides what lands in the view.

Autobase reorders nodes when new causal information arrives, undoing and reapplying the view. open and apply must therefore be deterministic and derive state only from the arguments they are given. A handler that reads a clock, a global, or the network will make peers diverge, and the undo path cannot roll those effects back.

Create the base writer app

The base-writer-app creates a new Autobase, announces it on Hyperswarm, and appends messages to it. It also accepts an add <writer-key> command that grants another peer write access.

Create the `base-writer-app` directory and add dependencies

Start the base-writer-app project with the following commands:

mkdir base-writer-app
cd base-writer-app
npm init -y
npm pkg set type="module"
npm install autobase corestore hyperswarm b4a bare-pipe bare-process

This will install the following dependencies:

autobase: A module for building multiwriter views over Hypercores.
bare-pipe: A module for working with pipes, used here to read stdin.
bare-process: A module for working with processes.
hyperswarm: A module for working with Hyperswarm.
corestore: A module for working with Corestore.
b4a: A module for working with buffers.

Add the `base-writer-app` logic

Create the base-writer-app/index.js file with the following content:

open builds the view from the store Autobase passes in (L13–L15), and apply reduces each batch of linearized nodes into it (L18–L33): a node carrying addWriter calls host.addWriter instead of being appended (L24–L27), every other node is appended verbatim (L28). The whole body runs inside a try/catch (L23,L29–L31): apply replays this node on every peer forever, so a malformed one—a typo, or a node from a hostile peer—must be skipped, never thrown. Passing null as the bootstrap argument creates a new base rather than loading one (L36). A Corestore replication stream is attached to every Hyperswarm connection (L39), and the key is printed only once the topic has been announced to the DHT (L43–L46)—advertising it earlier races a peer that looks the topic up immediately. The update event fires after each apply run, which is where the view is printed (L48). Reading stdin (L55–L64) splits input two ways: add <writer-key> appends an addWriter node (L58–L62), anything else appends a message (L63).

base-writer-app/index.js

import process from 'bare-process'
import Hyperswarm from 'hyperswarm'
import Corestore from 'corestore'
import Autobase from 'autobase'
import Pipe from 'bare-pipe'
import b4a from 'b4a'

const store = new Corestore('./base-writer-storage')
const swarm = new Hyperswarm()
process.once('SIGINT', () => swarm.destroy().then(() => process.exit(0)))

// Create the view. Derive it only from the store passed in, never from outside state.
function open (viewStore) {
  return viewStore.get({ name: 'chat', valueEncoding: 'json' })
}

// Reduce the linearized nodes into the view. Deterministic, and mutates only `view`.
async function apply (nodes, view, host) {
  for (const { value } of nodes) {
    // apply replays this node on every peer, forever, including on restart.
    // A malformed value (a bad typo, or a hostile peer) must not throw here —
    // that would crash every peer that ever processes it. Skip and move on.
    try {
      if (value.addWriter) {
        await host.addWriter(b4a.from(value.addWriter, 'hex'), { indexer: false })
        continue
      }
      await view.append(value)
    } catch (err) {
      console.error('skipping malformed node:', err.message)
    }
  }
}

// bootstrap is null, so this call creates a new Autobase rather than loading one.
const base = new Autobase(store, null, { valueEncoding: 'json', open, apply })
await base.ready()

swarm.on('connection', (conn) => store.replicate(conn))

// Announce the topic before advertising the key, so a peer that looks it up
// straight away finds this writer.
const discovery = swarm.join(base.discoveryKey)
await discovery.flushed()

console.log('base key:', b4a.toString(base.key, 'hex'))

base.on('update', () => printView())

await base.append({ from: 'writer', text: 'first message' })

const stdin = new Pipe(0)

// `add <writer-key>` grants write access. Anything else is appended as a message.
stdin.on('data', (data) => {
  const line = b4a.toString(data).trim()
  if (!line.length) return
  if (line.startsWith('add ')) {
    const key = line.slice(4).trim()
    base.append({ addWriter: key }).then(() => console.log('added writer:', key), console.error)
    return
  }
  base.append({ from: 'writer', text: line }).catch(console.error)
})

async function printView () {
  const lines = []
  for (let i = 0; i < base.view.length; i++) {
    const entry = await base.view.get(i)
    lines.push(`  ${i} ${entry.from}: ${entry.text}`)
  }
  console.log(`view (${base.view.length} entries):\n${lines.join('\n')}`)
}

{ indexer: false } adds the peer as a plain writer. Indexers additionally sign checkpoints, which is what advances base.signedLength and lets peers who are behind catch up without replaying everything. Promote a writer to indexer only once you trust it to stay online—a quorum of indexers must be reachable for the signed length to advance.

Run the `base-writer-app`

In one terminal, run base-writer-app with bare.

bare base-writer-app

It prints the base key and then the view, which starts with the one message the app appends on startup:

base key: be92e2a474c62c9b910d0849b592229971ba1b30d6bfa02fa6f4d819919eab7c
view (1 entries):
  0 writer: first message

Create the base peer app

The base-peer-app takes the base key as a command-line argument and joins the same Autobase. It has no write access at first, so it reads the view and waits.

Create the `base-peer-app` directory and add dependencies

Create the base-peer-app project with the following commands:

mkdir base-peer-app
cd base-peer-app
npm init -y
npm pkg set type="module"
npm install autobase corestore hyperswarm b4a bare-process

This will install the following dependencies:

autobase: A module for building multiwriter views over Hypercores.
bare-process: A module for working with processes.
hyperswarm: A module for working with Hyperswarm.
corestore: A module for working with Corestore.
b4a: A module for working with buffers.

Add the `base-peer-app` logic

Create the base-peer-app/index.js file with the following content:

open and apply are copied unchanged from the writer app (L17–L37)—both peers must derive the same view from the same nodes, and skip the same malformed ones rather than crashing on them. The base key arrives as a command-line argument (L7–L9) and is passed as the bootstrap argument, which loads the existing base instead of creating one (L40). base.local is this peer's own writer core; its key is what the writer app has to add (L44). After joining the swarm (L46–L47), the peer waits for the topic lookup to settle before drawing any conclusion from an empty swarm (L52), pulls what is available with base.update() (L53), then checks base.writable and, if it is still read-only, waits for the writable event before appending (L55–L62).

base-peer-app/index.js

import process from 'bare-process'
import Hyperswarm from 'hyperswarm'
import Corestore from 'corestore'
import Autobase from 'autobase'
import b4a from 'b4a'

const key = Bare.argv[2]

if (!key) throw new Error('provide a base key')

const store = new Corestore('./base-peer-storage')
const swarm = new Hyperswarm()
process.once('SIGINT', () => swarm.destroy().then(() => process.exit(0)))

// `open` and `apply` must match the writer's byte for byte, or the peers derive
// different views from the same nodes.
function open (viewStore) {
  return viewStore.get({ name: 'chat', valueEncoding: 'json' })
}

// Reduce the linearized nodes into the view. Deterministic, and mutates only `view`.
async function apply (nodes, view, host) {
  for (const { value } of nodes) {
    // apply replays this node on every peer, forever, including on restart.
    // A malformed value (a bad typo, or a hostile peer) must not throw here —
    // that would crash every peer that ever processes it. Skip and move on.
    try {
      if (value.addWriter) {
        await host.addWriter(b4a.from(value.addWriter, 'hex'), { indexer: false })
        continue
      }
      await view.append(value)
    } catch (err) {
      console.error('skipping malformed node:', err.message)
    }
  }
}

// Passing the bootstrap key loads the existing Autobase instead of creating one.
const base = new Autobase(store, b4a.from(key, 'hex'), { valueEncoding: 'json', open, apply })
await base.ready()

// base.local is this peer's own writer core. Its key is what the writer app adds.
console.log('writer key:', b4a.toString(base.local.key, 'hex'))

swarm.on('connection', (conn) => store.replicate(conn))
swarm.join(base.discoveryKey)

base.on('update', () => printView())

// Wait for the topic lookup to settle before deciding this peer is read-only.
await swarm.flush()
await base.update()

if (!base.writable) {
  console.log('read-only, waiting to be added as a writer')
  await new Promise((resolve) => base.once('writable', resolve))
}

console.log('now writable')

await base.append({ from: 'peer', text: 'hello from the second writer' })

async function printView () {
  const lines = []
  for (let i = 0; i < base.view.length; i++) {
    const entry = await base.view.get(i)
    lines.push(`  ${i} ${entry.from}: ${entry.text}`)
  }
  console.log(`view (${base.view.length} entries):\n${lines.join('\n')}`)
}

Run the `base-peer-app`

In another terminal, run the base-peer-app with bare and pass it the base key from the base-writer-app.

bare base-peer-app <SUPPLY THE BASE KEY HERE>

It prints its own writer key, replicates the view read-only, and waits:

writer key: 8def73b0a3e7382cef887fabec4b5d5332cf3c28da640c912a90be447075e7dc
read-only, waiting to be added as a writer

A peer that has not been added is a full reader: it replicates the base and rebuilds the view locally. Only appending is gated. If you want unknown peers to be able to write without a manual grant, use optimistic appends instead—but validate every optimistic block inside apply before calling host.ackWriter.

Grant write access

Copy the peer's writer key and, in the base-writer-app terminal, type:

add <SUPPLY THE WRITER KEY HERE>

The writer app appends an addWriter node. That node replicates to the peer, whose own apply runs the same host.addWriter call and reaches the same conclusion—the grant is part of the shared, replayable history, not a side channel. The peer's writable event then fires and it appends its message:

now writable
view (2 entries):
  0 writer: first message
  1 peer: hello from the second writer

Within a moment the same two entries, in the same order, appear in the base-writer-app terminal. Both peers have converged on one view built from two writers.

Type more messages into either terminal and watch both views update. Entries settle into an order that both peers agree on—if the two peers append while disconnected, the causal graph forks and Autobase merges it once they reconnect, which can move an entry that had already been displayed.

Where to go next

The view here is a raw Hypercore, which is the smallest thing that demonstrates linearization. Real apps usually open a Hyperbee in open instead, so apply can write keyed records and readers can query rather than scan. Key exchange is the other production gap: this guide passes keys through the terminal, whereas apps that pair over an invite use blind-pairing to hand the base key and the writer key over automatically.