Visibility & search

Visibility in Hopskip is a projection of the event log, not a separate database you keep in sync. Core continuously projects the log into an embedded store (SQLite) on Core nodes, and you read live workflow state back through a small set of typed queries.

There is no Elasticsearch, no separate visibility cluster, and no eventual-consistency lag beyond the projection pipeline itself.

Marking state searchable

Tag a workflow variable @searchable (or @indexed for deep state) and it is extracted at each checkpoint into structured rows. The value tracks the live workflow; mutate it and the next checkpoint re-extracts it.

from hopskip import durable, activity, Searchable, mark_searchable

@durable
async def order_processing(order):
    status = Searchable("status", "pending")       # re-extracted each checkpoint
    mark_searchable("customer_tier", order.tier)   # one-shot tag

    await activity("reserve_inventory", order.sku)
    status.set("reserved")

    await activity("charge_card", order.total)
    status.set("charged")

The same annotation exists in every SDK (@searchable in TypeScript, the Searchable type in Rust), reaching the same emit_searchable host import from idiomatic code in each language.

Reading it back

Every read is a typed query: the server, not the caller, decides which namespace it runs against. There is deliberately no endpoint that accepts a SQL string.

The console API

Route	Answers
`GET /api/workflows`	Every workflow instance with projected `@searchable` state, in your namespaces
`GET /api/workflows/{ns}/{wf}/state`	One instance’s current `@searchable` values
`GET /api/workflows/{ns}/{wf}/provenance`	One instance’s `@provenance` write history

Each read is scoped to the namespaces your token carries. A request for a namespace you are not a member of is refused (403) rather than returned empty. An empty result would be indistinguishable from “that workflow does not exist”, and you should be able to tell those apart.

GET /api/workflows/{ns}/{wf}/state returns each attribute’s current value. Every emission supersedes the last, so a workflow that passed through charged on its way to completed reads as completed. If you need a variable’s write history rather than its current value, that is what @provenance records (with its own envelope and sampling budget), and the log itself remains the authority for point-in-time reconstruction.

The CLI

hop logs reads a workflow’s projected log entries:

hop logs orders:order-1 --since-ms 1717000000000

physical_ms on each entry is HLC event time (the logical time workflows observe), not a projection wall-clock.

Because the columnar store is projected from the authoritative log, these results reflect live state, not a stale index.

There is deliberately no raw-SQL endpoint. Every read is a typed query so the server, not the caller, decides the namespace it runs against: with caller-written SQL, the namespace scoping would live in the query where the server cannot enforce it, and in a cluster arbitrary SQL gives a shard no partition dimension to filter on, so a stale ex-owner could answer for ranges it no longer holds.

Cluster-wide visibility

Queries that span shards (log reads, attribute lookups, workflow listings, SLO aggregates) are answered by scatter-gather across the owning Core nodes and merged. The projection lives with the data, so there is no central index to bottleneck on.

Each answer carries a per-shard freshness block, and a shard that was unreachable, whose epoch moved, or that could not show it had caught up is reported rather than quietly omitted, so a narrowed answer is never mistaken for a complete one. Because every request variant carries the hash range it is being asked about, a shard answers only for partitions it owns, which is what makes the merge safe to do at all.

Cold partitions that have aged out to object storage remain queryable in place, with no rehydration into Core.

`@searchable` vs `@indexed`

Annotation	Extracts	Use for
`@searchable`	A scalar value at each checkpoint	Status fields, tiers, small keys you filter and group by
`@indexed`	Deeper structured state	Nested state you need to query into, at higher projection cost

Prefer @searchable for the fields you filter and aggregate on; reach for @indexed only when you must query into nested structure.

Retention

Projected rows are not kept forever. When a completed workflow passes its namespace’s retention period it is purged from the log, and the same purge evicts its visibility rows: a projection row whose log is gone cannot be rebuilt, so leaving it behind would strand an unverifiable answer in the serving path. A purge that cannot evict the visibility rows refuses rather than proceeding.

Why this beats a separate index

No dual-write. The log is the source of truth; the projection can’t drift from it.
No operational dependency. There is no Elasticsearch cluster to run, scale, or recover.
Provenance preserved. Rows trace back to the Merkle-chained events that produced them.