Storage and Retention¶

vornik produces two kinds of data that grow over time: artifacts (files agents read and write) and operational history (task records, execution logs, cost rows). This guide covers where artifacts live, how to keep history from growing without bound, and how to back up and restore a deployment.

All settings here live in your daemon config (~/.config/vornik/vornik.yaml). See Configuration reference for the full key list.

Artifact storage¶

By default vornik stores artifacts on the local filesystem. You can instead point it at an S3-compatible object store — AWS S3, MinIO, or Ceph RGW — which is the better choice for durability and for deployments that may scale beyond a single host.

Filesystem (default)¶

storage:
  artifacts_path: /var/lib/vornik/artifacts
  # backend: "filesystem"   # default; may be omitted

AWS S3¶

storage:
  artifacts_path: /var/lib/vornik/artifacts   # still used for staging
  backend: "s3"
  s3:
    region: "us-east-1"
    bucket: "vornik-prod-artifacts"
    prefix: "vornik/prod"
    # Leave endpoint and credentials empty to use the AWS SDK's
    # default resolver and credential chain (IAM role, ~/.aws, env).

For production, prefer an IAM role over static credentials. Run the daemon under an instance profile that grants only the permissions it needs: s3:PutObject, s3:GetObject, s3:DeleteObject, and s3:ListBucket on your bucket and its contents.

MinIO (local development)¶

storage:
  artifacts_path: /var/lib/vornik/artifacts
  backend: "s3"
  s3:
    endpoint: "http://localhost:9000"
    region: "us-east-1"          # MinIO accepts any non-empty value
    bucket: "vornik-dev"
    access_key_id: "minioadmin"
    secret_access_key: "minioadmin"
    use_path_style: true         # required for MinIO
    force_ssl: false             # only safe for localhost development

Credentials via environment¶

You can leave the credential fields blank in YAML and supply them via environment variables instead — the recommended pattern for production:

export VORNIK_STORAGE_S3_ACCESS_KEY_ID="AKIAIOSFODNN7EXAMPLE"
export VORNIK_STORAGE_S3_SECRET_ACCESS_KEY="wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY"

Create the bucket first, and mind TLS

vornik does not create buckets for you. The bucket must exist before the daemon starts, or boot fails with a clear bucket not found error. Versioning, object lock, and lifecycle policies are yours to configure and are recommended for production.

force_ssl defaults to true and refuses plain-HTTP endpoints. Only set it to false for local MinIO development.

Retention¶

The retention sweeper prunes old operational data on a schedule. It is off by default — nothing is deleted until you enable it. Each category of data has its own age threshold in days.

retention:
  enabled: true            # default: false - opt in
  interval: "6h"
  task_llm_usage_days: 90  # cost history
  tool_audit_days: 180     # tool-call audit (never below 180, see below)
  tasks_days: 60           # finished tasks
  executions_days: 60      # finished executions
  artifacts_days: 60       # artifact records and their files
  task_messages_days: 30   # chat history (0 = keep with the parent task)
  memory_chunks_days: 0    # long-term memory (0 = keep forever)

What the defaults mean:

tool_audit_days never resolves below 180 days (six months), whatever the daemon or a project sets. That is the minimum the EU AI Act (Art 26(6)) asks deployers of high-risk AI systems to keep these logs. Raise the floor with tool_audit_floor_days. To keep them for less — because another law requires a shorter period, or because yours is not a high-risk deployment — set tool_audit_floor_override: true and say which in tool_audit_floor_override_basis; an override without a basis is ignored. The record of a connected assistant's refused calls (shown on its page under Admin → Assistants) is kept for the same time.
memory_chunks_days: 0 keeps long-term memory forever, which is the right posture for most deployments. Long-term memory is product data, not throwaway logs — set a finite value only if storage pressure forces it.
task_messages_days: 0 means chat history is pruned only when its parent task is pruned. Set it explicitly to trim chat faster than tasks.

Copies of conversations: diagnostic caches and upgrade dumps¶

Some stores hold copies of conversation text rather than the original. An erasure request does not reach them directly, so each one has a limit on how long it can keep a copy. These limits are always on while the sweeper is enabled; setting a key to 0 means "use the default", not "keep forever".

retention:
  blackbox_trace_cache_days: 1   # assembled task traces (default 1 day)
  project_gists_days: 7          # project summaries and their narrative (default 7)
  upgrade_dump_keep: 2           # database dumps left by vornik-update.sh
  upgrade_dump_grace_days: 14    # kept this long after their upgrade succeeded
  upgrade_dump_dir: ""           # where the backups are; empty = your home directory

Task traces (the black box's trace view) are a cache: a trace older than a day is rebuilt from the original records when you open it, so nothing is lost by removing it.
Project summaries are rebuilt every few minutes while a project's memory loop runs. A summary that has not been refreshed for a week, and a narrative that has not been rewritten for a week, are removed. A project page then shows its "no summary yet" state until the loop runs again.
Upgrade dumps. Each run of vornik-update.sh saves a full copy of the database in <home>/vornik-upgrade-backup-<date>/. The newest two are kept at most, and each is removed 14 days after the upgrade that wrote it succeeded. The newest dump of an upgrade that did not succeed is kept until a later upgrade runs: it is your way back. Only the dump file is removed; the previous binaries and the run's record stay, and the record notes when and why the dump went. vornik-update.sh applies this at the end of every successful upgrade, and the sweeper applies it between upgrades.

vornikctl retention upgrade-dumps            # preview
vornikctl retention upgrade-dumps --apply    # delete the dumps past the policy

vornikctl doctor reports dumps past the policy as upgrade_dumps.

With the sweeper off

With retention.enabled: false (the default) the task-trace cache and the project summaries keep their copies indefinitely. Upgrade dumps are still pruned, but only when an upgrade runs.

Per-project overrides¶

Any of these thresholds can be set on an individual project's config to override the daemon default. A common pattern is keeping cost data longer for billing while trimming chat sooner:

# In a project config:
retention:
  task_llm_usage_days: 365
  task_messages_days: 14

The minimum effective retention is one day. Setting a field to 0 means "inherit the default," not "delete immediately."

Recording a project's model exchanges¶

A project can opt in to keeping every model request and response of its agent steps, redacted, for replay and post-mortem:

# projects/<id>.yaml
recording:
  llm_exchanges: true    # default false

Off by default because it costs storage: one row per model call carrying the whole conversation at that point, so a 30-iteration step stores roughly 30 times its final context. The rows live exactly as long as their execution and are swept with it; vornikctl retention preview reports how many are waiting under llm_exchanges. Read them back with vornikctl execution exchanges <executionId> <stepId>, or export a replayable recording with --export. The switch is read at each model call, so a reload takes effect at a running step's next request.

Reading what a step was handed and what it handed back¶

Every agent step keeps, beside the prompt its model was first told, the two files at the container boundary: the task.json the daemon wrote for the container and the result.json the container wrote back. Both are stored after secret redaction (a [REDACTED:type] marker stands where a credential was, so do not read the stored input as exactly what the container saw where one appears), content-addressed, and pruned with the step outcome that references them — there is no separate retention setting. A file over 4 MiB is not stored; vornik_executor_step_io_skipped_total counts it.

vornikctl execution input  <executionId> <stepId> [--export task.json]
vornikctl execution result <executionId> <stepId> [--export result.json]

Bare JSON on stdout, so both pipe into jq; --export writes the file with mode 0600. A step run before the daemon kept these files answers "not recorded", and one whose file has since been pruned answers "gone".

Previewing and applying¶

vornikctl retention runs in preview mode by default — it shows what would be pruned without deleting anything. Add --apply to actually delete.

# Preview across every project (no deletes)
vornikctl retention

# Preview a single project
vornikctl retention --project personal-assistant

# Actually prune
vornikctl retention --project personal-assistant --apply

# Machine-readable output
vornikctl retention --json

Artifact pruning is permanent

Pruning artifacts removes both the database record and the underlying file (or object, on S3). It cannot be undone from within vornik — recovery means restoring from a backup. Run a preview first.

On SQLite¶

Retention and vornikctl retention work on both database drivers. On SQLite every table the SQLite schema has is pruned on the same windows as on PostgreSQL. Three differences:

Pruning project memory (memory_chunks_days, per-class TTLs) and the phone-approved idle-memory flow need PostgreSQL. Project memory itself is PostgreSQL-only, so there is normally nothing to prune; if old memory ever exists on SQLite, the sweep reports it as kept rather than deleting it.
Tables SQLite does not have (the response and embedding caches, the memory-eviction audit) are skipped.
SQLite runs without foreign-key enforcement, so since 2026.10.6 vornik applies the PostgreSQL cascade itself: deleting or pruning a task, execution or artifact also deletes or clears its child rows. Child rows left behind by deletes before 2026.10.6 are not cleaned up.
Since 2026.10.6, wiping an archived project (the grace-period wipe or delete-now) works on SQLite too, and a wipe that fails says so instead of reporting success.

Backup and restore¶

vornikctl backup captures a complete snapshot you can restore later.

# Auto-named archive in the current directory
vornikctl backup

# Explicit output path
vornikctl backup --out /backups/vornik-2026-05-19.tar.gz

The archive contains a full database dump, every artifact file (for the filesystem backend), and all project workspaces.

S3 backends are backed up separately

If you use the S3 artifact backend, vornikctl backup does not copy your S3 objects — back the bucket up with native S3 tooling (for example, aws s3 sync). The database dump and workspaces are still captured.

To restore, stop the daemon first, then:

vornikctl restore --from /backups/vornik-2026-05-19.tar.gz --force

The restore runs as a single transaction: it either imports completely or rolls back cleanly, never leaving a half-applied database.

As a safety check, restore refuses to overwrite a database that already contains projects or tasks, so you don't accidentally clobber a live system:

error: target database is not empty (1 project, 47 tasks)
       (pass --allow-non-empty to override)

If you genuinely intend to re-apply into a populated database, add --allow-non-empty.

Stop the daemon before restoring

Restoring into a database the daemon is actively using has undefined results. The --force flag is a contract that you have stopped the daemon, not a way to skip doing so.

Context budget and tier¶

Separate from disk storage, each chat session has a token budget — the model's context window. As a conversation fills that budget, vornik progressively tightens how many tools it exposes per turn, recovering prompt space exactly when it matters. This keeps long conversations working instead of failing once they approach the window limit.

This behavior only turns on when you tell vornik how large the context window is:

chat:
  # Set to your model's context window in tokens to enable tier-aware
  # tool management. 0 (the default) leaves it off.
  context_size: 200000

With this set, chat replies carry an X-Vornik-Context-Tier header (peak, good, degrading, or poor) that the UI uses to colour the session's context badge so you can see at a glance how much headroom remains.

If you want guaranteed full-tool turns, either raise the budget or shorten the conversation with /summarize.

Where to go next¶

Cost and caching — keep LLM spend predictable.
Memory and RAG — what long-term memory stores.
Configuration reference — every key and its default.

Approval before deleting unused project memory¶

For a personal knowledge project such as companion-personal, add this block to its project YAML in the configuration tree your daemon reads:

retention:
  memory_require_approval: true
  memory_idle_days: 365  # optional; 365 is the default in approval mode

Deploy the binary supporting these keys before adding them, then reload the configuration with vornikctl config reload. This repository change does not modify a running project's settings. Request production uses the existing daemon retention sweeper, which must be enabled (retention.enabled: true in the daemon config; default interval six hours). Pair an approver phone using the existing approval-device setup; pending requests appear in the inbox and on the paired phone's approval page. The operator alert channel sends a link when configured, without the source preview.

In this mode, automatic class-TTL and creation-age chunk deletion are disabled for the project. A final returned recall hit resets its idle clock, including memory used through context assembly. A never-used or historical chunk with no recorded use starts from its creation time. No history of old recalls is invented. The idle window is the greater of memory_idle_days and the chunk's stored original TTL duration, so an explicit longer ttl_days is preserved. Chunks with no expiry remain indefinite and do not receive deletion requests.

After the idle window, the daemon asks on the paired phone before deleting a specific chunk. The phone shows the project, source, bounded content preview and last-use date. Approve deletes that chunk and cleans its dependent links and graph provenance. Reject or leave unanswered to keep it; it remains searchable, and the daemon can ask again after 30 days. At most 20 requests per project are proposed each sweep. Only the paired-device principal can decide; API and companion keys cannot approve. A later recall, content replacement, TTL extension or retention-policy change invalidates the old deletion request.

vornikctl retention --apply also respects this protection; it does not file phone requests. With no available phone approval service, memory is retained. Explicit erasure and manual hard eviction remain separate actions. Disabling approval mode restores legacy TTL and age-based retention, so old expired chunks can become hidden or be removed by the next sweep. Back up the store before choosing that policy change.