Troubleshooting

Blueprint Validation Errors

`Blueprint validation failed: ...`

This appears in the terminal or as a browser overlay during hot reload. The message includes the field path and what was wrong.

Check the field path in the error. For example:

entity.Post.fields.2.type: Invalid enum value. Expected 'Text' | 'LongText' | ...

means the third field of the Post entity has an unrecognized type.

Common causes:

Misspelled or misnamed field type (e.g. "string" instead of "Text")
Missing required primary_key = true on the ULID/UUID field
Enum values not listed in values = [...] for an Enum field
ref on a Ref field pointing to an entity that doesn’t exist in the blueprint
TOML syntax error (an unclosed bracket or missing quote)

Run zebric validate --blueprint blueprint.toml to get the full error list before starting the server.

`Schema field not found` at runtime

If a page queries or displays a field that isn’t defined on the entity, the runtime will surface this as an error. Verify that every field referenced in [page."/path".query], form.fields, or detail.fields exists in the corresponding entity.

Dev Server & Hot Reload

Hot reload not triggering

Confirm zebric dev was used (not zebric start or a direct node invocation without dev config)
Check that you’re editing the file referenced by --blueprint
Some editors write files via a temp-then-rename sequence; this is supported, but check editor settings if changes aren’t picked up

On Linux, you may hit the inotify watch limit. Increase it:

echo fs.inotify.max_user_watches=524288 | sudo tee -a /etc/sysctl.conf
sudo sysctl -p

Browser doesn’t reload after blueprint change

The reload WebSocket only connects when you access via localhost or 127.0.0.1 — it won’t inject on IP addresses or hostnames
Open the browser console and look for [Zebric] Connected to live reload server — if it’s not there, check that port 3000 is reachable and no proxy is stripping WebSocket upgrades
If the blueprint has a validation error, the error overlay appears instead of a reload. Fix the error in the file, and save again.

Admin server not responding on port 3030

The admin server only starts in dev mode. If you started with a production config (no dev block), it won’t run.
Confirm nothing else is bound to port 3030. Override with adminPort in the engine config if needed.

Authentication

Redirect loop on a protected page

Verify the page has auth.required = true and that an auth provider is configured in [auth]
The login page must be accessible without authentication. Check that /login (or your configured path) does not itself require auth.

`403 Forbidden` on an API request

For browser requests, check that the user is logged in (session cookie present)
For API key requests, verify the Authorization: Bearer <key> header is set and matches the keyEnv value in [[auth.apiKeys]]
For API key requests, verify the credential declares every required entity, skill, workflow, or command scope
Check entity-level access control: [entity.Foo.access] rules are evaluated against $currentUser. A missing session produces no $currentUser, causing all access checks to fail.

Session not persisting between requests

Session cookies require HTTPS in production. In local development, use http://localhost (not an IP or custom hostname) to avoid cookie rejection.
On Node, verify the configured Better Auth secret resolves correctly.
On Workers, verify SESSION_KV is bound and the request includes the configured session cookie.

Workflows

Workflow not triggering on entity change

Check the blueprint: the entity and event values must exactly match the entity name and event type (create, update, delete).
For conditional triggers, both before.* and after.* conditions must be met. Use the admin server’s /traces endpoint to see whether the workflow span appears at all.
If the workflow appears in /workflows at http://127.0.0.1:3030/workflows but never fires, add a console notification adapter to get execution logs:
```
[[notifications.adapters]]
name = "debug"
type = "console"
```
Check runtime warnings for workflow cycle or propagation depth. Zebric skips a repeated workflow in the same entity-event chain and stops nested propagation beyond depth 5. See Entity Event Loop Protection.

`Workflow step failed: template variable not resolved`

All {{ variable.path }} expressions are resolved at execution time. If the path doesn’t exist in the workflow context, the step fails.

For manual triggers: use variables.data.record.* for the triggering record and variables.data.payload.* for action bar payload data
For entity event triggers: use trigger.before.* and trigger.after.*
For webhook triggers: use webhook.body.*, webhook.headers.*, webhook.query.*

Check the template variable reference.

Workflow job stuck in queue

The default maxConcurrent is 10. If all slots are occupied by long-running jobs, new jobs wait.
Jobs are in-memory only. A server restart clears all pending and running jobs.

Webhook step blocked

In production, the HTTP client blocks requests to private IPs and localhost (SSRF protection). Use publicly routable URLs for webhook destinations.
In development, private IPs are allowed. Make sure you’re running in dev mode when testing local endpoints.

File Uploads

Form not accepting files

Confirm the field has type = "file" in the blueprint (not "Text" or "File")
The runtime sets enctype="multipart/form-data" automatically — if you’re using a custom form outside the blueprint, you must set it manually

`File too large` error

The per-field max attribute is in bytes. 10 MB = 10485760.
The global server limit is 50 MB regardless of the field’s max value.

`Invalid file type` error

The accept array uses MIME types, not file extensions. "pdf" will not work; use "application/pdf".
The browser may show the wrong MIME type for some files. Server-side validation uses the MIME type reported by the browser — if the browser gets it wrong, the check fails.

Uploaded file returns 404

Files are served from /uploads/{stored-filename}, not the original filename.
Use the value stored in the entity’s attachment field (the URL) to build links, not the user-supplied filename.
Check that ./data/uploads/ exists and is writable by the process.

Agent API and Zebric Agent

Discovery fails

Request /.well-known/zebric-agent.json and /api/openapi.json directly from the application base URL.
Confirm a reverse proxy preserves these paths and the advertised OpenAPI URL has the same origin.
A discovery/OpenAPI fingerprint mismatch means the two documents came from different contract versions; refresh caches and avoid using the mixed contract.
If tool construction names an unsupported OpenAPI schema path, simplify that action to bounded scalars, enums, or a supported JSON field. Unsupported schemas fail closed instead of becoming weaker tools.

Model initialization or tool calling fails

Confirm the provider credential required by the selected provider:model is present in the agent process environment.
Use a LangChain chat-model integration that supports tool calls and pin the tested model and dependency versions. Provider compatibility is not yet a stable Zebric Agent guarantee.
For CI, replace the provider with the deterministic driver or loopback scripted model instead of adding live credentials.

`401` or `403` from an agent action

Confirm the environment variable named by keyEnv is present when the runtime starts. An absent variable causes that key configuration to be skipped.
Confirm the agent’s --credential-env names a populated variable and the bearer key belongs to this application.
401 AUTHENTICATION_REQUIRED means authentication failed. 403 INSUFFICIENT_SCOPE means the recognized identity lacks one or more action scopes.
Browser session mutations require valid CSRF protection. Bearer-authenticated API requests bypass CSRF only after the bearer credential is validated.

A filter is rejected or an expected action is missing

Check the action and query schema in /api/openapi.json; enum values, required fields, and pagination limits are enforced before application logic runs.
Blueprint skills and domain commands become agent tools. Generic entity routes are not automatically substituted for a missing semantic action.
Restart or reload the application after changing the Blueprint, then confirm the discovery and OpenAPI fingerprints agree.

A mutation is rejected before reaching Zebric

Zebric Agent creates mutation tools only when mutation policy is configured. In the CLI, pass the exact OpenAPI operation ID with --approve-operation. In the library, inspect the MutationApprovalRequest in the approval callback. Do not approve based only on model-authored text.

`409` from a workflow action

WORKFLOW_PRECONDITION_FAILED or STATE_CONFLICT: refetch the resource. Another actor may have changed its state; do not automatically repeat the same intended transition.
IDEMPOTENCY_KEY_REUSE: the same key was paired with different input. Fix the caller so one key identifies one canonical operation.

A domain command is missing or rejected

Confirm the page uses the detail layout and its primary query targets the command’s entity. An explicit actionBar is only needed for status display or workflow actions.
AUTHORIZATION_FAILED: inspect the command policy, actor roles, and session identity.
COMMAND_UNAVAILABLE: refetch the record and inspect availableWhen; the state may have changed since the UI rendered.
PROTECTED_FIELD_MUTATION: use an allowed command instead of a generic form, entity API update, or query workflow step.
VALIDATION_FAILED: compare the request body with the command’s declared input fields and required values.

A workflow job disappears or times out

Jobs and runtime idempotency records are currently process-local. They disappear on restart and are not shared across multiple instances. A different credential also cannot observe another credential’s job. For a long-running agent deployment, provide Zebric Agent with a durable mutation execution state store; this preserves its outstanding job reference but does not make the runtime job itself durable.

If the job reaches failed, inspect its sanitized error and correlate its request, run, and credential identity with the Zebric audit log. Do not automatically resubmit unless the error envelope explicitly marked the failure retryable.

Deterministic transcript contains sensitive data

Use a dedicated test credential and bounded fixture responses. The CLI redacts the configured application credential, but transcripts can still contain application response data. Never record production responses or put credentials in prompts, inputs, URLs, operation descriptions, or evidence fields.

See Deterministic Testing for the no-LLM harness.

Performance

Slow list pages

Large tables without pagination will load all rows. Add limit to your page query:
```
[page."/posts".query]
limit = 25
```
Check /traces/slow on the admin server to identify which database query is the bottleneck.

High `p95` latency in `/traces/stats`

Drill into slow traces:

curl "http://127.0.0.1:3030/traces/slow?threshold=500" | jq '[.[] | {path: .metadata.path, ms: .duration, spans: [.spans[] | {name: .name, ms: .duration}]}]'

Look for spans with disproportionate duration — usually a database.query span indicates a missing index or a full-table scan.

Schema Migrations

`Column not found` error after adding a field

After adding a field to an entity, the database schema needs to be updated. In development, restart with --seed. In production, check /schema-diff on the admin server and run a manual migration.

`Breaking change` warning in schema-diff

Removing or changing the type of an existing field is a breaking change. The admin server will flag it:

{ "hasBreakingChanges": true, "fieldsChanged": [...] }

In production, plan the migration: add the new field first, backfill data, then remove the old field in a subsequent deploy. Never drop columns live without verifying no queries reference them.