The bridge, interviews, web capture, storage

Linking topics to the model, interviews, safe web capture, and where files and jobs live.

The bridge to the data model#

A topic can point at model entities (and fields): the Pro plan is a kind of Product; a refund policy governs Invoice. The pointer is stored inside the topic's collection, never in the model. The model gets no new entity and no new relationship, and only people and tokens that can read the collection ever see the link: if you cannot read it, there is nothing to see and nothing broken.

• Made while reading. The analyzer is shown the model's entities (name, domain, a short definition: structure only, never data or field details; a model of more than 200 entities is first narrowed to the ones that look like the source) and says, for each topic, which entities it is about, with a relation (is a kind of / is about / governs / concerns) and a confidence. A document about one product becomes a topic already linked to the product entity, with no extra model call. It chooses every entity the topic is genuinely about (usually one or two, a safety limit of 12). At least 70% counts at once; less waits as suggested until a person accepts it. A rejected link is not proposed again.
• By hand. Topic page > In the data model > Link to an entity; fields can be linked too, but never personal, confidential or restricted ones. Suggest with AI runs the analysis on an older topic.
• Where it shows. The topic page; the entity page (Related documents, grouped by collection with its usage tag); the model graph (tick Show the topics of my document collections); and for agents, library_search and library_topic list the entities a topic is about, describe of an entity adds related_documents, and plan_query lists the relevant topics.
• Links are resolved against the published model when read, with the reader's own entitlements; one whose entity has left the model is flagged. What the model says a topic is (its is a kind of links) is shown as the topic's kind next to the collection's own topic type.

Interviews#

Knowledge > Interview captures what a person knows by asking them questions, one at a time, the way a curious colleague would. You say what you want to document in your own words; an interviewer model (Settings > AI > Model that conducts interviews) asks, reflects back what it heard, follows threads, asks for specifics and checks stored facts against what you say next. After every exchange a second step extracts the facts established (one per sentence, with a category such as trigger, step, decision, exception); they are the interview's record and you can correct or remove any of them. Write the draft produces a document from the facts (every fact must appear, nothing else is added); you can edit it and save new versions. Add to the collection turns it into a source (kind interview) that is read, indexed and turned into topics like any document. Because a person reviewed the draft, it is its own source page: the writer and checker are not run on it. The conversation belongs to whoever held it (a collection's manager can see them all). It needs write access, the editor role and a collection that allows AI, and counts against the heavy-action rate limit.

Fetching safely#

Only public http:// and https:// addresses. The name is resolved once and every address it has must be public (private, loopback, link-local, metadata, reserved, carrier-grade and unique-local ranges are refused, including odd spellings such as http://2130706433/ or IPv4-mapped IPv6). The connection is made to exactly that IP, with the original name for the Host header and the TLS certificate, so a name that answers differently a second time cannot redirect the request (DNS rebinding). Every redirect is checked again (at most 5), the size is capped (Largest web page or file captured), and addresses with a user name or password are refused. To read an intranet, list its hosts or networks in Settings > Web capture > Internal hosts web capture may reach. For pages built by JavaScript, add the browser service; its browser never touches the network itself, because every request it makes is answered by the same safe fetch.

Where files live and background work#

• Storage (Settings > Storage): local (default) keeps files in data/library inside the mounted data folder; s3 uses any S3-compatible bucket (see Optional services). Files are keyed <collection>/<source>/<name>: deleting a source deletes its file, deleting a collection deletes everything under its prefix, and on a versioned bucket every old version and delete marker goes too. Test the storage checks the saved values. Connectors cannot open files under data/library.
• Background work: processing runs as jobs in a queue in the database (claimed so that no two workers take the same job; a job whose worker dies is picked up again; failures retry with a growing pause, then show their error). By default the web app runs the workers itself (Background workers in this process); for heavy use add the worker container. Job payloads hold ids only, never document text.
• Activity: each document's processing is a session with source Library on the Activity page: its steps, how long each took, attempts, errors and the model calls with their cost. Sessions show an id, never the file name. Admin > Activity > Audit trail lists who created, changed or deleted collections, who granted or removed access, self-grants, uploads and removals of sources (by id), and role and setting changes.