Curltale AI

Docs › Get started

How it works

The two halves of Curltale AI, how a question is answered, and what is different.

Curltale AI holds on to two kinds of knowledge, built differently on purpose and connected by a bridge. The ontology knows exactly what your data means and is used to query it. The knowledge base knows what your people know and can prove it.

What it holds on to#

What Curltale AI holds on toLeft, the ontology: entities and fields, definitions and filters, relationships and sensitivity, used by the query tools to run one checked, read-only statement on your data. Right, the knowledge base: documents become verified source wikis, source wikis become topics by subject, claims keep the exact quote, and typed relationships connect topics. A bridge links topics to the entities they are about. Every layer is chunked, embedded and searchable.THE ONTOLOGY · WHAT YOUR DATA MEANSTHE KNOWLEDGE BASE · WHAT YOUR PEOPLE KNOWEntities and fieldscustomer, order, contract, their columnsDefinitions and filterswhat “active” or “unpaid” meansRelationshipshow things join, and how far to trust a linkSensitivityrestricted, personal, masked: enforcedQuery toolsone planned statement, checked, run read-onlyDocumentsfiles, web pages, interviewsSource wikia verified pageper documentTopicspages by subject,from full wikisClaimseach fact withits exact quoteLinksgoverns, part of,replaces…Agent retrievalfind chunks, read pages, prove with quotesThe bridge: a topic points at the entities it is about (governs, about, instance of)Every layer is chunked and embedded, found by meaning and words, inside the collections you may read
LayerWhat it isIndexed as
Original textThe text of each document with its places (page, slide, sheet).Passages: vectors and words
Source wikiA standalone page written from one document, checked claim by claim against it; every version kept.Every version chunked: vectors and words
TopicA page by subject, merged across every document about it, following its type's template.Every version chunked; the topic itself embedded with its claims
ClaimOne atomic fact with its verbatim quote, the source, the wiki version and passage. Superseded, never deleted.Embedded
RelationshipA typed link between two topics with why they relate.Embedded with its context
OntologyEntities, fields, definitions, filters, relationships, sensitivity, examples; versioned and published.Search index of the published version

Everything is searched by meaning and by words together (reciprocal rank fusion), always inside the collections the caller may read.

How a question is answered#

How a question is answeredA question from a person or an agent takes the data path, the knowledge path, or both. Data: understand the definitions from the ontology, plan one statement, check it and run it read-only under your access. Knowledge: find matching chunks of topic pages, read the whole pages and linked topics, prove the facts with claims and quotes. The answer carries its definitions and its sources.A questionfrom a person oran AI agentTHE DATA PATHUnderstandmeaning from theontologyPlanone statement withthe right filtersCheck, runrebuilt under youraccess, read-onlyTHE KNOWLEDGE PATHFindmatching chunks oftopic pagesReadwhole pages andtheir linked topicsProveclaims with quotes,the wiki if neededThe answernumbers with theirmeaning, facts withtheir sourcesBoth paths run under the same access rules. Most questions need one; the assistant uses both when a question needs both.

Chunks find, pages answer. For a document question the agent calls library_context: the question is matched against chunks of the published topic pages (and the topics' own vectors), the best chunk per topic is kept, results are reranked if a reranker is configured, and the agent receives the whole page with the matched passage, the graph of linked topics and the typed relationships that match, plus answerable (false means the documents do not cover it, and the agent is told not to answer from its own knowledge). Provenance (library_topic_claims, library_topic_sources), the graph (library_traverse) and the verified source wikis (library_source_search, library_source) are one call away. For a data question the agent calls plan_query and query_data: the plan carries the definitions and join paths, and the statement is parsed, checked against the caller's entitlements and rebuilt before it runs.

What we do differently#

The usual approachCurltale AI
Documents are chopped into passages and searched as they are.Each document is first rewritten as a verified page, checked claim by claim against the original; anything doubtful waits for a person.
The assistant gets a handful of fragments and stitches an answer together.The assistant gets whole topic pages by subject, already merged across every document that mentions the subject, plus the topics linked to them.
Sources are a list of file names at the end.Every fact has a claim with its exact quote, the page version it came from and the passage it sits in.
When a document changes, old answers linger.What a removed source supported is withdrawn and the affected pages are revised surgically, checked against the sources that remain.
Contradictions are averaged away.A change that contradicts a published fact is held, both versions side by side, until someone decides.
Text-to-SQL guesses what "active" means.The ontology defines it, and every statement is checked against the caller's access and rebuilt before it runs.
One big index for everyone.Isolated collections with their own grants and a public or internal tag that travels with every result.

This documentation covers version 0.2.0. Something missing or wrong? Open an issue.