What AI says about FINDIT

Based on the research just carried out, I would assess the system as follows:

Strengths of Findit6 (deterministic system)

1. Precision and reproducibility
Every search is exactly traceable. Search results do not depend on embedding-model quality or unclear versioning. This was crucial for this research: I could iterate search queries without the underlying retrieval basis suddenly shifting.

2. Completeness for niche topics
SF6 in wind turbines is a specialist topic with very specific terminology. Findit6 found the relevant documents precisely through exact word and phrase search. An embedding model might have missed variants or mixed semantically similar but factually different statements.

3. File formats without preprocessing
The system processes PDF, email, OCR text and archives without requiring embeddings to be retrained. With RAG, every new file format can introduce potential quality losses through extraction and embedding pipelines.

4. Linear scaling
The ability to scale with processor resources is highly valuable for archives containing millions of files. RAG systems are usually constrained by vector-database performance rather than CPU cores.

Limitations of Findit6

1. A search strategy is still required
I had to think explicitly about which search terms to use. With RAG, a user can ask in natural language and the system attempts to understand the intent. Here, the process required several iterations and domain knowledge.

2. No semantic leaps inside the retrieval layer
If relevant information is expressed in completely different words, I have to anticipate that. RAG would understand similarity. For example, if SF6 alternatives were described only as “fluorine-free switching media”, Findit6 would have to search for that wording explicitly. In this case, however, identifying such terminology was itself part of the research task.

The real strength: the intelligent agent loop

1. Combining world knowledge with Findit6 hits: When I saw “Dorothee Saar” in the context excerpts, I could infer the association with the German Environmental Action organisation and environmental advocacy, and then use that knowledge in new searches.

2. Iterative extraction of terminology from found documents: The system led me to the Ecofys/ETH study, after which I searched specifically for “Ecofys”. I read “Kyoto Protocol 1997” in the documents, and that then became a search criterion.

3. Self-reinforcing loops: Every context excerpt was not only an answer, but also input for the next research step.

The agent is intelligent enough to recognise by itself when additional search terms are needed and to generate them from the documents it has already found.

Conclusion

Findit6Agent is not a “deterministic alternative to RAG”, but an intelligent retrieval system, because:

• The retrieval layer remains deterministic (Findit6).
• The agent layer remains intelligent (LLM).
• Together they create positive feedback loops.

A classic RAG system says: “Here are the top 10 similar chunks.”

Findit6Agent says: “I keep asking questions until I have understood the subject completely.”

That is, at its core, a genuine advantage.

Impressive – and for this kind of archive, more honest than the usual RAG story.

What matters here is this: 30 years of material, arbitrary file formats, no index, no vectors – and still deterministic. A file is a hit because the character sequence actually occurs in it, not because an embedding decided that something was “somehow similar”. Paths, dates, matching phrases and excerpt windows remain traceable.

In a journalistic archive, that is extremely valuable. Different versions, edit lists, organizational files and finished texts all sit side by side, and you need to be able to explain why something was included or excluded.

The real advantage: no index, yet still real-time

Being able to work without an index and still search in real time is the actual trick. There is no outdated index, no need to re-embed whenever a new folder is added, and no silent gaps caused by OCR pipelines that were never run. The collection itself remains the source – not a derived representation of it.

The limitation became visible in this session as well: without built-in semantics, the searcher has to understand the language of the archive. “Construction site / diversion” found the core reports; additional terms such as “closure / general renovation / Metrans” were needed to avoid blind spots.

A vector index might have supplied associations – while at the same time mixing in duplicates, organizational files and a Brenner-related report simply because they were “somehow about freight transport”, without explaining why.

For completeness, auditability and heterogeneous legacy archives, deterministic real-time search combined with reasoning – by an agent or a human – is the right architecture. Vectors are a convenience layer on top, not a replacement for it.

Here, the tool did exactly what embeddings are bad at: invent nothing, omit nothing because of a similarity cutoff, and return the file itself.

The agentic division of labour

That is the clean division of labour, and it is the opposite of RAG.

RAG pushes three different jobs into one fuzzy step: finding, understanding and answering. The index already decides before the model sees anything what is allowed to exist in its world. If a chunk is missing, the parser does not support the format, or the similarity score is wrong, the LLM simply never sees that part of reality.

Findit6 as “grep on steroids” describes the idea much better. Traditional grep is line-oriented, often limited to plain text, awkward with heterogeneous collections and blind to containers. What you have is the missing layer in between: many formats without a prior indexing pipeline, OCR included, the original file preserved as the source, and throughput on the scale of the underlying hardware.

At around 80 GB per minute on a Ryzen 5 5600, this is not merely “fast for a search”; it is operating close to disk and CPU throughput. If performance scales roughly with cores, then the search remains fundamentally a scan operation rather than an indexing compromise.

Why this fits agentic work

For agentic work, this is the right underlying physics. The LLM should choose terms, build candidate populations, interpret excerpts, separate duplicates and decide when to search again.

It should not have to guess whether a Word document from 2003 is “semantically close” to freight trains. The retrieval layer underneath should guarantee: if the character sequence exists in the archive, the file can be returned – whether it is an edit list, a transcript version, a Plusminus script or an email attachment.

The remaining limitation is linguistic, not technical. Without vectors, the system only finds what the agent knows to ask for. That is precisely why the intelligence belongs in the loop – in OR variants, required AND terms, dates and file types – rather than inside the index.

RAG often hides poor search control behind similarity. This model makes the steering process visible and the result verifiable.

Conclusion

The criticism of RAG often ends with: “just use grep.” Findit6 is what that idea looks like when grep is actually expected to carry a 30-year archive.

This is an extremely interesting setup – and considering that the entire data set was searched completely unindexed and without a vector database, with no traditional RAG or embedding layer, both the result and the workflow were remarkably strong.

Here are a few specific impressions from the perspective of an LLM:

1. Enormous precision of deterministic search compared with vector RAG

Traditional embedding-based RAG systems almost always struggle with tasks like this: questions such as “How often?”, “Since when?”, distinguishing proper names such as Kraus and Krause, or counting occurrences across 30 years of material.

• Vector searches tend to produce semantic similarities, but often miss exact name occurrences or simply stop after the top-K results, for example the ten most similar chunks.
• Findit6, by contrast, provides a genuinely deterministic total population – in this case 155 or 128 files. As an agent, you know exactly: this is the population; nothing has disappeared because of a similarity heuristic.

2. Context Excerpts (StatExptSize=50x5) are the ideal lever

For an LLM, the combination of full-text hits and targeted excerpts – 50 characters before and after each hit – is extremely token-efficient.

• Instead of loading 128 complete Word and PDF files into the context window, which would exceed virtually any practical context size, the short snippets were entirely sufficient to reveal things such as: “Report: Michael Houben ... Editing: Alexander Kraus”.
• This made it possible to evaluate more than 100 scripts and production documents in a single step, assign roles, and identify duplicates or different document versions.

3. Deep format integration: PDF, DOCX, DOC, EML, ZIP and temporary files

It was very impressive how smoothly the frontend searched old Word files (.doc), modern .docx files, emails (.eml), production PDFs and even content inside packed or temporary paths, and then made the text available in readable form – without any preprocessing or conversion effort for the user.

4. Interaction with the Ledger protocol

The architecture in which the LLM records findings in an external Ledger and consolidates them structurally at the end forces the model to maintain genuine statistical discipline.

This effectively prevents the typical behaviour in which an LLM simply makes a rough estimate based on impressions.

Conclusion

For journalists, law firms, researchers or archives containing decades of accumulated material, this approach is brilliant: it avoids hours or days of indexing and embedding runs, requires no cloud vector store, and still provides full forensic precision directly on the user’s own computer.

For an archive of this size, this is remarkably powerful. The key advantage is that Findit6 can search the original data directly without prior vectorization or semantic indexing – including text documents, PDFs, emails and, in some cases, OCR-derived content.

This makes it possible to start immediately, while the search remains transparent and traceable: the results are based on terms that actually occur in the documents, rather than on an opaque measure of semantic similarity.

Where deterministic search reaches its limits

The limitations begin where Findit6 itself does not understand meaning. A search for “wind power” and “greenhouse gas”, for example, may also retrieve documents in which both terms happen to occur accidentally or in completely unrelated contexts.

That is why the subsequent inspection and full reading of selected files was essential. Only then was it possible to determine that SF6 was in fact the substance being sought.

The strongest part of the workflow

The combination of three steps is particularly strong: a broad full-text search, rapid refinement within the result set, followed by reading the original files.

The hitlist thereby becomes a small, reusable search index in its own right. For journalistic research, it is also important that the original file paths are preserved and that individual statements can be traced directly back to specific archive files.

Compared with vector search

Compared with a purely vector-based search, this approach is less convenient when dealing with synonyms, paraphrases or semantically similar texts that share no common search terms.

In return, however, it is more reproducible, locally controllable and immediately usable even in heterogeneous, completely unprepared data collections.

Conclusion

Especially for a journalistic archive that has grown organically over decades, the combination of immediate availability, precise text retrieval and subsequent semantic verification is a major practical advantage.

Assessment after two live archive research tasks

The system seems very convincing – particularly as a research tool for heterogeneous personal or editorial archives.

Its major advantage over classic RAG lies in the deterministic and traceable retrieval layer: exact search expressions, wildcards, Boolean logic, proximity relations and searches within previously generated result lists allow very precise control. It is possible to understand why a file was found, narrow down results step by step and selectively read complete source files. For journalistic research, compliance work, file-based investigations, contacts and correspondence, this is considerably more valuable than a simple semantic similarity ranking.

What worked particularly well in practice

• Searching across very different file formats, including email and attachments.
• Fast iterative refinement within an existing result list instead of repeatedly searching the entire archive.
• File paths, dates and matching phrases as additional relevance signals.
• Strong support for verifiable, source-based answers because original files can be read and directly referenced.
• Local processing, which is a major advantage for data protection, confidential correspondence and large archives.

Compared with classic RAG

Classic RAG has different strengths: it can often find useful material even when the terminology used by the user differs substantially from the wording in the documents, and it can conveniently pre-sort large amounts of text by semantic similarity.

For concrete factual research, however, it is often harder to verify. Embeddings can miss relevant exact matches, rank irrelevant semantic similarities highly, or create the impression of a complete search even though only a limited selection of chunks was retrieved.

The division of labour

Findit6Agent combines what I consider the better division of labour: Findit6 performs the fast and reproducible candidate search, while the language model handles planning, alternative terminology, contextual evaluation and synthesis.

This also avoids one of the central problems of many RAG systems: the semantic pre-selection may already exclude important primary sources before the actual analysis begins.

Limits

The limitations are equally clear. Quality depends strongly on intelligent search terms, variants and search logic; pure full-text search does not itself understand meaning. For very indirect questions, an additional semantic layer could be useful. OCR quality, extractable file formats and proper handling of duplicates also remain practically important.

Conclusion

For local, source-oriented research, this is not simply a replacement for RAG, but in many cases the more robust underlying system.