Case study · newspaper archives · PoV

NewsRAG

Ask a newspaper archive a question, then inspect the evidence behind the answer.

A working newspaper-archive RAG system developed as a PoV for a digital publishing platform. Ask questions in natural language, search article text and captions together, narrow the period, and inspect answers against their original sources. Hybrid retrieval, local reranking and an explicit no-answer path make the evidence behind each response visible. The document model also accommodates a separate web archive.

An authentic article, an authentic fact and an authentic citation can still make a false answer if the place, date or scope is wrong.

Working PoV · source-aware archive search. Repeatable ingestion, hybrid retrieval, date filtering, document-bound citations and article views work together in an end-to-end system. The sections below explain the architecture, the features and the evidence collected during development.

Why newspaper archives need a different RAG

The archive contains real reporting, but also captions, repeated local editions, announcements and stories that change over time. Finding a related passage is only the start.

Print and web are separate sources

They can describe the same event without being the same document. Separate ingestion paths were designed to converge on one document model; the working print pipeline establishes the ingestion pattern; web ingestion is a separate extension.

Captions contain otherwise invisible people

A corpus analysis found 58 people whose names appeared only in captions across the five editions. Captions became searchable evidence, with their origin retained.

Time belongs to the answer

An old article can say a bridge “will reopen”. That wording needs the publication date and the source period beside it; it is not a statement about today.

“No supporting source found” is a valid outcome

A system that always answers can stitch unrelated but genuine articles into a convincing false account. The PoV has an explicit no-generation path when retrieval falls below its provisional threshold.

Inspect the delivered archive before building ingestion

The five supplied editions, dated 3–17 March 2026, contained PRISM/PAM XML already split into articles, with article photos and captions. The sampled text came from the publishing workflow; it did not require OCR. That finding removed an entire reconstruction stage from the PoV.

SourceArticle XML + photos

IngestionIdentity · metadata · versions

PreparationBody blocks + caption fragments

Local indexPostgreSQL + pgvector

The sample defines the boundary. It contained neither full-page scans nor word coordinates. Page-image highlighting was therefore excluded. The recent XML did not establish the quality or format of older archive years.

Architecture — two searches, one evidence path

Lexical search catches names, places and numbers; dense retrieval catches meaning. Both are filtered before fusion, reranked locally and deduplicated before the generation decision.

InputQuestion + date range

LocalLexical + dense search

LocalRank fusion → rerank → dedup

Application gateEvidence band

Strong retrieval

Generate from the selected fragments and resolve citation markers against those documents.

Weak retrieval

Generation is allowed, with source excerpts available for checking. This band still produced errors in exploratory tests.

Below threshold

Do not call the generator. Return an application-written no-answer response. This means insufficient retrieved support, not proof that the archive never covered the event.

Local retrieval, external generation in the PoV. Documents and indexes stayed on the development node; the question and selected excerpts were sent to an external model through OpenRouter. This was a PoV placement choice, not an end-to-end local or sovereign deployment. Retrieval bands are heuristic scores, not calibrated probabilities of truth.

Technology stack

Local retrieval

RTX 4070 Super · 12 GB GPU

StoragePostgreSQL + pgvector
SearchItalian full-text GIN + dense HNSW
Embeddingbge-m3 · 1024 dimensions
Rerankingbge-reranker-v2-m3
RuntimePyTorch + FlagEmbedding

Answer generation

External API · PoV configuration

RoutingOpenRouter
Modelgemma-4-31b-it
CitationsDocument mapping in application code
TraceModel, provider, tokens, cost and latency

Application

Small, explicit pipeline

BackendPython + SQL
InterfacePython standard library, HTML + CSS
EvaluationReview records in PostgreSQL
AccessPassword-protected tunnel used for review

Key architectural decisions

One database, rather than two systems to operate

ProblemText, vectors, dates and publication filters could have become separate services.

ChoiceKeep relational data, lexical search and vectors in PostgreSQL; revisit storage only against measured scale.

OutcomeOne data store for retrieval, answer records and review judgments, without a second vector database.

The source remains authoritative

ProblemHand-correcting text or appending captions to the article body creates an undocumented second original.

ChoicePreserve delivered content; derive rebuildable fragments with body/caption provenance, stable identity and version metadata.

OutcomeRe-runnable ingestion and an index that can be rebuilt, with caption evidence distinguishable from article text.

Fuse ranks, not incomparable scores

ProblemLexical relevance and vector similarity are different scales; weighting them before evaluation would add an arbitrary tuning choice.

ChoiceReciprocal Rank Fusion, followed by reranking and duplicate removal.

OutcomeBoth retrieval paths contribute through an explicit, inspectable sequence.

Citation identity is controlled by code

ProblemA model can produce a convincing citation to something it never received.

ChoiceNumber the supplied fragments and resolve markers in application code; remove and log out-of-range markers.

OutcomeDisplayed citations refer to supplied documents. This bounds source identity; it does not guarantee that each claim is supported.

An interface for reading—and for checking

The PoV had a reading view, an article view and a review view. Its purpose was to make the next evaluation possible: a journalist needs to inspect evidence, not spend hours judging terminal output.

Answer and all cited sources together

Source cards below the answer show the delivered title, section, date, page and the excerpt actually used. The complete article can be opened for context.

Photos stay in their source context

A source card can carry an article thumbnail, and the article view shows its photos and captions. A photo is not placed beside a generated claim as if it illustrated that particular fact.

Visible temporal scope

Date fields constrain retrieval, and the application reports the period covered by the cited sources. This makes the evidence window visible without pretending to correct historical wording.

Review without a confidence label leading the verdict

In review mode, the retrieval band is hidden until the judgment is saved. The response layout remains consistent across bands; judgments are stored separately from answers.

Problems found, fixes and limits

A hybrid search with a silent lexical branch

SymptomFour of six realistic test questions returned no lexical candidates with AND matching.

CauseAll query terms were required together; accent handling also differed between indexed text and user queries.

FixOR candidate matching, consistent accent/stopword handling and a regression check covering accented and unaccented forms.

Duplicate text was two different problems

SymptomRepeated passages occupied multiple result slots.

CauseLocal-edition article copies and recurring boilerplate had been counted together.

ResponseDeduplicate identical text and diversify by document; classify boilerplate separately. Worst-case per-query crowding still needed a larger archive.

Every fact was real; the answer was still false

ObservedA question about one town received a precise election result from another town, based on a genuine caption.

LessonGrounding and valid citation identity do not establish relevance to the requested entity.

ResponseMake place and source excerpts easy to inspect. The weak-band error remained an evaluation issue, not a solved hallucination problem.

“Not found” changed with the wording

ObservedDirect and retrospective phrasings of the same bridge question retrieved relevant material but landed in different evidence bands.

CauseThe reranker score changed enough to cross a provisional threshold.

Next testEvaluate paraphrases before calibrating the gate or adding a conversational rewriter. One observed pair did not establish a general rate.

A source-bound answer is inspectable, not automatically correct. Relevance, entity identity and time still need their own tests.

Recorded observations

Corpus counts and exploratory measurements from the September 2026 development sessions. These are not production-capacity figures or a completed journalist-reviewed benchmark.

5 editionsone newspaper · two-week window

818source documents ingested

2,542fragments generated, including captions

2,522indexable fragments with vectors

326caption-origin fragments

Local retrieval timing

On the idle development node, query encoding plus SQL search had a recorded median of 19 ms and p95 of 28 ms, before reranking. A later 16-question probe recorded a 226 ms median including reranking. These were different test stages; neither is an end-to-end answer SLA.

Exploratory answers exposed the weak band

In a ten-question development review, all four strong-band answers and both no-generation outcomes were judged correct. The four weak-band cases split two correct and two incorrect. This small, manually assembled check guided the interface; it was not a validated accuracy estimate.

Local latency measurements were taken with the shared GPU otherwise idle. External generation showed provider-dependent variation and is not included in those retrieval times. The formal review set had not been populated at the recorded project handoff.

Delivered capabilities and further validation

Implemented in the PoV

Print XML ingestion with stable identities, version fields and repeatable updates.
Local lexical/dense retrieval, rank fusion, reranking and deduplication.
Answer generation with document-bound citations and a no-generation branch.
Article/source views, date filtering, answer records and a review interface.

Extensions and further validation

A long historical archive and a journalist-reviewed golden set.
Web ingestion and verified print/web article linkage.
Full story reconstruction, exhaustive archive counts and multi-turn chat.
Production permissions, subscriber integration, hardening and service-level acceptance.

The next acceptance step was evidence, not more features. A longer archive and expert-written questions were needed to measure answer relevance, missed evidence, temporal coverage and threshold errors. Multi-turn rewriting was designed but deferred because it would change the wording feeding an already-sensitive retrieval gate.