A paper pipes RAG into structural monitoring, scoring a historic minaret zone by zone

On October 15, 2026, Scientific Reports published an open-access paper that puts retrieval-augmented generation inside structural health monitoring. The subject is the minaret of the İskenderpaşa Mosque, a 16th-century structure in Trabzon, Türkiye. Four vibration sensors feed three LLM/VLM agents — one reads the signal, one reads photographs, one reconciles them — and the output is a risk score per structural zone rather than a single number for the tower. Kemal Hacıefendioğlu, Volkan Kahya and Tunahan Aslan wrote it; the paper is CC BY 4.0, and the full text and figures download without a subscription.

Key points

• Risk is reported by zone, not as one tower-wide figure: transition 1.00 (Critical), base and lower shaft 0.75 each (High), balcony 0.63 (Medium), upper shaft 0.26 (Low), cap 0.00. The strongest pairwise signal anomaly is S2–S3 at severity 1.00 and an 8.9% flag rate.
• Turning RAG on changes the verdict: without it, the fusion layer returns a global risk of 0.816 (High); with it, 0.491 (Medium). The authors repeatedly stress this does not mean the second number is more accurate — retrieved context suppresses visual over-calling.
• The paper reports no accuracy, precision, recall or F1. Measurements 13–20 have no independent, time-synchronised damage labels, so a supervised benchmark is not possible. That sentence is the most important one in the study.

Background

Monitoring historic masonry runs into a familiar split. Vibration data can tell you something changed but not which component changed. Photographs show cracks but cannot separate damage from decorative banding. The paper aims at the layer in between: putting signal anomalies, visual evidence and structural priors into one traceable decision chain.

There is prior work. The same group published a companion paper in July 2026 on multi-camera vision-based monitoring of masonry minarets, also an LLM/VLM-orchestrated multi-agent framework, but without retrieved memory. The new increment is a structured memory package — geometry, materials, baseline dynamics, finite-element notes and an inspection playbook stored as retrievable context — that the three agents cite while they reason.

How it works

Four branches run under a manager-orchestrator written in Python. The signal branch cleans four channels, characterises a baseline with power spectral density, windows adaptively, normalises robustly, then trains a 1D convolutional autoencoder on Measurements 1–12 and evaluates on 13–20. The threshold sits at quantile 0.99958, flagging 3 of 7,171 training windows (0.04%), consistent with the healthy-training assumption.

Localisation uses relation-based autoencoders on sensor pairs: pairwise change scores are computed first, then projected onto six structural zones through a predefined sensor-to-zone map. The vision branch has a VLM read inspection photographs linked to those sensors and return a severity and confidence per zone. The RAG branch is deliberately plain — no vector database, no fine-tuning, just lexical matching over the project memory package by task keywords, acting as context injection rather than model update. Fusion uses a reliability-weighted rule, 0.75 signal to 0.25 vision, with the visual weight clipped into a 0.20–0.70 band based on confidence statistics.

What the authors say

The paper draws its own boundary in plain language. At the end of Section 3 it states that without independent, time-synchronised damage truth, the reported anomaly and risk scores cannot stand in for classification performance, and that the RAG-induced drop in risk reflects a change in decision behaviour rather than improved diagnostic accuracy.

The discussion lists four constraints. The study covers a single case whose geometry, repair history and support conditions are specific to this minaret, so generalising to other masonry towers needs care. The evidence base is thin: four channels and limited image coverage per zone. The global-anomaly-first policy resists false localisation but can delay detection of very early local deterioration before any global anomaly appears, which is why the authors keep periodic targeted local inspection in the loop even during stable intervals. And Measurements 1–12 constitute an operational baseline, not an independently verified damage-free state; environmental and operational variability need longer campaigns.

There is also an engineering admission: the original exploratory notebook did not fix random seeds, so reruns can vary slightly. The authors recommend pinning NumPy/PyTorch seeds and deterministic backend settings in a future code release.

Why it matters

The contribution is not a monitoring product. It is a demonstration of what auditable looks like. Every JSON output preserves the retrieved document references, the sensor-to-zone mapping, the fusion weights and the parsing mode, so any risk score can be traced back to its inputs, prompts and intermediate findings. For heritage conservation — a field where someone signs off and carries liability — that traceability matters more than a point or two of accuracy.

Three takeaways for readers. In engineering decisions, RAG here constrains rather than enhances: it makes the model more conservative and harder to fool with decorative brickwork. The LLM does not replace the numerical pipeline; windowing, thresholding, pairwise severity and fusion are deterministic code, and the model explains and reports. And a single case with four channels and no ground truth is a methodology demo. Teams copying it should solve the labelling problem first; without inspection truth, no claim of being more accurate holds up. For another look at multi-agent orchestration in an engineering setting, our Manifold R²SI breakdown covers a system with the same separation between orchestration and execution, applied to simulated robot tasks instead of a physical structure.