SwatGPT started with a fairly ordinary student problem: the answer probably exists somewhere on the college website, but finding it takes longer than asking another person. Wi-Fi setup lives in the ITS knowledge base. Course requirements live in the catalog. Dining hours live somewhere else. Everyone still calls the dining hall Sharples.
SCCS had a GPU, and I wanted to make that information easier to use. The result is SwatGPT, a campus assistant built on LibreChat, with self-hosted inference and a retrieval system for Swarthmore information.
The interesting part turned out to be everything around the model: collecting useful pages, keeping a question about Cornell attached to the right Cornell, recovering tools after a cache expires, and knowing when a working chat interface is quietly answering without its sources. This is an engineering account of how the project changed, including the parts I still cannot measure well.
From a prototype to a campus service
The spring version used my own crawler, OpenWebUI, and its built-in retrieval. I started inference with llama.cpp because getting a model to respond was the first thing I needed to understand. That prototype exposed the real work: cleaning institutional web pages, managing an index, and serving more than one conversation comfortably.
In August I rebuilt the application around a current LibreChat fork. “Rebuilt” needs a qualification: LibreChat supplies the chat interface, streaming, conversation storage, and much of the backend. I did not write those systems from scratch. My work was the campus-specific retrieval and ingestion, integration, configuration, and the changes needed to operate it through SCCS.
The repository gives the August work a clearer timeline than my recollection of the spring prototype:
| Stage | What changed | What it addressed |
|---|---|---|
| August 15 | Targeted knowledge-base, website, and catalog scrapers | A broad crawl produced too much repeated or irrelevant material |
| August 16 | Qdrant, custom retrieval, ingestion, interface gates, deployment | More control over what reaches the model and how the app runs |
| August 16–23 | Hybrid search, Dash tools, MCP recovery fixes | Campus entities, current information, and failures after startup |
| September 9 | Retrieval metrics, generation ledger, admin tools, directory lookup | Visibility into the service and another source of structured context |
The early prototype is useful background, but the detailed implementation below follows the September 9 repository snapshot. Those are different kinds of evidence; I do not want a remembered frustration to become a benchmark.
Separate the model from the information
Generation runs through vLLM on loon's RTX PRO 6000 Blackwell with 96 GB of VRAM. The configured model is Qwen3.6-35B-A3B-FP8, a pretrained Qwen model. SwatGPT's campus knowledge comes from supplying context at request time, rather than training a new foundation model.
The same GPU runs two smaller services through Hugging Face Text Embeddings Inference: gte-modernbert-base for 768-dimensional embeddings and gte-reranker-modernbert-base for reranking. Thinking is disabled in the chat configuration. For the kind of campus questions this interface invites, I wanted the answer to arrive without a long visible reasoning phase.
The stateful application lives on a separate VM, eagle. That includes LibreChat, MongoDB for users and conversations, Meilisearch for conversation search, Qdrant for campus retrieval, and the Dash MCP server with its PostgreSQL cache. Keeping this tier separate means an app rebuild does not require disturbing the model server.
The separation also makes failures easier to describe. If embeddings or Qdrant fail, chat can continue without retrieved context. If generation fails, there is no model response. Calling both situations “the app is down” hides a distinction that matters to the person debugging it.
A smaller, more deliberate corpus
The first crawler treated the college website too uniformly. Navigation, footers, old news, and useful instructions all became text. A large corpus can look impressive while mostly teaching retrieval that every page belongs to the same institution.
The replacement has three collection paths. scrape_kb.py uses MediaWiki's API for ITS articles. scrape_www.py filters the public sitemap to student-facing sections, extracts the main body with trafilatura, and discards bodies shorter than 300 characters. It also excludes categories such as news, profiles, and galleries. scrape_catalog.py collects courses, programs, and selected policies through the catalog's own AJAX endpoints, with request pacing and backoff.
| Source | Markdown files in the inspected server checkout | Useful for |
|---|---|---|
| ITS knowledge base | 487 | Setup instructions and troubleshooting |
| Main college website | 1,080 | Offices, campus services, and policies |
| Course catalog | 1,961 | Courses, programs, and academic requirements |
| Total | 3,528 | The input to a full corpus rebuild |
That count describes the files on loon, not a fresh scrape or a live database query. The Git-tracked snapshot contains 3,492 Markdown files; the difference is 36 catalog files present in the server checkout. It is a small reminder that the repository and the deployed data are not automatically identical.
Cleanup is still imperfect. Two consolidated printer documents carry the knowledge base's main-page URL rather than a useful article URL. Retrieval can find the right instructions and still give an unhelpful citation. Source metadata is part of answer quality, not just a field to attach at the end.
Chunking around meaning, then indexing
Ingestion first splits Markdown around headings, then uses a sentence splitter with a 512-token target and 64-token overlap. Each embedded chunk starts with its title and section breadcrumb. A paragraph about prerequisites should remain associated with the course whose page it came from, even when the paragraph does not repeat the course name.
That breadcrumb is also included in the sparse representation. The payload retains the body, source URL, title, section, and relevant catalog metadata. LlamaIndex helps with this offline chunking; it is not on the request-time retrieval path.
Rebuilding the index uses a fresh timestamped Qdrant collection. The script embeds the corpus, uploads its points, switches the kb alias to the new collection, and then removes older collections. Chat always queries the alias. An incomplete upload therefore does not become the active corpus halfway through a rebuild.
This is a full rebuild rather than an incremental ingestion system. Scheduling scrapes, detecting meaningful changes, and validating a new index before switching the alias remain useful next steps. An atomic switch prevents a partial index from appearing; it does not prove the replacement is better.
Why semantic similarity was not enough
The failure that motivated hybrid search was a question about a campus entity. Ask whether Cornell is open late, and dense retrieval can find plenty of material about library hours without preserving the fact that Cornell means Cornell Science Library at Swarthmore.
The current path combines semantic and literal matching:
- Embed the latest user message and build a sparse term-frequency vector from its words.
- Query Qdrant for the top 20 dense hits and top 20 sparse hits, with the two searches running concurrently.
- Deduplicate by point ID and cap the candidate pool at 32, matching the reranker's default batch limit.
- Score candidates with the cross-encoder, discard scores below 0.3, and select up to eight chunks.
- Preserve an eligible high-ranking lexical hit when otherwise all eight slots would go to other candidates.
The sparse tokenizer lowercases text, extracts alphanumeric tokens of at least two characters, and maps them to 32-bit FNV-1a hashes. Term counts become sparse values; Qdrant applies IDF weighting. Python computes document vectors during ingestion and TypeScript computes query vectors. Those implementations must agree, or a word in the query will stop addressing the same sparse coordinate as that word in a document.
The lexical rule has a precise limit. A candidate must already pass the 0.3 reranking floor. If none of the selected eight belongs to the sparse channel's top three, the eligible candidate with the best sparse rank replaces the final slot. This protects a useful entity match; it does not force every literal mention into context or guarantee a correct answer.
There are other compromises. Candidate merging puts dense results first before applying the 32-item cap, so the final pool is not a perfectly balanced fusion. And the reranker sees chunk bodies, while initial indexing includes title breadcrumbs. These choices keep the implementation small, but deserve measurement before being treated as optimal.
A retrieval failure should be visible
Embedding, search, and reranking share a single 1.5-second abort budget. A network error or timeout returns no context and allows the message to proceed. Retrieval also starts alongside other message-building work, rather than being an entirely separate blocking phase.
The benefit is availability. The cost is that an assistant can look healthy while losing the information that makes it useful. The September work therefore adds a structured result: hit, empty, disabled, error, or timeout, along with elapsed time and the number of chunks returned. Those outcomes feed metrics instead of collapsing every unsuccessful retrieval into the same blank result.
The retrieval tests cover request payloads, ordering, deduplication, the lexical rule, and service failures. They explicitly check that a lexical hit below the score floor is dropped. These are checks of implementation behavior. They do not establish answer accuracy across real student questions.
Current information takes another path
A scraped page can explain a dining plan, but today's menu and departures need fresher data. Damian built the Dash MCP server, which exposes read-only tools for the public information shown by the campus dashboard: hours, dining, events, transit, alerts, and other feeds.
Most tool calls read a PostgreSQL snapshot refreshed by background polling. Dining also has a bounded read-through path: if a requested date, meal, or location is absent or stale, the server can fetch it from the upstream feed and save the result. Responses include provenance and freshness metadata, with stale cache data available when upstream requests fail.
The most instructive integration bug was unrelated to the answer itself. LibreChat cached application-level MCP tools with a twelve-hour lifetime. After expiry, recovery fetched the tool list, but publication lacked the ordered revision required by the cache writer. The write was refused. A running server could have perfectly valid tools while the application believed none were available.
The August 23 fix gives recovery publications the next revision and makes remaining refusals visible in warning logs. Restarting had temporarily hidden the problem; repairing the cache transition addressed why it returned.
Operating the application is part of building it
The interface removes many capabilities inherited from LibreChat: model selection, agent creation, web search, code execution, and user-added MCP servers. Route gates enforce disabled features beyond hiding their navigation links. SCCS's Keycloak provides the login path. This leaves a campus-oriented product that is easier to explain and maintain.
Deployment runs through a self-hosted GitHub Actions runner. The workflow rebuilds the application and MCP images, updates the Compose stack, polls application readiness, checks MCP health and connectivity, and rejects an initialization log reporting zero tools. One limitation in that final check: an absent initialization log is printed but does not itself fail the deploy. It is stronger than checking that containers started, but still has a gap.
September's additions make operations more concrete: Prometheus metrics, monitoring configuration, a per-response generation ledger, usage summaries, and admin controls. The ledger records tokens, time to first token, duration, tools, and retrieval outcomes. Admin review and export have separate capabilities and audit records.
This changes how I should describe privacy. Inference is self-hosted, but conversations are stored and authorized administrators have review functionality. Local inference alone does not mean messages are invisible to operators. The presence of monitoring code also does not prove every dashboard or alert receiver is deployed and working.
The same September snapshot adds a separate directory lookup path. It detects supported housing questions, matches them against an in-memory directory index, and supplies structured context. The formatter handles missing entries, ambiguity, and hidden housing information. This is lookup over an imported snapshot, not a claim that an LLM learned the student directory; a repository review also cannot verify that a production import is current.
What I can measure, and what I still cannot
Earlier versions of this post quoted adoption, token volume, latency improvements, and a hand-checked accuracy percentage. Those numbers did not come with a reproducible evaluation set or a defined reporting window, so they are not evidence for this version of the write-up.
What I can point to is narrower: the corpus files, retrieval parameters, regression tests, deployment checks, and observability code. Even the ingestion script's --verify option only prints dense-search results for a few smoke questions. It does not exercise the complete hybrid-and-generation path or score the resulting answers.
The next useful experiment is a modest campus evaluation set: named places, course codes, paraphrased instructions, stale information, ambiguous questions, and follow-ups. The current retriever uses the latest message without rewriting it from conversation history, so “what about the other one?” remains a predictable weakness. I want to measure source retrieval and answer correctness separately, then compare dense-only and hybrid retrieval on the same questions.
This project taught me how many ordinary systems problems sit inside a chatbot: data quality, caches, timeouts, deployment checks, and permissions. I am still learning how to evaluate the answers rigorously. Having a service that responds is the starting point for that work.
Sources and scope
This account follows the SwatGPT repository at 601e2bf47, inspected alongside its server checkout. The spring prototype section retains my earlier account; the current fork's history establishes the later implementation. No production conversations were inspected and no live accuracy or load benchmark was run for this article.
- Collection and ingestion: website scraper, catalog scraper, and chunking, embeddings, and alias replacement.
- Retrieval: hybrid-search change, candidate merging, selection and failure behavior, and regression tests.
- Live data and deployment: Dash MCP design, cache-recovery fix, and production workflow.
- Later work: generation ledger, admin review, directory lookup, and monitoring configuration.
SwatGPT is an SCCS project built on LibreChat and its contributors' work. This is my account of the integration and engineering, not a claim of sole authorship of the stack.