I invite you to upgrade to a paid subscription. Paid subscribers have told me they appreciate me creating the programming projects and would like to see more of them in the future. Coding Challenge #122 - Database Driven LLM WikiThis challenge is to build your own database powered LLM Wiki.
Hi, this is John with this week’s Coding Challenge. 🙏 Thank you for being a subscriber, I’m honoured to have you as a reader. 🎉 If there is a Coding Challenge you’d like to see, please let me know by replying to this email📧 Coding Challenge #122 - Database Driven LLM WikiThis challenge is to build your own personal knowledge base tool - a system where an LLM agent reads your curated sources, extracts the key information, and builds you a living, interlinked wiki that grows smarter with everything you add. Most people’s experience with LLMs and documents is RAG: upload files, ask questions, get answers stitched together from retrieved chunks. It works, but the LLM rediscovers knowledge from scratch every time. Ask a subtle question that spans five documents, and the system has to find and piece together fragments it’s seen before. Nothing accumulates. LLM Wiki takes a different approach. Instead of just retrieving from raw documents at query time, an LLM agent incrementally builds and maintains a persistent wiki - a structured collection of markdown files that sits between you and your sources. When you add a new source, the agent reads it, extracts key information, and integrates it into the existing wiki: updating entity pages, revising topic summaries, noting where new data contradicts old claims, strengthening or challenging the evolving synthesis. The knowledge is compiled once and then kept current, not re-derived on every query. You’re in charge of sourcing, exploration, and asking good questions. The LLM does all the grunt work: summarising, cross-referencing, filing, and the bookkeeping that makes a knowledge base actually useful over time. This challenge is inspired by Andrej Karpathy’s LLM Wiki concept - a pattern for building personal knowledge bases using LLMs. Karpathy describes the idea in the abstract; our job is to build a working implementation. Under the hood, the system stores vector embeddings and full-text indexes for every wiki page in Oracle AI Database. When you ask a question, it finds relevant pages using hybrid search, and an LLM synthesises an answer with citations. The agentic behaviour - ingestion workflows, multi-step querying, lint passes - is orchestrated using LangGraph’s state machine model, while LangChain handles the LLM integration. It’s a practical introduction to agent-based knowledge management, vector search, full-text search, and building tools that genuinely compound in value over time. The Challenge - Building LLM WikiYou’re going to build a personal knowledge base that an LLM agent writes and maintains for you. It starts by ingesting source documents into a wiki of markdown files, then lets you query it through a web interface or CLI chat. Step by step you’ll add wiki scaffolding, source ingestion, index and log management, vector storage, semantic retrieval, project management, linting, and a user interface. By the end, you’ll have a tool that genuinely helps you build and navigate a growing body of knowledge. Step ZeroIn this introductory step you’re going to set your environment up ready to begin developing and testing your solution. You’ll need to make a few decisions and get some infrastructure running:
Testing: Verify your Oracle Database container is running and you can connect to it. Load your embedding model and generate a test embedding to confirm it returns a vector of 768 dimensions. Make a test call to your LLM API to confirm it returns a valid response. Verify your environment file is being read correctly and no credentials are in your source code. Step 1In this step your goal is to build the wiki scaffolding - the system that creates and manages the directory of markdown files that will become your knowledge base. The wiki lives on disk as a directory of markdown files. Before you can ingest sources or answer questions, you need the infrastructure to create pages, write content to them, and link them together. Think of this as the file system layer of your knowledge base. Start by defining what a new wiki looks like on disk. When you create a wiki, the system should scaffold a directory structure with subdirectories for different page types: summaries for source summaries, entities for pages about people/companies/concepts, topics for overview pages, and a raw directory for the original source files. Alongside the directories, create a schema file (call it SCHEMA.md) that defines the conventions for this wiki - what the directories are for, what naming conventions to use, what frontmatter fields pages should have, and how cross-references should be formatted. Pages should carry YAML frontmatter at the top with metadata: title, type (entity, concept, summary, overview), date created, date updated, tags, and a list of sources the page draws from. The frontmatter makes pages queryable later and lets tools like Obsidian’s Dataview plugin generate dynamic views. Cross-references between pages should use standard markdown links or wikilinks ( Build a simple CLI that lets you create a new wiki, list existing wikis, and inspect a wiki’s structure - how many pages it has, what directories exist, and what the schema says. This CLI is just for development and testing; you’ll replace it with a proper interface later. Testing:
Step 2In this step your goal is to build the source ingestion pipeline - the agent workflow that reads a source document and integrates its knowledge into the wiki. This is the heart of the system. When you drop a new source into the raw directory and tell the agent to ingest it, a multi-step workflow begins. The agent reads the source, extracts key information, discusses the takeaways with you (in interactive mode), and then updates the wiki across multiple pages. Model the ingestion workflow as a graph in LangGraph. Each node handles one concern: read the source, extract entities and concepts, identify claims and key information, write a summary page in the summaries directory, update or create entity pages for each entity found, update or create concept pages for each concept, revise topic overview pages, flag contradictions with existing content, update the index, and append an entry to the log. A single source might touch 10-15 wiki pages. The agent should be able to detect contradictions. When a new source makes a claim that conflicts with something already in the wiki, the agent should note the discrepancy on the relevant page rather than silently overwriting or ignoring it. The user should be able to see where sources disagree and make their own judgement. The agent should also identify gaps - entities or concepts referenced in the source that don’t yet have pages - and create stub pages or flag them for later attention. Think carefully about how you prompt the LLM for each of these tasks. The quality of the wiki depends entirely on the quality of the extraction and synthesis. You’ll likely need different prompts for different page types: a summary page prompt, an entity page prompt, a concept page prompt, and so on. The schema file you built in Step 1 should guide these prompts. The original source file should go into the raw directory and never be modified. The agent reads from it but never writes to it. This is your source of truth. Testing:
Step 3In this step your goal is to build the index and log - two special files that help the agent (and you) navigate the wiki as it grows. The index ( The log ( The key design decision is that the agent owns both files. Every ingestion should update the index with new pages and revised summaries. Every operation should append to the log. The agent should read the index at the start of every query to know what’s available. The agent should read the log at the start of every session to know what’s been done recently. Build the index and log maintenance into your LangGraph workflows from Step 2. After the agent finishes writing wiki pages for an ingestion, it should update the index and append to the log as the final nodes in the graph. If an ingestion fails partway through, the log should record the failure. Testing:
Step 4In this step your goal is to add semantic search over your wiki pages using vector embeddings and full-text search stored in Oracle AI Database. So far the agent navigates the wiki by reading the index and following links. That works at moderate scale, but as your wiki grows to hundreds of pages, you’ll want semantic search - finding pages by meaning, not just by browsing the catalogue. Take every wiki page (excluding the index and log themselves, and excluding raw source files), generate a vector embedding for it using your embedding model, and store the embedding alongside the page’s path, title, type, tags, and a snippet or summary in Oracle Database. The metadata fields should all be stored and indexed so you can filter by type (”only entity pages”) or by tag. Create a vector index on the embedding column for fast cosine similarity search. Also create an Oracle Text full-text index on the page content (or at minimum on the title and summary fields). Vector search finds semantically related pages even when the words don’t match. Full-text search catches exact names, technical terms, and phrases that vector search might rank lower. Together they give you robust retrieval. Think about when embeddings should be generated. Every time the agent creates or updates a page during ingestion, the new or revised page needs to be re-embedded and stored. Pages that weren’t touched by an ingestion should keep their existing embeddings. You’ll need to track which pages changed so you only re-embed those. Also think about what you embed. You could embed the full page text, but long pages might dilute the semantic signal. You could embed a summary or the first N paragraphs. You could embed both the title and the body separately and combine scores. Experiment and find what works for your test corpus - different approaches suit different types of content. Testing:
Step 5In this step your goal is to build the query system - the agent workflow that answers your questions by searching the wiki and synthesising a response. Now that you have a searchable wiki, you need the agent to put it to use. When you ask a question, the agent should follow a multi-step process: read the index to identify candidate pages, search for semantically relevant pages using the hybrid search you built in Step 4, read the most relevant pages in full, and synthesise an answer with citations to specific pages and sections. Model the query workflow as a LangGraph graph. The nodes might include: read index, hybrid search, read candidate pages, and synthesise answer. If the agent finds gaps - the question touches on something the wiki doesn’t cover well - it should say so honestly rather than speculating. The system should support different answer formats depending on the question. A comparison between two concepts might be best as a table. A timeline of events might be best as a chronological list. A straightforward explanation might be best as prose. Give the LLM the flexibility to choose the format, and provide guidance in your prompts. An important capability: the system should offer to file good answers back into the wiki as new pages. When you ask a question that generates a useful analysis, comparison, or synthesis, the agent should ask if you want to save it. If you do, it writes the answer as a new wiki page, adds it to the index, logs it, and embeds it. This way your explorations compound in the knowledge base just like ingested sources do. Support follow-up questions within a session. If you ask “tell me about X” and then “what about Y?”, the agent should understand from context that Y relates to the broader topic X is part of. LangGraph’s state carries conversation context forward between queries. Testing:
Step 6In this step your goal is to add wiki project management so you can maintain separate knowledge bases for different topics. A single wiki is useful, but you’ll likely want separate knowledge bases for different areas of your life - one for your research topic, one for book notes, one for health and fitness, one for career learning. Each should be isolated, with its own set of pages, its own embeddings, and its own agent memory. Add support for named wiki projects. Store project metadata - name, creation date, last ingestion timestamp, page count, source count - in Oracle AI Database. Each project’s embeddings and metadata should be isolated so that searches against one wiki never return pages from another. The user should be able to create a new wiki project, list existing projects, and select which project to work with. When the user starts the system, they should be able to specify a project name and immediately pick up where they left off. All wiki data should persist between runs. The markdown files live on disk in their project directories. The embeddings and metadata live in Oracle AI Database. When the user comes back and selects a project, everything should be exactly as they left it - the same pages, the same index, the same log, the same search capability. Testing:
Step 7In this step your goal is to build a linting system that health-checks your wiki and helps it stay consistent as it grows. As your wiki accumulates pages and sources, inconsistencies creep in. A page makes a claim that a newer source contradicts, but the older page was never updated. A concept is discussed across five pages but never got its own dedicated page. A page references another that was renamed or deleted. Links only go one way. Gaps appear where you have half the story. Humans abandon wikis because this maintenance burden grows faster than the value. The agent can handle it. Build a lint operation as a LangGraph workflow. The agent should walk the wiki systematically and check for: contradictions between pages (two pages make incompatible claims), stale claims (a page asserts something that a newer source has revised or disproven), orphan pages (pages with no inbound links from other wiki pages), missing pages (entities or concepts referenced but lacking their own page), broken cross-references (links that point to non-existent pages), and data gaps (areas where the wiki is thin and could benefit from additional sources). The agent should report its findings as a prioritised list: critical issues first (contradictions, stale claims), then warnings (orphans, missing pages), then suggestions (gaps, possible new sources to look for). Each issue should include the specific pages involved and a suggested action. The lint pass should also suggest new questions to investigate and new sources to look for - what’s missing from the wiki that would fill important gaps? This turns the lint operation from a bug-finding exercise into a research planning tool. Make the lint operation interactive by default. The agent presents its findings, and you accept, reject, or modify each suggestion before any changes are made. The agent should not modify pages without confirmation unless you explicitly run in auto-fix mode. Testing:
Step 8In this step your goal is to build a user interface for your LLM Wiki, providing a chat-based interface for querying and managing your knowledge base. Your wiki agent currently works through a development CLI. Now give it a proper interface. You have a choice: build a web interface (similar to the Code Sherpa challenge), a CLI chat interface, or both. The core requirements are the same regardless. The interface should provide a chat panel for querying the wiki. The user types questions in natural language, and the agent responds with synthesised answers that cite specific wiki pages. Responses should render markdown so that tables, lists, and formatted text display clearly. Citations should be clickable links that open the referenced page. The interface should support the full query workflow you built in Step 5: semantic search, multi-page synthesis, follow-up questions with context, different answer formats, and saving answers as new wiki pages. Provide a way to browse the wiki structure: a page tree or list showing categories (entities, concepts, summaries, overviews) and the pages within each. This helps the user understand the shape of their knowledge base at a glance. When the agent is processing a query, show a loading state so the user knows something is happening. LangGraph workflows can take several seconds as the agent reads the index, searches for pages, reads them, and synthesises an answer. Build a separate CLI tool for source ingestion that doesn’t require launching the full interface. A user should be able to run something like Testing:
Going FurtherYou’ve built a working personal knowledge base with an LLM agent that ingests sources, writes wiki pages, answers questions, and keeps everything consistent. Here are some ways to take it further:
Share Your Solutions!If you think your solution is an example other developers can learn from please share it, put it on GitHub, GitLab or elsewhere. Then let me know via Bluesky or LinkedIn or just post about it there and tag me. Alternately please add a link to it in the Coding Challenges Shared Solutions Github repo Request for FeedbackI’m writing these challenges to help you develop your skills as a software engineer based on how I’ve approached my own personal learning and development. What works for me, might not be the best way for you - so if you have suggestions for how I can make these challenges more useful to you and others, please get in touch and let me know. All feedback is greatly appreciated. You can reach me on Bluesky, LinkedIn or through SubStack Thanks and happy coding! John You're currently a free subscriber to Coding Challenges. For the full experience, upgrade your subscription.
|
Comments
Post a Comment