Armada AI

Almanac: air-gapped document Q&A

An on-device RAG application designed for document question answering on Armada edge hardware without internet access.

Sole engineering owner across document ingestion, local model serving, retrieval and cited question answering.

Purpose

Built as a proof of capability and sales-enablement application for edge hardware, with a default documentation corpus and support for a prospect’s own documents. Answers cite their sources, and low-confidence retrieval is handled explicitly.

Engineering

Developed local model serving, document ingestion, retrieval, reranking and cited-answer generation. The document pipeline covers parsing, layout, OCR, tables, figures, chunking and embeddings.

Moved model serving from vLLM to Ray to run multiple model types concurrently, with per-model resource control and on-demand loading and unloading. Integrated access control with existing edge roles.

Documents and questions

Workflow overview

Document ingestion

  1. Input

    Documents

    The selected corpus

  2. Process

    Parse & structure

    Layout, OCR, tables and figures

  3. Process

    Chunk & embed

    Prepare searchable passages

  4. Store

    Knowledge index

    Indexed passages for retrieval

Supplies indexed passages

Knowledge indexRetrieve & rerank

Question answering

  1. Input

    Question

    A document question

  2. Process

    Reformulate

    Prepare the retrieval query

  3. Process

    Retrieve & rerank

    Select context from the index

  4. Output

    Cited answer

    Local generation with sources

Ingestion prepares the knowledge used during question answering. This is the local workflow; installation and evaluation status are described below.

Model adaptation

Fine-tuned models for Almanac using supervised fine-tuning (SFT), LoRA and QLoRA.

Current status

Document ingestion, question answering, evaluation tooling and inference planning work in an isolated VM. A fresh installation with the internet disconnected has not yet been tested. Retrieval and generated-answer quality evaluation is underway; completed answer-quality results are not yet reported.

Tools

SFT · LoRA · QLoRA · Ray · Local model serving · Document processing · Hybrid retrieval · RBAC