Knowledge-pipeline architecture

The system in one view

Final plan information moved downstream from a regulated document cycle into a checked knowledge layer, a routed retrieval experience, and two operating teams. The source documents remained authoritative. Neither CMS nor actuaries certified the conversion or assistant, and the model never became the source of truth.

Historical system · 2024–2025

From final plan material to source-traceable answers

Upstream authority

Final EOC, ANOC, Medicare Summary of Benefits, and applicable ACA benefit-summary material, downstream of the regulated document cycle.

1. 1

   Begin with final materials

   Final EOC, ANOC, Medicare Summary of Benefits, and applicable ACA benefit-summary material remained the upstream authority.

2. 2

   Structure the source

   Plan material became structured SharePoint and Excel knowledge with product, plan-year, document, and section context.

3. 3

   Compare, resolve, and review

   Python checked character content, counts, and position for every converted item. Discrepancies received manual review, followed by a complete visual review of every page across the included lines of business.

4. 4

   Route before retrieval

   A Copilot Studio worktree narrowed the relevant product, plan, year, market, document, and topic before GPT-4o retrieval.

5. 5

   Meet staff where they worked

   Teams was the primary interface. SharePoint and PowerApps provided alternative ways into the same operating model.

6. 6

   Cite the exact section

   Answers required inline citations to exact SharePoint sections, with onward links to the original PDFs.

7. 7

   Fall back to searchable sources

   When the assistant could not support an answer, staff could search SharePoint. There was no automatic live-agent handoff.

8. 8

   Support operating work

   Medicare Stars Outreach and Product used the resulting knowledge system for plan-material questions and change work.

What made the architecture trustworthy

The difficult boundary sat between the final member-facing documents and the searchable knowledge layer. A successful text conversion was not enough. Character and position checks could expose missing, duplicated, or displaced content; manual discrepancy review could determine whether the problem belonged to the conversion or the source; and complete visual page review could catch structure that counts alone could not prove.

The worktree then constrained retrieval. It established context before GPT-4o searched, reducing the chance that similar plan names, years, or benefit language would be treated as interchangeable. Inline citations preserved the short path from an answer to an exact SharePoint section, then onward to the original PDF.

Scope and boundaries

Boundary	What the historical system included	Limits
Corpus	2024–2025 EOC, ANOC, Medicare Summary of Benefits, and applicable ACA benefit-summary material, with formulary context inside plan documents	A complete drug-level formulary or standalone Medicaid/CHIP corpus
Validation	Every-item Python character, count, and position comparison; manual discrepancy review; complete visual page review	CMS or actuarial certification of the conversion, knowledge layer, model, or assistant
Retrieval	Copilot Studio worktree routing before GPT-4o retrieval	A model that inferred every routing branch without constraints
Interfaces	Teams primary; SharePoint and PowerApps alternatives	A current-branded interface or claim of present operation
Traceability	Mandatory exact-section citations and onward links to original PDFs	An answer whose confidence replaced source evidence
Fallback	Searchable SharePoint for unsupported answers	Automatic live-agent handoff
Use	Medicare Stars Outreach and Product work	Adoption counts, causal outcomes, Star Ratings effects, or enterprise-wide use

Read the architecture alongside the workbook

For present-day planning, the plan-material validation workbook covers source authority, routing, review, fallback, and release decisions. Return to the JHP Knowledge Pipeline for the project context.