Eighty calls and still no reliable profile

The profile failed before the research was done

A Jefferson Health and Lehigh Valley Health Network profile took more than 80 individual tool calls. Roughly 30 failed. The work consumed roughly 100,000 tokens and more than five minutes, then still returned guessed bed counts, missing facilities, no service-capability data, and an incomplete outpatient inventory. ^{1 1}

Those figures are a first-party design observation from my own development record. They do not compare the system against other products, and they do not prove that the later approach solved every data problem. They document a narrower failure: even with public sources and capable software, a basic institutional profile was expensive to assemble and still unreliable in ordinary research use.

I knew the problem before I could name the architecture. AJHCS interns and research assistants would reach the same wall from different directions. A question about a health system could require federal datasets, audited statements, facility rosters, quality measures, market geography, leadership records, and current reporting. The records were public in the legal sense, but that did not make them easy to find, join, challenge, or cite. I had run into the same problem in academic work. Access to a source was only the beginning; a researcher still needed the time and technical knowledge to turn scattered records into an answer that another person could inspect.

That gap is why I created Open-Informatics. I did not want another dashboard that hid its inputs or another general assistant that could write past missing evidence. I wanted research infrastructure that made a bounded claim, retained its source trail, admitted what it could not retrieve, and left the final judgment with a person.

Build the retrieval layer first

Healthcare Data MCP grew from that failed profile. Its job is deliberately plain: turn difficult public healthcare sources into callable, focused tools. Responses carry source metadata and structured fields. Caches have declared boundaries. Errors include recovery information where the underlying surface supports it. The system can expose missingness instead of filling an empty field with a plausible answer.

This does not make public healthcare data complete. Source agencies publish on different schedules. Names and identifiers drift. A health system can change faster than annual and quarterly files. Some data that appears public still takes considerable work to acquire or interpret. Building a connector moves the research boundary; it does not erase it.

The design change was still meaningful. The failed profile had asked one model to discover sources, remember identities, recover from calls, reconcile facilities, and write a conclusion inside one long session. Healthcare Data MCP moves repeatable retrieval and identity work into maintained software. A researcher can inspect the tool contract and the returned provenance instead of trusting the fluency of the final paragraph.

Judgment needs its own boundary

Retrieval alone does not know which question matters. Healthcare Agents provides specialist workflows for healthcare administration questions and evidence packs that make source expectations visible. The agents can frame a workup, identify the relevant domain, and organize a handoff. They cannot make a clinical, legal, coding, billing, audit, or compliance determination on a person's behalf.

That boundary matters to the architecture. A source can be current and still be used badly. A specialist prompt can be careful and still lack the local policy, private contract, patient context, or accountable authority needed for a decision. The system should make those limits easier to see, not use technical sophistication to talk around them.

The useful pattern became retrieval followed by a bounded workup: bring back the evidence, show where it came from, name the gaps, and give a human enough structure to examine the recommendation.

Research to public use

One question, four kinds of infrastructure.

1. 01

   Evidence

   Healthcare Data MCP

   Turns scattered public records into bounded calls that retain source metadata, recovery hints, and explicit gaps.

2. 02

   Judgment

   Healthcare Agents

   Routes an operating question to a narrow specialist workflow, then returns a workup for human review.

3. 03

   Reference

   USHSO

   Keeps health-system facts, sources, missingness, and interpretation in a comparative public record.

4. 04

   Distribution

   AJHCS

   Publishes research and analysis through an institution whose scholarly decisions remain independent.

Reference work makes research reusable

The United States Health Systems Observatory, or USHSO, is the reference and synthesis layer. It keeps a health system's facts, sources, missingness, and interpretation in a form that can be revisited rather than rebuilt from scratch for every question.

The Jefferson Health profile became an early test of that idea. Historical generated report artifacts remain in the project's audit trail, but they are retired as an active data-entry contract. The current profile engine uses catalog-backed facts, reusable sources, typed missingness, and supporting knowledge records. That change is less glamorous than publishing a new report. It is also what lets a correction propagate through a maintained reference instead of disappearing inside yesterday's prose.

Distribution is institutional work

AJHCS is the distribution layer in this shared initiative. It gives healthcare strategy research a public home through articles, reports, interviews, podcasts, and other editorial work. Open-Informatics designed and maintains the current publishing system. AJHCS retains authority over its scholarly decisions.

That separation protects both projects. A lab should be able to improve publishing infrastructure without turning software ownership into editorial control. A publication should be able to use common research tools without presenting their output as an automatic scholarly judgment. The handoff is technical and institutional at the same time.

What exists, and what still does not

Open-Informatics now has working public repositories, a maintained research product, and publishing infrastructure in production. It also has unfinished source coverage, delayed datasets, identifiers that need reconciliation, and software that must be tested whenever a provider changes an API or a public file changes shape.

The four parts do not form a push-button research machine. A question may use one layer or several. A person still decides what to ask, whether the sources are fit for the claim, what uncertainty belongs in the answer, and whether the result is ready to publish.

That maintenance burden is part of the point. Researchers should not each have to rediscover the same broken download, reconcile the same organization names, or guess which field changed. Shared infrastructure can absorb some of that repeated work. It cannot remove the need for skepticism.

I created Open-Informatics because students and researchers should be able to ask serious questions of complex institutions without first buying an enterprise data contract or becoming a specialist in every source system. The work succeeds when they can produce research that was previously out of reach and show another person exactly how they got there.

Notes

1. Healthcare Data MCP, "Health System Profiler MCP Server: Design Document" (March 2, 2026). This first-party design record documents the Jefferson Health and LVHN profiling failure and the proposed source architecture. Its before-and-after table is a design target, not an independent performance evaluation. Read the design record. ↩
2. Project documentation reviewed July 21, 2026. The Healthcare Data MCP repository documents structured responses, source metadata, cache boundaries, recovery behavior, and current limitations. The Healthcare Agents repository documents specialist workups, evidence packs, evaluation limits, and human authority boundaries. The Open-Informatics site documents the evidence, judgment, reference, and distribution model. Volatile repository and product counts are intentionally omitted.