Testing whether explicit source structure changes how AI systems reason from consequential documents.
Testing whether explicit source structure changes how AI systems reason from consequential documents.
Project Details
Updated 09/08/26 · By grantmaking.aiProject Summary
Yes, I used generative AI as a drafting and editing assistant throughout this application. It helped me organize my existing research and ideas, develop concise responses, and revise wording for clarity and length. The underlying project, research framework, factual claims, analytical judgments, and substantive conclusions are my own work, and I reviewed and approved the final copy.
Most source material was written for human readers. Human readers routinely infer how facts connect, where a claim stops, what a qualification applies to, whether something is intended or demonstrated, and what remains uncertain.
AI has to reconstruct many of those distinctions from prose. When that reconstruction is wrong, the answer may still sound plausible. An unsupported relationship, lost boundary, or strengthened claim can then become part of the context for later reasoning.
My thesis is:
Source material written for humans leaves readers to understand how things connect, where they stop, and what’s uncertain. AI can’t do that reliably — those distinctions need to be made explicit.
I am investigating a source-side approach built around bounded analytical records.
The original source remains the evidence. The record sits beside it as a structured, bounded interpretation of what an analytical process was able to establish. It can preserve supported elements and relationships, source boundaries, qualifications, provenance, uncertainty, unresolved or rejected relationships, and limits on how far a conclusion can responsibly reach.
Records can be broad or highly detailed. Multiple records can address different parts of the same source, different questions, dates, or levels of resolution. Over time, they could form a reusable analytical layer around a body of source material rather than requiring important distinctions to be reconstructed from scratch every time an AI reasons from it.
Producing those records responsibly turned out to require more than one general-purpose AI analyst. I developed the AI Source Analysis Framework as a proof-of-concept architecture separating source preparation, framing, routing, specialized analysis, record production, and review.
The Framework is the process. The record is the intended outcome.
I have begun comparative development testing against source-only conditions. In one bounded test, a targeted reasoning departure appeared and compounded in the source-only condition but did not appear during the corresponding structured source-and-reasoning condition.
That is a development observation, not evidence of general effectiveness.
The proposed research asks whether reasoning from bounded analytical records—or eventually from machine-readable representations derived from them—measurably changes downstream AI behavior, which structural distinctions matter, when simpler approaches perform just as well, and where the approach fails.
The broader proposition remains:
When AI becomes an intended reasoning consumer, source representation itself becomes part of the reasoning environment.
What are this project's goals? How will you achieve them?
The immediate research goal is to determine whether bounded source-side analytical records can reduce unsupported reconstruction and the propagation of those departures through later AI reasoning.
Finding the right information and reasoning correctly from it are related problems, but they are not the same problem.
A retrieval system may find the correct passage. A long-context model may receive the entire document. A structured system may represent a relationship. There is still another question:
What does the source actually support connecting?
A source can establish two facts without establishing a relationship between them. It can describe an intended result without establishing that the result occurred. A statement can apply within one boundary without supporting a broader conclusion. It can leave identity, causation, timing, hierarchy, exceptions, or other relationships unresolved.
During repeated summarization and conversation, these distinctions can change while the resulting AI output remains fluent and internally coherent.
I use drift to describe the propagation of an earlier reasoning departure through later reasoning. The original problem may be small: an unsupported relationship, a lost qualification, an expanded boundary, an incorrect attribution, or a strengthened claim. Once incorporated into the conversational context, that interpretation can become a premise for what follows.
The current proof of concept attempts to move selected structure upstream by producing records that explicitly preserve what was supported, what was not established, the relevant source boundary, provenance, uncertainty, and claim limits.
During the proposed research I will expand comparative testing across source types and AI systems; reproduce and classify failures involving relationships, scope, provenance, uncertainty, temporal state, and claim strength; compare record-supported conditions with source-only conditions and simpler alternatives; test whether added structure can overconstrain or worsen reasoning; and document negative and null results alongside successful demonstrations.
A recent development test used the European Union Artificial Intelligence Act. Two AI environments received the same underlying source and conversational sequence under different source-and-reasoning conditions. A targeted reasoning departure appeared early and compounded in the source-only condition. It did not appear during the tested sequence in the structured condition.
That experiment does not isolate every variable or establish a general causal effect. Its value is narrower: source representation can be changed, tested, compared, and revised rather than treated as an invisible constant.
The next research stage is also more concrete than it was when I began the project.
The Framework has reached a deliberately stabilized manual research stage. The question is no longer simply whether its sequence can technically be automated. The question is which analytical invariants must survive formalization and automation.
Those include source-boundary preservation, independent support for relationship endpoints, relationship warrant, relation typing, qualification and modality, claim limits, provenance, uncertainty, and the ability to preserve rejected or unresolved relationships rather than forcing every proposed connection into a positive structure.
I also want to test whether the analytical record can remain the underlying artifact while being transformed into different technical representations.
A possible pathway is:
Source material → Framework examination → bounded analytical record → machine-readable serialization → retrieval, graph, or other downstream infrastructure
Under that model, the graph or other machine representation would be an export of the analytical record rather than the thing that determines what the source was entitled to support.
Success does not require validating the current Framework as designed. A useful result could show that only certain distinctions matter, that some controls do not survive automation, that simpler techniques perform equally well, that particular source domains benefit while others do not, or that the approach contributes little beyond existing methods.
Those are empirical questions.
The longer-term question remains:
If AI is becoming a reasoning consumer of human knowledge, should we continue asking it to reconstruct the structure of that knowledge every time it reads, or should we begin designing records for the reader it has become?
How will this funding be used?
I am seeking $50,000 for approximately twelve months of independent research and development.
Most of the funding would support the time required to make this my primary project. The remainder would support AI model and API access, software and computing, acquisition and preparation of source material, comparative testing, evaluation documentation, development of machine-readable record representations, selective automation experiments, and public research materials.
The proposed next stage has six related components: formalizing the analytical invariants that need to be preserved; developing machine-readable representations of the records; testing selective automation without silently changing those invariants; comparing increasingly automated implementations with the stabilized manual process and simpler alternatives; exploring integration with existing retrieval or graph infrastructure; and evaluating downstream reasoning, error propagation, consistency, cost, and throughput.
At the $25,000 minimum, I could conduct a reduced program using fewer source domains, models, and comparative conditions.
At the $50,000 goal, I could sustain the research for approximately twelve months, broaden the source and model set, improve the evaluation methodology, formalize and test record representations, document failures more systematically, and prepare reusable demonstrations and public research materials.
Formal evaluation, implementation, specialized analytical development, scaling, and independent testing also require expertise beyond my own. Where resources permit, the project would benefit from technical and research collaboration rather than assuming that one person should perform every specialized part.
The funding is for research and evaluation rather than building a consulting practice or sales operation.
Who is on your team? What's your track record on similar projects?
I am Greg R. Welch, an independent researcher and information designer with more than three decades of experience designing, organizing, publishing, and presenting information across print and digital systems.
I came to this research through use rather than formal machine-learning or computer-science training.
For several years I tried incorporating generative AI into practical client services. Some ideas encountered adoption problems. Others progressed far enough to expose failures I did not consider reliable enough to build a responsible service around.
Eventually I stopped asking:
What else can I use AI for?
and began asking:
Why does this keep failing?
My background affected where I looked. I had spent decades designing information for human readers. When AI became another reader, I began asking whether the source environment itself needed to change.
That led first to the idea of producing a responsible analytical record and then to the architecture required to produce one.
I have developed the AI Source Analysis Framework independently and funded the work myself. The current result is a working proof of concept for producing structured analytical records from human-oriented source material for later AI reasoning.
The architecture has changed substantially in response to failure.
An early attempt to use one general analytical approach proved inadequate because different kinds of source structure require different forms of examination. That led to specialized analytical instruments and a routing layer rather than forcing all material through one model.
Very detailed treatment of large sources also became impractical, leading to explicit levels of analytical resolution.
Testing exposed failures involving unsupported relationships, provenance, source boundaries, claim strengthening, and conversational drift, which led to additional controls and repeated redesign.
The Framework now separates source preparation, framing, routing, specialized analysis, record production, and review, while permitting outcomes such as insufficient evidence, unresolved relationships, rejected structures, and cases for which no current analytical instrument exists.
The records themselves are also becoming a separate research object. I am developing methods for multiple analytical resolutions, continuity across later records, revision and comparison, and possible future serialization into retrieval, graph, or other machine-readable environments.
I am the sole project lead. There is currently no company, university, research laboratory, employee, or institutional sponsor behind the work.
The next stage requires formal evaluation and technical collaboration beyond what I can responsibly claim to provide alone.
What are the most likely causes and outcomes if this project fails?
The strongest possibility is that source-side analytical records matter less than I currently suspect.
More capable models, better prompting, ordinary retrieval techniques, automated graph construction, or much simpler source treatments may produce comparable results at substantially lower cost.
The current Framework may also contain controls that are useful during manual development but unnecessary, impractical, or too expensive to preserve at scale.
There is also a more fundamental risk.
A persistent analytical record can preserve a useful distinction, but it can also preserve an error. If the process incorrectly establishes a relationship, boundary, classification, or interpretation and that conclusion becomes reusable structure, later AI systems may encounter the error repeatedly rather than generating it only once.
The source-side approach therefore does not eliminate an error problem. It may relocate part of that problem upstream.
Human judgment during record preparation could introduce selection bias. Explicit structure could overconstrain later reasoning. Analytical records could become too expensive to justify at useful resolution. Machine serialization could strip away qualifications that were preserved successfully in the human-readable record. Automation could recreate the same inference problems the Framework was designed to expose.
The existing evidence is small and developmental. It does not demonstrate general reliability, causal effect, or superiority to RAG, GraphRAG, ordinary source use, or other alternatives.
Those are not merely risks to the project. They are central research questions.
If the current Framework fails but the research identifies a smaller set of source-side distinctions that materially affect later reasoning, that would still be useful.
If careful evaluation shows that simpler approaches perform just as well, that would also be useful.
If the work identifies source types or conditions under which persistent analytical structure causes more harm than benefit, that is also a result worth documenting.
The objective is not to prove that the current proof of concept is correct. It is to determine how far the source-side proposition actually holds.
How much money have you raised in the last 12 months, and from where?
I would leave this section essentially unchanged, because the revised positioning does not alter the funding facts.
Use the existing current text unless the status of any pending application has changed since the proposal was posted. The live page currently reports $0 in outside funding and identifies the pending applications/inquiries.
Closing disclosure
Replace the current final italic note with:
This proposal describes the research proposition, current proof of concept, bounded analytical records, and proposed evaluation program at a high level. It intentionally does not publish all internal production methods used to create the current records. The Framework specifies analytical procedures and controls; it does not guarantee source truth, eliminate hallucination, mathematically constrain model behavior, or establish a demonstrated performance advantage over other approaches.
That remains consistent with the project's public-positioning boundaries and the position paper's explicit proof-of-concept limits.
September 1 project update
After editing the proposal itself, I would post a new comment rather than altering or deleting the August 25 update:
01SEP26: Project update — clarifying the central research artifact
Since my August 25 technical update, I have continued refining how I describe the project.
The underlying research proposition, funding request, and need for comparative evaluation have not changed. What has become clearer is the distinction between the analytical architecture and the thing it is intended to produce.
The Framework is the process. The record is the intended outcome.
The AI Source Analysis Framework began as an attempt to make AI analysis of source material more disciplined. As the work developed, I increasingly saw the durable artifact as the bounded analytical record that remains beside the source: what was supported, how elements were related, where the boundary was, what remained uncertain or unresolved, where the evidence came from, and how far the resulting claims can responsibly reach.
That clarification changes the emphasis of the project.
The Framework is a proof-of-concept production architecture. The larger research question is whether records produced through that kind of governed source-side examination can become a useful analytical layer for later AI reasoning.
It also makes the next technical stage clearer.
A record need not remain only a human-readable document. If its analytical distinctions survive formalization, the same record information may eventually be serialized into JSON, graph tables, retrieval inputs, or other machine-readable representations.
That leads to a useful experimental distinction:
Does structured representation affect later reasoning?
and, separately:
Does the way that structure earned its place in the representation affect later reasoning?
The next research stage is therefore formal evaluation, machine-readable record representation, selective automation, integration experiments, comparative testing, and independent technical examination.
None of this establishes that the Framework will outperform conventional retrieval, automatically generated graphs, better prompting, or simpler source treatments.
That remains the question to test.
— Greg R. Welch
People
Updated 09/08/26 · By grantmaking.aicreator
Funding Details
- -
- -
- -
- -
- -
- -
- -
- $50,000
- -
- -
25AUG26: Project update — external technical comparison and next research stage
Since posting this proposal, I have identified several public technical programs that provide useful external comparison points for the research question described here.
I developed the AI Source Analysis Framework independently and encountered Microsoft Research’s GraphRAG only after developing the Framework and submitting these proposals. I am not making a priority claim; what matters is the independent convergence on a related problem.
GraphRAG, Anthropic’s Contextual Retrieval, and OpenAI’s knowledge-retrieval work all address, in different ways, how external information is structured and supplied to models for later reasoning.
The comparison has sharpened the question behind this project:
What persistent structure is the source actually entitled to support before that structure is supplied to later AI reasoning?
This matters because model-generated interpretation can itself become persistent—as a relationship, graph edge, contextual annotation, summary, or classification that later models may retrieve and reason from.
The issue is therefore not simply whether structured information is useful. It is how that structure earned its place there.
This does not change the Manifund proposal. The project remains an investigation of source representation as a variable in AI reasoning, including the possibility that simpler approaches may perform just as well.
What has changed is that there is now a clearer external comparison. A matched evaluation could compare raw or conventionally retrieved source material, automatically generated graph structure, Framework-produced bounded analytical records, and potentially the same Framework findings serialized into graph-compatible form.
That would help distinguish:
Does structured representation affect reasoning?
from:
Does the way that structure was established affect reasoning?
The Framework has also reached a deliberately stabilized manual research stage. The next question is not simply whether the workflow can be automated, but which analytical invariants must survive automation—including source boundaries, relationship warrant, qualification, claim limits, provenance, and unresolved or rejected relationships.
This gives the proposed work a more concrete next stage: machine-readable representation, selective automation, controlled comparison, and evaluation of downstream reasoning, error propagation, variability, cost, and throughput.
I am not claiming that the Framework will outperform GraphRAG, conventional RAG, or simpler source treatments. That is what the proposed evaluation would need to determine.
— Greg R. Welch