Professor Franklin
Accessibility settings

Preferences are saved in this browser when storage is available.

Chapter 2 · Reading 6 of 9

Trust boundaries in AI-assisted workflows

Security+ SY0-801 · Objectives 2.6

The aid office wants an assistant to summarize long appeals. A student’s attachment contains a paragraph written as instructions to the reader. A human reviewer understands that it belongs to the appeal. What tells the application where the student’s authority ends? Follow one request through the system before deciding that the assistant is safe to use.

Treat externally supplied text, retrieved documents and model output as untrusted inputs. Indirect prompt injection crosses a boundary when an assistant mistakes content inside a document for an authorized instruction. Training-data poisoning changes the examples a model learns from. Inference-time manipulation changes an input presented to an already trained model. These conditions require different evidence and different owners.

Inventory the assistant’s sources and tools

Inventory models, training sources, retrieval indexes and callable tools. Filter sensitive material before submission and enforce authorization outside the model. Record which source documents and tool decisions contributed to a result. Repeat evaluations after changes and include a case where the assistant must decline an action that exceeds the user's authority.

Map one assistant request end to end

Suppose a help desk assistant summarizes a case and can send a reply. Identify the authenticated user, retrieval service, documents returned, generated recommendation and outbound connector. At each step ask whose identity is used and where approval is enforced. An overprivileged retrieval account can disclose another user's material before the outbound connector is ever called. A restricted retrieval service can still feed misleading content into a perfectly authorized conversation.

Give each boundary its own test. With synthetic accounts, verify that the retriever returns only the current user's records, that a reviewer can trace each factual claim to its source, and that the connector rejects an operation outside the approved scope. Save the decision and a minimized audit reference. Routine troubleshooting rarely requires storing every private prompt in full.

Why accurate summaries can still expose data

Consider two pilot versions. The first retrieves every appeal in a shared folder and asks the model to mention only the current student. The second limits retrieval to records the counselor may read before any text reaches the model. The first depends on generation behaving perfectly after disclosure has already occurred inside the application. The second gives the retrieval service a narrower job that administrators can inspect and test.

Now imagine the reply cites an award that does not exist. Tight retrieval permissions did their job, yet the counselor still needs a way to check the claim. Save the authorized source reference alongside each consequential statement and let the counselor open the underlying record. This second question concerns the quality and consequences of a recommendation, even when every access check worked.

Document assistant boundary

Inspect the three records, then use the evidence to answer the question.

1. Record A

Help desk assistant reads uploaded invoices

2. Record B

An invoice contains text asking to email student records

3. Record C

A connector can send external mail without review

Apply the reading

What evidence supports your decision in the document assistant boundary exercise?

Compare your reasoning

The invoice supplies untrusted content. The connector supplies authority. Restrict its scope and require approval before external delivery.