Professor Franklin
Accessibility settings

Preferences are saved in this browser when storage is available.

Chapter 2 · Reading 7 of 9

Evaluating bias, hallucinations and explainability

Security+ SY0-801 · Objectives 2.6

Suppose an assistant flags student employees who sign in after midnight. Day-shift supervisors praise its accuracy. The people repeatedly asked to prove their identity are library closers and residence staff. Whose experience should count when the institution evaluates the pilot?

Test a model as a decision aid

An assistant recommends disabling a student account and cites a policy section. A hallucination is plausible generated content unsupported by the underlying evidence. Open the actual policy, check its version and locate the cited clause. Compare the recommendation with raw events. A fluent explanation, a high model confidence score and a fabricated citation can all arise from the same unsupported inference.

Bias can enter through sampling, historical labels, proxies and deployment context. If most training examples come from daytime staff, night-shift users may look anomalous simply because their schedules are underrepresented. Build an evaluation set with independently checked labels. Calculate false positives among legitimate events and false negatives among truly harmful events for relevant groups. Report counts alongside percentages. Small groups can have unstable rates. Examine alternate explanations and consider the harm of each error.

Explain the decision to the affected person

Explainability asks what information helps a reviewer understand a decision and its limitations. An analyst needs contributing observations, model and policy versions, uncertainty and a path back to original evidence. A generated rationale is itself another output to verify. Give a human reviewer authority to override a recommendation and record the reason. Monitor outcomes after deployment as work patterns change.

Privacy and ethical review begin before collection. Define a legitimate purpose, minimize identifying fields and set retention and access rules for prompts and review logs. Test with synthetic or appropriately authorized records. Give affected users a meaningful way to correct inaccurate information and appeal consequential decisions. Track whether the appeal actually reverses an incorrect restriction and whether repeated errors disproportionately affect one group.

NIST Generative AI Profile

Read the denominator

Say four day workers and six night workers were incorrectly flagged. Raw totals make the difference look small. If there were one hundred legitimate day cases and twenty legitimate night cases, the difference is substantial. The figures also leave questions open. Were the two sets collected during the same dates? Did an independent reviewer label legitimate work consistently? Did people stop reporting mistakes after repeated challenges?

Interview affected staff, review a sample of both flagged and unflagged cases and test whether a change improves outcomes for each group. A reviewer should be able to explain a specific account decision in terms the worker can verify. If performance changes abruptly between two releases, the next task is to find out what changed in the underlying system.

Alert review sample

Inspect the three records, then use the evidence to answer the question.

1. Team A

Team A had 4 false alerts among 100 legitimate events.

2. Team B

Team B had 6 false alerts among 20 legitimate events.

3. Ground truth

Reviewers used the same independently checked labels.

Apply the reading

What information makes a disparity in error rates interpretable?

Compare your reasoning

Report the denominators, independently verified labels, uncertainty and relevant differences between populations.