Chapter 2 · Reading 7 of 9
Evaluating bias, hallucinations and explainability
Security+ SY0-801 · Objectives 2.6
Suppose an assistant flags student employees who sign in after midnight. Day-shift supervisors praise its accuracy. The people repeatedly asked to prove their identity are library closers and residence staff. Whose experience should count when the institution evaluates the pilot?
Test a model as a decision aid
An assistant recommends disabling a student account and cites a policy section. A hallucination is plausible generated content unsupported by the underlying evidence. Open the actual policy, check its version and locate the cited clause. Compare the recommendation with raw events. A fluent explanation, a high model confidence score and a fabricated citation can all arise from the same unsupported inference.
Bias can enter through sampling, historical labels, proxies and deployment context. If most training examples come from daytime staff, night-shift users may look anomalous simply because their schedules are underrepresented. Build an evaluation set with independently checked labels. Calculate false positives among legitimate events and false negatives among truly harmful events for relevant groups. Report counts alongside percentages. Small groups can have unstable rates. Examine alternate explanations and consider the harm of each error.
Explain the decision to the affected person
Explainability asks what information helps a reviewer understand a decision and its limitations. An analyst needs contributing observations, model and policy versions, uncertainty and a path back to original evidence. A generated rationale is itself another output to verify. Give a human reviewer authority to override a recommendation and record the reason. Monitor outcomes after deployment as work patterns change.
Privacy and ethical review begin before collection. Define a legitimate purpose, minimize identifying fields and set retention and access rules for prompts and review logs. Test with synthetic or appropriately authorized records. Give affected users a meaningful way to correct inaccurate information and appeal consequential decisions. Track whether the appeal actually reverses an incorrect restriction and whether repeated errors disproportionately affect one group.
Read the denominator
Say four day workers and six night workers were incorrectly flagged. Raw totals make the difference look small. If there were one hundred legitimate day cases and twenty legitimate night cases, the difference is substantial. The figures also leave questions open. Were the two sets collected during the same dates? Did an independent reviewer label legitimate work consistently? Did people stop reporting mistakes after repeated challenges?
Interview affected staff, review a sample of both flagged and unflagged cases and test whether a change improves outcomes for each group. A reviewer should be able to explain a specific account decision in terms the worker can verify. If performance changes abruptly between two releases, the next task is to find out what changed in the underlying system.
Inspect the three records, then use the evidence to answer the question.
1. Team A
Team A had 4 false alerts among 100 legitimate events.
2. Team B
Team B had 6 false alerts among 20 legitimate events.
3. Ground truth
Reviewers used the same independently checked labels.
Apply the reading
What information makes a disparity in error rates interpretable?
Compare your reasoning
Report the denominators, independently verified labels, uncertainty and relevant differences between populations.