Verifying Behavior and Finding the Edge Cases AI Misses

In Part 1, you learned the foundational AI skill: reading code carefully and asking what it actually does. This part’s skill is the natural follow-up. Once you can read code and predict its behavior on a given input, the next question is: what about all the other inputs? Especially the ones the person who wrote the code did not think about.

This chapter is about being systematic with that question. The skill applies to all code, but it matters also for AI-generated code, because AI-generated code often works on the obvious cases, but breaks quietly on others.

The Stance Shift

In the previous part, when you read a piece of code, the question was: “what does this do?”

In this part, the question grows a second half: “what does this do on every input it might actually see?”

Surface-level checking — “does the code run? does it produce output? does the output look right?” — is what most people do. It catches the obvious bugs. The bugs that hide behind surface-level checking are the ones the test inputs did not happen to expose.

When you write code yourself, you have a small advantage here: you know what you were thinking, so you have at least some idea of what cases you might have missed. When you read code that an AI generated, you do not have that. The AI may have considered cases you wouldn’t, or skipped cases you would have remembered without thinking. You have to be more systematic than you might be with your own code.

The good news is that the systematic version is not hard. It is a short checklist applied to every piece of branching or looping code.

A Checklist of Edge Cases

For any function or loop that processes a list or a value, run through this list:

The point is not that every case applies to every program. The point is to scan the checklist every time, so the cases that do apply get noticed before they bite you.

The next three sections walk through three short pieces of AI-generated code. Each works on the case the AI was clearly thinking about. Each breaks on one or more cases from the checklist.

Example 1: The Plausible-Looking Max Function

Suppose you ask an AI assistant: “write Python code that prints the longest session in a list of minutes.” It produces this:

minutes_list = [45, 30, 90, 60, 75]

longest = 0
for minutes in minutes_list:
    if minutes > longest:
        longest = minutes

print(f"Longest: {longest}")

This program prints Longest: 90. The AI tests it in its head, sees the right answer, and is satisfied. So might you.

But run the checklist. Empty list: what if minutes_list = []? The loop runs zero times, longest stays at 0, and the program prints Longest: 0. That is wrong — the longest of nothing is not 0, it is undefined. The program should probably say something like “No sessions yet” instead.

Zero or negative values: what if every session is somehow 0? The condition minutes > longest is 0 > 0, which is False. So longest never gets updated, and the program prints Longest: 0. Misleading, but in this case it happens to be the right number. What if the list contained negative numbers? Then longest = 0 is a wrong starting value, and the result would be 0 for any list of negative numbers — also wrong.

The fix here is the version from Counting, Accumulating, and Searching: handle the empty case explicitly, and start longest at minutes_list[0] instead of 0.

The general lesson is that this kind of bug does not show up in a single test. The AI’s version “works” if you only feed it lists like the one it had in mind. The checklist is what catches the rest.

scores = []

best = 0
for score in scores:
    if score > best:
        best = score

print(f"Best score: {best}")

Example 2: The Off-by-One Classifier

Suppose you ask an AI assistant to write code that classifies a session as short, solid, or long, with the thresholds at 30 and 90 minutes. It produces this:

minutes = int(input("Minutes: "))

if minutes < 30:
    print("Short.")
elif minutes < 90:
    print("Solid.")
else:
    print("Long.")

Looks fine. Run it with 15, 60, and 120. You get Short., Solid., Long. Perfect.

Now run the boundary values check. What does the program do at exactly 30? 30 < 30 is False, so we move on. 30 < 90 is True, so the answer is Solid. What about exactly 90? 90 < 30 is False. 90 < 90 is False. So we fall through to else, and the answer is Long.

Did the AI mean for 30 to be “Short” or “Solid”? Did it mean for 90 to be “Solid” or “Long”? The code makes a choice — 30 is “Solid” and 90 is “Long” — but the original request didn’t say anything about the boundary, and the AI just picked one. If you intended 90 to be the largest “Solid” session and only sessions over 90 to be “Long”, this code is wrong, and a casual reading would not catch it.

This kind of bug is everywhere in AI-generated branching code. The thresholds are usually approximately right. The choice between < and <= at the boundaries is usually a coin flip. The AI cannot read your mind about whether you wanted 30 to be a short session or a solid one, so it just picks. Your job, when reading code like this, is to look at every comparison operator and ask: “is this the right one?”

30

Short.

90

Solid.

Example 3: The “Type ‘done’ to Quit” Loop

Suppose you ask an AI assistant for a loop that asks the user for subjects until they type done. You get:

subjects = []

while True:
    subject = input("Subject (or 'done' to quit): ")
    if subject == "done":
        break
    subjects.append(subject)

print(f"Logged {len(subjects)} subjects.")

This works on the obvious case. Run it, type Math, Physics, done, and the program prints Logged 2 subjects.

Now run the unusual input format check. What if the user types Done with a capital D? "Done" == "done" is False, so the loop does not exit. The program adds "Done" to the list of subjects and keeps asking. The user is confused; the AI did not consider it.

What if the user types done with a trailing space? Same problem: "done " == "done" is False, and "done " ends up in the subject list.

The fix is one line — normalize the input before comparing:

while True:
    subject = input("Subject (or 'done' to quit): ").strip().lower()
    if subject == "done":
        break
    subjects.append(subject)

This is the kind of fix that is easy to miss because the original code runs without errors. There is no crash. There is no traceback. The user just gets a confusing experience. Only the checklist surfaces this kind of bug.

A Reference Table of Failure Modes

Bringing the three examples together, here are the most common control-flow failure modes in AI-generated code. Treat this as a quick reference when reviewing loop and branch code from any source:

A useful habit is to keep this table in mind whenever you read AI-generated code. You do not need to check every row on every program. You need to check the rows that are relevant to what the program is doing.

Below, you get to practice asking for code from an AI assistant. When you get the code, study it to identify any edge cases it misses. If you do not find any, submit the program for evaluation. Running the AI-generated code does not currently properly work, but you can try it in the sandbox.

Math
Art
Done

Where You Are in the Nine-Skill Arc

In Part 1, the AI skill was reading: understanding what a piece of code actually does. This part’s skill is verifying: confirming that what it does is what it should do, on every input it might see.

These two together form the foundation of every later AI-collaboration chapter:

You can think of Part 1 as “read the code”. Part 2 as “stress-test the code”. Together they answer: does this code do what I think it does, and does it keep doing it when the input is unusual? Every later part builds on this base.

Verifying Is Cheap

The same stance from Part 1’s reading chapter still applies: you are being a programmer first, regardless of who or what produced the code. What is new in this part is that the checklist is now what you do instead of trusting. A bug found by the checklist is a small specific win. A checklist that comes up empty is also useful — the code held up, and you can use it with confidence. The cost of running it is low, and as the code in later parts grows more complex, the trade-off only gets more favorable.

Check Your Understanding

1. Why is “the code runs and produces output” not the same as “the code is correct”?
2. List three cases from the edge-case checklist that you would check on any loop that processes a list.
3. In Example 2, what change would you make to the comparison operators if you wanted 30 to be classified as “Short” and 90 to be classified as “Solid”?
4. In Example 3, why does subject == "done" fail when the user types Done? What is the fix?
5. In one sentence, what is the AI-collaboration skill this part was about?