The Difference Between Reading a Calculation and Verifying It

1. Why reading is not enough

The truss engine's error lay in two lines: the stiffness matrix lacked the 1/L factor, the force recovery had it. Each line was reasonable on its own; the problem was that they were written with different assumptions. In the Cross engine, carry-over used the correct factor (0.5) — just on the wrong member. Someone reading the code line by line could have approved every line.

Reading is good for understanding what the code intends. The only reliable way to know what it does is to run it and compare the result with something independent of the code.

2. Verifying the verifier

The most instructive moment of the audit was a mistake. The slope-deflection, three-moment and stiffness-matrix pages looked correct when read. Even so, each was run with 20 random inputs and compared with a separately written reference solver. Two pages matched; the three-moment page showed a 107% deviation.

Before drawing a conclusion, two hand-solvable cases were checked directly: two equal spans (known answer M_B = −qL²/8 = −45 kNm) and 6 m + 4 m (−35 kNm). The page was exactly right in both. The error was in the reference solver: a sign in its reaction formula had been reversed. Once fixed, the agreement was complete.

3. You cannot find what is missing by reading

The response spectrum page did not include the long-period range of TBDY at all (T > T_L = 6 s) and gave 33% too much acceleration there. Every existing line was correct. What was wrong was a line that had never been written. Finding an omission by reading requires knowing in advance what is missing.

What found this error was sweeping the full input range: some of the random inputs were deliberately chosen above 6 s and the result was compared with the site's TBDY engine.

4. Honesty of the screen

Two errors concerned the screen, not the calculation. The spectrum page asked the user for a damping ratio but used it nowhere: 2% or 20%, the result was the same. The code-pack selector, in some cases, showed the US code on its button while calculating to TBDY.

In both cases the calculation engine was correct. But the user was getting wrong information from the screen about what the calculation rested on. For an engineer this is as dangerous as a wrong number.

5. Rounding is a source of error too

The first version of the test for in-page calculations found "mismatches" on two pages. Both were errors in the test. The page showed a = 21.1 mm, the engine 21.05 mm — the difference was only the one-decimal rounding on screen. In a footing example q_min was almost zero (1.69 kPa), where a relative tolerance becomes meaningless.

6. Five rules

1. At least one hand-solvable case for every calculation. It is the referee when two implementations disagree.
2. An independent second implementation. A different method (slope-deflection against Cross) or a different author.
3. Random and boundary inputs. Including the points where formulas change regime and the ends of the range.
4. Check what the user sees. Not only the engine, but every value and every label on the screen.
5. Verify the verifier. The reference solver, the test and the tolerance can all be wrong.

These rules have been added to archi-civil.com's tests; each comes from a situation that actually happened in the audit.