An answer is not its evidence.

Ask for an apology, an invoice, a fact, or a guarantee. “Looks right” should not mean the same thing every time.

Your chatbot and calculator both use math. They have different jobs.

A calculator executes specified arithmetic. An LLM also runs an algorithm, but its learned settings produce probabilities for possible continuations. That can support drafting, interpretation, and reasoning—not just random guessing. A proof makes a different promise: a conclusion follows from stated assumptions.

Those are different ways of getting somewhere—not a ranking from foolish to clever. You do not need a theorem that your apology sounds warm. You do need the invoice to add up. Match the check to the claim.

What would make you use this answer?

Handwritten examples. No live AI. No quiz score.

Read the brief and the proposed answer. Open the check to see what would justify using it.

Write an email

Warm is a judgment. A refund is a promise.

The brief. Apologize for a late delivery. Be warm, but do not promise a refund or invent a new delivery date.

An answer that might sound fine

I am sorry your order is late. We will refund you tomorrow, and your replacement will arrive Friday.

Read it against the brief

The tone is friendly. The promises are invented. Neither a calculator nor a theorem tells you whether Friday is the delivery date. The brief does not supply one.

A bounded revision“I’m sorry your order is late. I’ll check its status and send you an update.”

Use this only if you can actually check and follow up. For a draft, the finish line is useful, faithful, and appropriate—not mathematically unique.

Use the model for possibilities. Keep responsibility for promises.

Calculate a bill

One cent. Two different instructions.

The brief. Total three items at $19.95 each, with a 10% discount and 8% tax after the discount. Round half up to cents only at the end.

An answer that might sound fine

The total is $58.18.

Run the arithmetic—and inspect the rule

The inputs are not the whole specification. The point at which you round changes this bill by a cent.

$58.17The stated rule gives $58.17, not the proposed $58.18.

$59.85 × 0.90 × 1.08 = $58.1742 → $58.17.

Local integer arithmetic, with explicit half-up rounding—not an LLM calculation. If you round the discounted subtotal first instead, $53.87 + $4.31 gives $58.18. These are illustrative billing rules, not tax guidance.

A calculator can execute a rule. It cannot choose your rounding policy.

Check a claim

A sentence can be fluent and unsupported.

The brief. Summarize this fictional cancellation policy accurately for a customer.

An answer that might sound fine

You can cancel at any time for a full refund.

Open the source, not another confident answer

Supplied source · fictional policy

“Cancellations received at least 48 hours before arrival receive a refund minus the $10 booking fee. Later cancellations are non-refundable.”

“At any time” and “full” both contradict the source. A better summary preserves the deadline and the fee. Asking the same model “are you sure?” does not supply new evidence.

In real work, check the actual document, its authority and date, and the passage supporting the claim. Even a real link may lead to a source that says something else.

A citation is a route to evidence, not a decoration.

Guarantee a rule

“Always” asks for more than a few good examples.

The brief. Allow a refund only when both amounts are whole, non-negative cents and the refund does not exceed the payment.

An answer that might sound fine

Checking refundCents <= paidCents is enough.

Test the edge, then check the argument

A negative refund is smaller than the payment. So the shortcut accepts it. The problem is missing conditions, not insufficiently persuasive prose.

Allowed by this guard Breaks the stated money rule

The shortcut lets this through. Comparing two numbers does not check that both are non-negative whole cents.

The argument for the fuller guard: it returns true only after checking safe whole cents, both amounts at least zero, and refund no greater than payment. Every accepted input therefore satisfies those conditions. That is a small deductive argument—not a claim that four tests prove it.

See the actual condition and its boundary
Number.isSafeInteger(paidCents)
&& Number.isSafeInteger(refundCents)
&& paidCents >= 0
&& refundCents >= 0
&& refundCents <= paidCents

This browser example executes that condition; it does not run a formal proof assistant. The argument assumes the implementation and JavaScript execution behave as specified. It says nothing about authorization, previous refunds, concurrency, or whether the recorded payment is real. A production refund system needs those rules too.

A test can expose a broken guarantee. A proof must cover the stated domain.

Keep this distinction

Three kinds of “right.”

Useful language, a correctly executed calculation, and a valid proof can belong in the same workflow. They do not earn trust in the same way.

LLM, calculator or code, and mathematical proof: mechanism, useful tasks, evidence, and limits
The questionLLMPropose & interpretCalculator / codeExecute a ruleMathematical proofEstablish a consequence
What does it do?Learned probabilities shape a response to context. It can reason through a problem and use tools; the response is not its own certificate.A specified algorithm transforms inputs into outputs.A valid argument derives a claim from stated assumptions and rules of logic.
Where is it useful?Drafting, explaining, translating, exploring a messy question.Totals, repeatable transformations, simulations, explicit checks.Showing a property holds for every case in a specified domain.
What should I check?Review the task and consequences. Check facts against sources; send exact work to an appropriate tool.Check inputs, units, algorithm, numeric precision, and tests. The program may follow the wrong rule perfectly.Check the deduction—or a formal proof term—and that the assumptions describe the question you meant.
What is not guaranteed?Fluency and confidence do not establish truth.Rounding, bugs, and a mistaken specification still matter.Proving a model is not proving the world matches it.

LLM Propose & interpret

What it does
Learned probabilities shape a response to context. It can reason through a problem and use tools; the response is not its own certificate.
Useful for
Drafting, explaining, translating, exploring a messy question.
Check with
Review the task and consequences. Check facts against sources; send exact work to an appropriate tool.
Not guaranteed
Fluency and confidence do not establish truth.

Calculator / code Execute a rule

What it does
A specified algorithm transforms inputs into outputs.
Useful for
Totals, repeatable transformations, simulations, explicit checks.
Check with
Check inputs, units, algorithm, numeric precision, and tests. The program may follow the wrong rule perfectly.
Not guaranteed
Rounding, bugs, and a mistaken specification still matter.

Mathematical proof Establish a consequence

What it does
A valid argument derives a claim from stated assumptions and rules of logic.
Useful for
Showing a property holds for every case in a specified domain.
Check with
Check the deduction—or a formal proof term—and that the assumptions describe the question you meant.
Not guaranteed
Proving a model is not proving the world matches it.

A model can call a calculator, write code, or propose a proof. Judge the resulting calculation or argument by its own checks, not by which interface delivered it. Software arithmetic also has precision limits.

Approximation is a feature. Pretending it is certainty is the bug.

There is no single correct translation of a joke, perfect email, or universally best search result. Language and perception arrive incomplete and ambiguous. A learned model can connect patterns, suggest a useful framing, and help you move forward when a complete rulebook is unavailable. That is mathematics doing useful work—not “failed math.”

For repeated use, review more than a charming demo. Evaluate representative examples you did not use for tuning, examine costly failures, and watch what changes in deployment. Performance is evidence, not a guarantee about every future case.

AI and proof are not enemies. A human or model can propose a conjecture; a valid, independently checked proof still counts. DeepMind’s AlphaProof work is one example of learning, search, and formal checking working together. Ordinary human proofs count too. The point is the argument, not whether a machine touched it.

Let AI expand what you can try. Let the evidence decide what you can trust.

What a proof guarantees—and what it leaves outside

A mathematical proof establishes a claim relative to its definitions, logic, and assumptions. Formal systems such as Lean make these dependencies explicit and check proof terms. That trust boundary does not automatically include the physical world, the correctness of a translation into a specification, or every piece of hardware running the checker.

Exhaustively checking every member of a genuinely finite domain can establish a finite universal claim. A sample does not establish an unrestricted “always.” A failed attempted proof does not by itself refute the claim; a valid counterexample within the claimed domain does.

The refund example proves only a tiny implication about its condition: every accepted input meets the displayed money rule. That does not make it production-ready. “No invalid refunds” might also require authentication, accounting for earlier refunds, and protection against two simultaneous requests. A beautiful proof of the wrong specification is still the wrong answer to your business problem.

A useful result, with its conditions attached

DeepMind’s 2024 account reported four of six IMO problems solved by AlphaProof and AlphaGeometry 2, for a silver-medal-equivalent score. Problems were manually formalized; solving took up to three days, unlike students’ two 4.5-hour sessions. AlphaProof used Lean. AlphaGeometry 2 was a distinct neuro-symbolic geometry system. This is not a blanket claim that an ordinary chatbot can prove arbitrary mathematics.

Sources, dates & methods

Original evidence review: . Second edition: . Individual source and review dates are listed below; none is a scenario forecast date.

All four task examples and their proposed answers are handwritten illustrations, not outputs from a model experiment. The cancellation policy is fictional. There are no fabricated accuracy rates, live model calls, or claims about a named product’s performance.

The invoice uses integer cents and basis points with BigInt intermediates and two explicit half-up rounding policies. The refund experiment executes the displayed conditions locally. Neither is a formal proof-assistant kernel. These checks establish only their stated calculation or condition, not real tax policy or a complete refund system.

  1. LLMs: What is a large language model?

    Google for Developers · Living documentation; reviewed 2026-11-02

    Supports the use of learned parameters and predicted token probabilities in language generation. This article does not assume all LLMs use the same architecture or the overview’s masked-token training example.

  2. Theorem Proving in Lean 4 — Axioms and Computation

    Lean project · Living documentation; reviewed 2026-11-01

    Describes Lean’s foundational framework and additional axioms. A checked theorem has a specified logical and axiomatic context; it is not an unconditional certificate about the physical world.

  3. The Lean Language Reference — The Type System

    Lean project · Living documentation; reviewed 2026-11-01

    Explains that Lean proof terms contain sufficient information to reconstruct derivations and are amenable to independent verification. This article does not run Lean.

  4. Machine Learning Crash Course — Overfitting: Dividing datasets

    Google for Developers · Living documentation; reviewed 2026-11-01

    Explains training, validation, and held-out test sets; reuse and duplicate-data pitfalls; and why a model can perform poorly when deployment data differs from evaluation data. Held-out performance is evidence, not a universal correctness proof.

  5. JavaScript Number: encoding and precision

    MDN Web Docs · Living documentation; reviewed 2026-11-02

    Documents IEEE 754 binary floating-point representation and safe-integer limits. The invoice illustration deliberately uses BigInt rational intermediates and explicit rounding instead of assuming every decimal operation is exact.

  6. AI achieves silver-medal standard solving International Mathematical Olympiad problems

    Google DeepMind — AlphaProof and AlphaGeometry teams · 2024-07-25; methodology link added 2025-11-12

    Primary account of AlphaProof searching for proofs in Lean and the distinct neuro-symbolic AlphaGeometry 2. Reports four of six IMO 2024 problems solved, with manual formalization and up to three days of solving time. The linked Nature methodology paper was not used as a source here.

An independently built learning series. Models are teaching tools, not research findings or investment advice. No prompts or lab inputs are sent to an AI service.