Using ChatGPT to Review a Contract: What It Misses

Pasting a contract into a general chatbot and asking "is this fair?" is free, takes thirty seconds, and gives you something. The question is what it leaves out — and whether that matters for the decision you are about to make.

This is a fair comparison, not a sales pitch, so here is the honest version of both sides.

What a general chatbot does well

Explaining a clause you do not understand. Paste one paragraph, ask what it means, get a clear answer. For this, a chatbot is excellent and you do not need anything else.

Answering a specific question you already know to ask. "What does 'joint and several liability' mean here?" is exactly the shape of question they handle well.

Being free. That is not nothing.

If your need is "help me understand this sentence", stop reading — you already have the right tool.

Where it breaks down on a whole contract

1. It answers the question you asked, not the one you should have asked.

This is the fundamental problem. A contract review is not a question-answering task; it is a search task. The clause that costs you money is the one you did not know existed, so you never asked about it. A chatbot will happily discuss the non-compete you pasted while saying nothing about the clawback in clause 4.6, because you did not mention it.

2. It has no checklist, so coverage is inconsistent.

Ask the same chatbot about the same contract twice and you get different lists. Nothing forces it to check for a liability cap, a cure period, an assignment restriction, or a set-off between garden leave and the non-compete. It reports what struck it as notable in that particular generation.

Run a structured review and the absence of a clause is itself a finding. That is a different mechanism, not a better model.

3. It rarely quotes, so you cannot verify.

Chatbot output tends toward paraphrase: "the termination clause is somewhat one-sided." Which clause? What does it say? Without the verbatim text and a location, checking the claim means re-reading the contract yourself — which is the work you were trying to avoid.

4. It does not know what is normal.

"This non-compete is 24 months" is a fact you could read yourself. "24 months is roughly double the market for a non-executive role, and unenforceable outright in California" is the sentence that changes your decision. Market comparison requires a deliberate instruction to compare, and a general chat turn does not include one.

5. It gives advice, not language.

"You should consider negotiating the indemnity" is not actionable. The replacement sentence you can paste into a redline is. There is a large gap between knowing a clause is bad and knowing what to put in its place.

6. Long documents degrade quietly.

Attention thins over a forty-page agreement, and nothing warns you when a schedule was skimmed. There is no error, no gap in the output — just a clause that never gets mentioned.

7. Your document may become training data.

This is the one people miss. Several consumer AI products reserve the right to train on what you paste. If your NDA prohibits disclosure to third parties — most do — then pasting it into such a tool may itself be a breach, before you even read the answer. Check the tool's terms, and check your own contract's.

What that looks like in practice

We ran the same deliberately flawed employment agreement through a small general model and through a structured review with a full checklist. Same document, same instructions to look for problems.

	General chat turn	Structured review
Issues found	5	15
Dated obligations extracted	0	8
Missing protections named	6	19
Clause quoted for each finding	rarely	every one
Replacement wording supplied	no	for each issue

The structured run caught three things the general one missed entirely: a $12,000 signing-bonus clawback that triggers on any departure, a 90-day option exercise window, and the fact that under the contract's own California governing law the worldwide non-compete is void.

That last one is the point. The most valuable sentence in the whole review was not "this non-compete is aggressive" — it was "this is unenforceable where you work, so it should not affect your decision." A general chat turn did not get there because nothing prompted it to cross-reference the governing-law clause against the restriction.

The complete report is published, including the contract, so you can judge that comparison yourself rather than take our word for it.

When to use which

Use a chatbot when you want one clause explained, you are curious rather than committing, or the stakes are low enough that a partial answer is fine.

Use a structured review when you are about to sign, there is real money or real restriction involved, and the cost of missing something exceeds the cost of the review — an employment contract, an NDA or a freelance agreement are the usual cases.

Use an attorney when there is meaningful equity, a live dispute, an executive contract, or a downside you could not absorb. A structured review makes that hour cheaper — you hand them a marked-up document instead of a stack of paper — but it does not replace their judgment or their accountability.

FAQ

Can ChatGPT review a contract?

It can explain clauses and answer questions about them well. What it does not reliably do on a whole document is systematic coverage, quoting the text it relies on, comparing terms against market practice, or telling you what is missing. Those are different tasks, not a better or worse version of the same one.

Is it safe to paste a contract into an AI chatbot?

Check two things: whether the tool trains on your input, and whether your contract restricts disclosure to third parties. Many consumer AI products do train on input by default, and most NDAs restrict onward disclosure, so the combination can put you in breach.

Why would I pay when a chatbot is free?

You would not, for understanding a single clause. The case for paying is a document you are about to sign where a missed clause costs more than the review — a clawback, an uncapped indemnity, an auto-renewal, a restriction on your next job.

Is a paid AI review as good as a lawyer?

No, and any tool claiming otherwise is worth distrusting. A structured review is more thorough than a tired human on page forty and far cheaper, but it cannot exercise judgment about your situation or take responsibility for the outcome. It is best used to make legal advice cheaper, not to avoid it.

---