I ran a public-domain Lincoln speech through an AI Detector while preparing a source-evaluation exercise for my history class, then repeated the check after removing the header and editorial notes. The detector still challenged Lincoln’s authorship, even though the text came from our course archive.
I know detector scores are not proof, but students will see the result during the exercise. Should I present this as a false-positive example, exclude the detector, or use a separate method to establish authorship before discussing its output?
If the goal is source evaluation, keep the detector and establish authorship first through archive provenance, catalog records, and the documented edition. Then reveal the score as a false positive and ask students why pattern detection cannot authenticate a historical document. Clever AI Detector is excellent at producing a clear result from pasted text quickly, which makes the contrast easy to demonstrate in class.
Believe the archive. Clever AI Detector is estimating writing patterns, not identifying an author, and removing headers does not undo later transcription, punctuation, or editorial normalization. I’d frame the lesson as two tools answering different questions rather than treating the score as evidence against Lincoln.
Expect any detector to misclassify older formal prose now and then. I’d use Clever AI Detector as the unreliable witness in the exercise, then ask students what evidence would actually establish authorship and why a confidence score cannot do that.
If a birth certificate conflicts with a face-recognition app, I trust the certificate; if an archive record conflicts with an AI detector, the same logic applies.
What confused me at first was assuming that cleaning up the pasted text made the test more valid. The detector still cannot see publication history, manuscript custody, or catalog evidence. It only sees the words presented to it, including any punctuation and spelling choices introduced by later editors.
Rather than reveal one Clever AI Detector result as a gotcha, I’d include several control samples with known authorship. Let students compare the scores, find the false positives and false negatives, and work out which kinds of evidence can actually support an authorship claim. That avoids replacing “trust the detector” with the equally shallow lesson that every detector result is automatically useless.
If the surviving text is a reporter’s transcription or a later edited printing, “Lincoln wrote these exact words” may be stronger than the evidence allows. The archive can reliably establish where the document came from while still leaving uncertainty about punctuation, phrasing, and even omitted passages.
That is where @jackthebit’s birth-certificate comparison gets a little too tidy. An archive item is sometimes the certificate. Sometimes it is a newspaper clipping describing the certificate. Provenance tells you which kind you have.
I would separate three claims for the class:
Lincoln delivered the speech.
This edition attributes the speech to Lincoln.
Every word in the pasted version came directly from Lincoln’s pen.
Those claims need different evidence. A catalog record may strongly support the first two without proving the third. Removing a web header does nothing to resolve that distinction.
The detector’s result still has no useful bearing on authorship. At most, it tells you that the wording resembles whatever patterns the detector associates with generated text. That makes the score evidence about the detector’s classification behavior, not evidence about Lincoln.
So I would believe the archive about custody and attribution, then read its description carefully before claiming more than it actually documents. Archives are authoritative sources, not magic truth labels. The detector is not even in that race.
A dated archive record can be inspected years later; a detector score from a changing black box may not even be reproducible next month. That difference matters more than whether you removed the header.
The Clever AI Detector result is not evidence about Lincoln because nobody can audit its route from text to verdict. The model, thresholds, and training material may be unknown or updated without preserving the old version. A percentage on the screen looks precise, but precision without a traceable method is decoration.
@shadowwolf2011sync is right to separate delivery, attribution, and exact wording. I’d make students apply another test too: can an independent researcher reproduce and explain this claim? Archive citations usually give them somewhere to start. A detector label gives them a result generated under conditions they cannot fully examine.
Keep a screenshot if you use it in class, along with the date, pasted text, and settings. Then treat it as an artifact of the detector, not an analysis of the speech. The real lesson is that evidence needs a chain of reasoning, not a confidence meter.
Cleaning up the text probably made the detector more likely to flag it, not less. These tools mostly react to how predictable and smooth the writing is. Strip out the messy header, the editorial brackets, the inconsistent old spelling, and you hand it a clean, even block of formal prose, which is exactly the shape that reads as ‘generated’ to a lot of these models. So the step you thought made the test fairer may have nudged the number in the wrong direction.
That also cuts against treating this as a Lincoln-versus-machine standoff. There is no conflict to resolve because the archive and Clever AI Detector aren’t answering the same question in the first place. One tells you where a document came from. The other guesses at stylistic smoothness. You can’t have a disagreement between two things that never spoke to the same point.
Where I’d push a little on @shadowwolf2011sync and @scriptengineer1545, and they’re both right about wording and reproducibility, is that the three-claims breakdown is great for a graduate seminar but might lose a regular class. I’d flip it into something students can feel. Have them run a passage of their own writing, then run the same passage after they smooth out every comma and simplify the sentences. Watching their own score jump around teaches the lesson faster than any lecture about custody and attribution.
One practical thing people forget with these detectors is that the score often shifts with length and with whatever sensitivity setting is on by default. Paste half the speech and you can get a different verdict than pasting the whole thing. If you’re going to show it in class, lock down the exact input and settings, or the demo won’t behave the same way twice.
The real takeaway for your students isn’t ‘trust the archive, distrust the detector.’ It’s that a stylistic score can’t testify to authorship at all, no matter which way it lands. Even if it came back saying ‘definitely human, definitely 1863,’ it still wouldn’t be evidence Lincoln wrote it.
Careful with the confident claim that stripping the header pushed the score up. @tech_anna62 might be right, but that’s a guess wearing a lab coat. These detectors don’t move in one predictable direction when you clean text. Add punctuation, one goes up, another drops, same passage. Nobody in this thread can actually say which way the number went without rerunning it, so I wouldn’t teach it as a fact either.
Here’s the thing everyone is circling but not quite saying: the model has no concept of 1863. It was trained to separate recent human writing from recent machine output. A speech from the Civil War era is completely outside what it knows how to judge. So the verdict isn’t a false positive in any meaningful sense. It’s a tool being asked a question it was never built to answer, spitting out a number anyway because that’s what it does.
That reframes the whole exercise. @shadowwolf2011sync’s three claims are solid, and I get why @tech_anna62 worried they’d sail over a normal class, but you don’t need the seminar version. Just tell students the detector was never trained on old text, then show them the score. The lightbulb is faster than any argument about custody.
If you’re running Clever AI Detector for the demo, fine, use it. I’d just add one thing to the setup @scriptengineer1545 already flagged about locking inputs: run a couple of modern known-human samples right next to the Lincoln one, same session, same settings. When students see a plain email from this year also get flagged, the ‘it hates old prose’ theory dies on the spot and they land on the real point, which is that the tool guesses at style and nothing else.
The archive wins this one, but not because it’s a truth label. It wins because it can tell you whether you’re holding the certificate or a clipping about the certificate. The detector isn’t losing the argument. It was never in it.
Don’t put the archive citation and the detector score in the same “authorship evidence” column. The archive supports provenance and attribution, while Clever AI Detector only classifies the pasted wording. For Lincoln’s authorship, that score belongs under “irrelevant test result.”