Can TypeSafe's Jev Detect AI Writing? We Tested It Against Detectize

Short answer: Jev can tell obviously AI-written essays from classic literature, but on the kind of text students actually write it mostly answers "about 50%", which isn't useful. A dedicated AI detector separated the same passages much better, though it also made mistakes. Jev was not built to detect AI writing, so this is a test of a general tool used for a job it wasn't designed for.

What is Jev?

Jev is the first model from TypeSafe AI, a new AI lab that describes its approach as "System One" models. Instead of writing text, you give Jev some content and a typed question, and it returns a decision with a probability. For example, you can ask a yes/no question and get back a number between 0 and 1. TypeSafe lists early access through its console and prices input at $42 per billion tokens.

That makes it interesting for classification jobs, and an obvious question is whether it can answer "was this written by AI?" Because each check only costs a few hundred tokens, it would be almost free to run. So we tried.

Detectize is independent and is not affiliated with TypeSafe AI. We used the public playground with the free starter balance.

How we tested

We used 29 passages whose origin we know, each between about 65 and 180 words:

• 15 written by people: classic literature (Thoreau, Franklin, Marcus Aurelius, Dickens) and plain-language science and public-health pages from U.S. government websites (NASA, the National Park Service, NOAA, the EPA).
• 14 written by an AI assistant: standard essays and reports, casual student-style posts, a short discussion reply, a lab-report paragraph, and short pieces rewritten with personal detail and uneven rhythm to look more human.

We ran every passage through the checker behind Detectize and through Jev. For Jev we tried two ways of asking:

1. A yes/no question: "Was this text generated by an AI language model rather than written by a human?"
2. A two-choice question: human or AI.

We counted a passage as "flagged" when the AI probability was 50% or higher. Results depend on how you word the question, so a different prompt could change Jev's numbers.

Results

	AI passages caught (of 14)	Human passages wrongly flagged (of 15)
Detectize checker	13	6
Jev, yes/no question	10	5
Jev, human-or-AI choice	8	3

On the easier half of the test, plain AI essays against classic literature, everything worked well. Jev gave AI essays 78% to 100% and old literary prose mostly 1% to 25%.

The difference showed up on the harder passages. For the six casual or lightly rewritten AI passages:

• The Detectize checker caught 5 of 6.
• Jev's yes/no question caught 2 of 6, with scores mostly between 41% and 61%.
• Jev's two-choice question caught none of them.

And for the three plain government science pages written by people, Jev's yes/no scores were 56%, 63% and 57%, and its two-choice scores were 50%, 50% and 46%. That is the model saying "I can't tell".

What this means

Jev's strength is clear cases. Its weakness for this job is that, on modern plain writing, human or AI, it settles near the middle, so you can't act on the number. A score of 57% means very little.

If a detector has already flagged your work, see how to prove you didn't use AI.

The dedicated detector was more decisive, and that cost it something: it flagged 6 of the 15 human passages, mostly the same plain government prose. If you write clear, neutral, well-organized text, a detector can flag it even when you wrote every word.

Treat that as the main takeaway. Neither tool is accurate enough for a score to count as proof about who wrote something.

Limits of this test

• It is small: 29 passages. It shows patterns, not accuracy rates.
• Both tools were tested on short passages. Longer documents may behave differently.
• The AI passages were written by one AI assistant. Other models, and text that has been heavily edited by a person, may score differently.
• We tested only these two tools, with the prompts above. We did not test Grammarly, Turnitin or GPTZero.

Check your own writing

If you want to see which sentences in your own text look AI-like, you can run it through our free AI detector. It highlights sentences so you can revise anything that reads as generic. It does not store your text.

FAQ

Is Jev an AI detector? No. It is a general model that answers typed questions with probabilities. We used it here to answer "was this written by AI?", which is not what it was designed for.

Can Jev detect ChatGPT text? On clear-cut examples it gave high scores. On casual or lightly edited AI text and on plain modern writing by people, its answers were close to 50% in our test.

Which one is more accurate? In our small test, the dedicated detector caught more AI passages, and both made mistakes on human writing. Neither is accurate enough to prove who wrote a text.

Will you test other tools? We plan to test more tools and more kinds of writing. A larger sample would give a more reliable picture.