AI Detection Tools Grow More Accurate, Prompting New Questions for Austin

The rapid rise of generative artificial intelligence has sparked an equally fast-growing market for software designed to determine whether text was written by a person or a machine.

Since consumer-facing AI tools such as ChatGPT surged in popularity in 2023, schools, businesses and publishers have increasingly turned to AI detection software to identify undisclosed machine-generated writing.

Companies behind the technology say their products have improved significantly in recent years, although experts continue to caution that no detector is perfect.

Rachel Li, chief of staff for GPTZero, said the company's detection model analyzes hundreds of characteristics within a document and has been trained using millions of examples of both human and AI-generated writing.

Among the indicators are highly structured sentence patterns and limited variation in writing style, features that researchers say appear more frequently in machine-generated content.

Copyleaks CEO Alon Yamin told KXAN that his company's software also evaluates hundreds of signals, including sentence length, punctuation usage, and recurring phrases.

"AI writes in a very systematic way," Yamin said.

Early concerns about AI detectors centered on false positives, where human writing is incorrectly labeled as AI-generated, and false negatives, where machine-generated text goes undetected.

Several university systems discouraged faculty from relying heavily on detection software when the technology first emerged, citing concerns about accuracy and fairness.

Even OpenAI, the creator of ChatGPT, has acknowledged the limitations of AI detection. In a statement published in May, the company said it was unable to develop a detector that was "reliable enough" for decisions carrying significant consequences.

OpenAI also warned that ChatGPT itself cannot accurately determine whether a document was written by AI and may generate answers that have "no basis in fact."

Despite those concerns, AI detection companies report significant improvements.

Li and Yamin both said their products now achieve accuracy rates exceeding 98%.

Independent testing conducted by AI consulting firm Graphite in May found false positive rates below 2% for GPTZero, Copyleaks, and Pangram after analyzing more than 15,000 human-written articles. The testing also showed low false negative rates when the systems evaluated thousands of AI-generated documents.

The growth of AI detection tools has also fueled the rise of so-called "humanizers," programs designed to rewrite AI-generated text to appear more natural and evade detection.

Li said modern detectors are increasingly capable of identifying collaborative writing in which humans edit or refine AI-generated drafts.

Meanwhile, educators continue to emphasize that technology alone should not determine authorship.

Rachel Kane, a professor at Arizona State University's Walter Cronkite School of Journalism and Mass Communication, wrote recently that repeated exposure to AI-generated writing has made many examples easier to recognize because of their predictable structure and tone.

Volunteer editors at Wikipedia have reached a similar conclusion, publishing guidance that advises readers not to rely solely on AI detectors or personal instinct but instead look for abrupt shifts in writing style and consistency within a piece.

As generative AI systems continue to evolve, experts say the contest between AI creation and AI detection is likely to remain an ongoing technological "arms race."