Author builds free database to combat AI-generated content with curated non-fiction

A 41-year-old author based in Santa Cruz has developed a free online platform designed to help readers discover high-quality non-fiction books, utilising major English-language prize winners and finalists as a primary quality metric. The tool, which allows users to perform semantic searches across approximately 6,500 titles, was created to counter the decline of physical library browsing and the rising prevalence of AI-generated content.

The project was developed during the summer with assistance from AI coding tools, specifically Claude Code and GPT-5.6, to gather and structure data from various online sources. The creator has personally covered all hosting and API costs to ensure the platform remains free for users, viewing it as a resource to help readers find celebrated works that might otherwise be overlooked by standard algorithmic recommendations on commercial platforms.

The database employs semantic search capabilities, which the author describes as the most appealing current application of AI for improving research workflows. This feature allows users to move beyond simple keyword matching to explore concepts such as "classic biographies that are surprisingly weird," pulling from the corpus to surface titles that may not appear in traditional search results. The goal is to replicate the serendipitous experience of browsing open stacks in a physical library.

Visualisations included in the database offer additional context on the non-fiction landscape. These features include a colour-based arrangement of roughly 5,000 books and a ranking of publishers based on their non-fiction award success over the past century. The author also noted historical trends in the field, observing that the number of non-fiction prizes peaked in 2014 before a potential slow decline began in 2020.

The author suspects the "golden age" of non-fiction writing occurred between the 1980s and the early 2000s, attributing this period to factors such as the jet plane, the erosion of social restrictions, early digital technologies like MARC standards, and the rise of broadcast media. The platform serves as both a discovery tool and a reflection on how technological and social changes have shaped the production and accessibility of non-fiction literature.

In addition to the book database, the author highlighted the effectiveness of Pangram, a tool recently integrated into Substack to rate whether content is human-written or AI-generated. Testing revealed the tool to be "dismayingly effective" in identifying AI-generated text, a development the author welcomed as a means to distinguish human-authored work in an increasingly automated digital landscape.

The initiative addresses a perceived gap in how readers engage with literature today. While undergraduate students often rely on search engines for sources, the author argues that the filtered, organised sampling provided by academic research libraries offers a superior intellectual experience. The new platform aims to preserve the value of that curated discovery process in a digital format.