Can AI research and write a piece of legislation? Sure, though not well. Should AI be used to write a piece of legislation on a subject impacting millions of beleaguered renters in the most-populated city in the U.S.? No, definitely not.

That didn’t stop one New York legislator from trying it out, then putting a limp, ineffective piece of legislation on the docket. As first noted by City&State earlier this week, Assemblyman Clyde Vanel-sponsored Bill 6896 would require landlords to provide renters a copy of their lease agreement upon request. It’s a small, though worthy case for New York City renters who are so often screwed by the byzantine operations of both corporate and individual lessors.

The bill, which was referred to the assembly housing committee in May, also contains a small, though notable disclosure. The paragraph reads:

“The Office of Assemblyman Clyde Vanel is actively exploring new ways to enhance government efficiency and draft better laws. This bill and its memo were researched and written by artificial intelligence, with the accuracy and language reviewed and refined by humans. This disclosure will appear on every bill where artificial intelligence is used to conduct research and drafting without significant human intervention.”

The document itself is just a few paragraphs long. There’s nothing that necessarily stands out about the document save for the dry language and the capitulation to landlords that would limit the number of requests to just two per year.

The entire bill was an experiment conducted by Vanel’s office. According to multiple reports, Vanel’s office used Auto-GPT, a kind of AI “agent.” These systems are effectively several versions of ChatGPT working in concert to complete an overarching task. Vanel chairs the Assembly subcommittee on internet and new technologies, and his legislative director Tyler Fritzhand told Fast Company they gave Auto-GPT the task of researching New York law, finding a gap in the law, then writing both the bill and the memorandum justifying the law. All the office had to do was type in the command, set out the parameters, and let it go.

It’s unclear what API the office used, though Fritzhand said they only paid a few bucks as the system ran for a few hours. It ultimately produced several bills, one of them trying to somehow fix loopholes in New York’s gun laws. According to the legislative director, that document was too wonky. Gizmodo reached out to Vanel’s office for comment, though we did not immediately hear back.

The thing is, no rental advocacy groups have been really clamoring for anything that the bill would accomplish. Advocacy groups and rental lawyers told The New York Times that access to leases was an issue that few advocates really cared about. It would only apply to landlords who had a copy of the document in the first place, and—notably—the bill does not mention anything about enforcement, so any compliance would depend on the generosity of the landlord themselves. Vanel’s office reportedly knew that this wasn’t a top-of-the-list issue for any tenants’ rights organization, but thought the AI had caught onto something other people hadn’t considered.

As can’t be stated often enough, modern language models aren’t effective at determining the benefits for humanity. AI can analyze trends in data, but LLMs are not interpretive, they are derivative. It’s why two New York City lawyers were fined thousands of dollars for trying to use ChatGPT to write legal documents. It’s also why AI-based “lawyers” like DoNotPay’s model was quashed by the courts, as modern chatbot models are only effective at generating text that reads human-like.

There is a mountain of other issues with rental properties in New York leading to the current housing crisis. Rent prices have skyrocketed, even with thousands of apartment spaces left vacant. Often, renters have never seen or heard directly from their landlords as the lessors hide behind LLCs and separate agencies. Renters have also had to deal with the prospect of landlords trying to install facial recognition cameras in buildings.

Real rent control legislation, like the proposed “good cause” eviction bill that in part would halt pseudo-evictions by jacking up rent prices, was rejected in this year’s budget amid a complete failure to pass any housing legislation. The lack of any kind of housing package while in the midst of a housing crisis gets no help from AI.