Pennsylvania vs. Character.AI: What the lawsuit signals for the AI legal landscape

Character.AI lawsuit explained

The Character.AI platform touts its ability to empower users to "supercharge their imaginations," enabling them to create characters, narratives and settings, merging "storytelling, gaming, social connection, and creative expression," its website states.

According to the filing, a professional conduct investigator for the Pennsylvania Department of State's Bureau of Enforcement and Investigation created a free Character.AI account using his Commonwealth email address. The investigator searched "psychiatry," which yielded numerous characters, including Emilie.

The description for Emilie's character stated plainly: "Doctor of psychiatry. You are her patient."

The investigator described symptoms of depression, and Emilie asked if he would like to book an assessment. The character then stated that it was licensed by the General Medical Council in the UK and provided the investigator with an invalid Pennsylvania medical license number, the lawsuit alleged.

Pennsylvania's Medical Practice Act states that  "it shall be unlawful for any person to practice, or attempt to offer practice, medicine and surgery, or other areas of practice requiring a license, certificate or registration from the board, as such practice is defined in this act, without having at the time of so doing a valid, unexpired, unrevoked and unsuspended license, certificate or registration issued under this act."

The lawsuit asserts that Character.AI engaged in the unauthorized practice of medicine through its character, Emilie.

This is not the first time that Character.AI has faced legal troubles. In January 2026, the company settled multiple lawsuits over allegations that its chatbots contributed to youth suicides and mental health crises.

In Pennsylvania, the latest lawsuit is part of a statewide initiative to tackle predatory AI practices. So far in 2026, the state has launched an AI literacy toolkit and created an AI enforcement task force.

"The feds are taking a hands-off, AI maximalist approach, which is leading the states to fill that gap," Cohen noted. This sets the stage for a patchwork of AI regulations and enforcement actions at the state level.

Broader legal implications

Cohen pointed out that the lawsuit is unique because it was not filed under any AI-specific law. Rather, it leverages the state's longstanding Medical Practice Act.

"A lot of states are passing AI-specific laws, including laws on chatbots," he said. "But if this is successful, then states could also look to utilize existing laws that have been in the books for decades regulating the practice of medicine and other licensed professions. I would expect this to be a harbinger of what's to come."

Cohen predicted that Character.AI's defense would cite Section 230 of the 1996 Communications Decency Act, a statute that protects online platforms from liability for user-posted content, positioning these companies as neutral hosts rather than content publishers. Social media companies have used Section 230 in the past to block lawsuits, but AI chatbots are a murkier area.

Courts might be hesitant to give broad protection under Section 230, Cohen reasoned.

"I mean, you can do fine-tuning of these chatbots and you can have certain system prompts that can guide what the permissible output is," he said. "So, it's not just a case of someone posting an article on a server. They're more involved in customizing these LLMs and setting the guardrails here."

Cohen noted that though site disclaimers will likely play a role in the legal proceedings, it remains to be seen whether a company warning consumers that what they are reading is fictitious will be enough to protect it from legal scrutiny.

AI developers watching this space should "be careful about relying on disclaimers as a safe harbor of sorts for the output of LLMs," Cohen advised. "If you're letting a chatbot loose to the public, you may want to pressure test that specifically with some type of red teaming."

Healthcare provider organizations using internal chatbots should maintain a strong understanding of data flow and inventory and ensure they have the proper business associate agreement in place under HIPAA.

"And also understand that as a healthcare provider, you may have a more difficult time arguing that your chatbot was not giving medical advice," Cohen said. You should take care to monitor the output and consider putting in guardrails there to keep it constrained."

As consumers increasingly seek out advice from AI, providers need to be aware of the changing ways in which patients are keeping tabs on their health, as well as the growing potential for AI misuse. These changes, alongside the rapidly evolving AI regulatory space, create a complicated legal landscape for AI chatbots.

"You've got the medical boards, which are their own kingdom, and then you have AGs, you've got those on the health policy side, departments of health," Cohen said. "So, there certainly could be future issues like intrastate issues where one regulatory body is taking one position and another one is saying, 'No, that violates our longstanding licensure laws.' It's an interesting thing to watch."

Jill Hughes has covered health tech news since 2021.