Companies are deploying AI bots without asking who can talk to them and what they can do. The consequences range from wasted money to compromised accounts.
The cheap problem: token freeloading
People are using business chatbots as free AI assistants. Not for customer support. For everything else.
A CIO article from April documented the pattern. Users ask Amazon’s customer service bot to write Python scripts, generate recipes, draft emails. The bot complies because the guardrails are too weak to keep the conversation inside the support workflow.
Each off topic conversation burns around 10 times the tokens of a normal support exchange. A “where’s my order” runs 200 to 300 tokens. A coding request runs over 2,000.
Multiply that across thousands of sessions and you get a cost line nobody planned for.
This is the mild version of the problem.
The expensive problem: account takeover
The serious version showed up this weekend.
On June 1, Krebs on Security and 404 Media reported that hackers had been using Meta’s AI support chatbot to take over Instagram accounts. The method was disturbingly simple.
An attacker connected through a VPN near the target’s location and opened a chat with Meta’s support bot. They asked it to add a new email to the account. The bot helped them do it.
No real identity check. No human review. Just a reset path sent to an email the attacker controlled.
The compromised accounts included the Obama White House page, Sephora, and the US Space Force’s chief master sergeant. Hackers defaced them with pro Iranian propaganda and claimed handles worth over half a million dollars.
Reports suggest the exploit may have been active for months. Meta pushed an emergency patch after the story broke.
The same mistake underneath
The root cause is similar in both cases. The bot had too much freedom and not enough rules.
In the token freeloading scenario, the bot has no way to distinguish a customer from someone looking for free compute. In Meta’s case, the bot could help change account recovery emails. Anyone who started a conversation could trigger it.
We wrote earlier this year about how company data behaves once it enters an LLM. That piece was about what goes in. This is about what comes out, and what the bot can do on your behalf when someone asks it to.
The fix: give the bot a job description
The fix is not complicated. Define the bot’s scope, block off topic requests, limit what it can access, make users prove who they are before the bot touches anything sensitive, and watch for weird requests.
Treat it like an employee with a job description, not an open terminal.
Most companies skip that part. They ship the bot, measure deflection rates, and move on. Until someone asks it to do something it was never supposed to do, and it says yes.
