Tokens are the next AI concept to enter the HR lexicon

Move over, Monopoly money, poker chips, Bitcoins and NFTs. HR, meet AI tokens.

First, LMGTFY: AI tokens are units vendors use to represent work being processed by large language models (LLMs).

“In the AI context, it is a unit of data,” Julia Dhar, managing director and partner at the Boston Consulting Group, said. “You can think about it like a brick in the wall; you could think about it like one dollar; you can, maybe, think about it like an atom that allows a generative AI model to both decompose and then recompose language when, for example, a human submits a query to a large language model.”

A token can represent a single word, a group of letters, a punctuation mark, but—most importantly—each token represents a bit of work and energy needed by the LLM. Understanding how tokens work, how much are available for use, and what a token translates to in output, have suddenly become matters for HR and people teams to consider.

“The way that we are talking about tokens at the moment in organizations is effectively as a resource that is being consumed by users of large language models, and that, of course, is because a main way that the providers of the large models charge companies to use them is based on how many tokens,” she said.

AI vendors have designed new pricing systems that work for an AI-powered enterprise software landscape. Unlike many cloud-based software product offerings which usually are priced per-seat software licenses, AI costs can fluctuate based on activity and many invoices for AI tokens have skyrocketed.

Right now it’s “an unbelievably complicated set of organizational challenges all coming together,” according to Dhar. “It’s budgeting; it’s access to talent and incentive design; it’s use of resources, and this question of what work gets done and what work is valuable.”

Some companies have even looked to token use as a proxy to understand employee AI adoption and operational productivity impact with the tools, but some have found the measure doesn’t deliver when it comes to understanding AI use. HR leaders are realizing token use doesn’t actually equate to output or indicate the value of the output.

“I’m not opposed to the idea that we should all be mindful that tokens are—for most of us—a constrained resource, like they don’t fall from the sky for free. I would love if it nudged us executives, CHROs, and CTOs together as quickly as possible to…examine the work that happens in this organization, understand where that is best done by an individual human, by a collaboration of humans (that’s the whole point of a team, by the way), a partnership between humans and technology, or done standalone by a machine.”

Dhar told HR Brew that organizations can’t replace R&D and strategic org design with an offering of AI tools. Many orgs are farming out this design phase to their AI-enabled employees, rather than engaging in strategic conversations about what work belongs to which talent group (humans, teams, collaborations between AI and humans, or AI tools independently).

“We can also see we have received many of the benefits of thousands of flowers blooming,” she said. “But you still need CEOs, chief technology officers, CHROs to ask what kind of garden are we trying to plant, and who are we hoping comes to visit this garden?