Taming the monster
How, then, do organizations begin to protect themselves from GenAI risk? The answer comes back to data governance. To use GenAI safely and effectively, organizations must be much more vigilant about the data used by underlying LLMs – what it is, who it references, and what biases and inaccuracies it may contain. They also need to be more skeptical about the quality of the end product.
Most organizations lack the resources to build their own LLMs from scratch and, therefore, use a foundational model such as GPT. It’s not always easy to interrogate these models for bias and accuracy, although developers are under pressure to be more open about the processes they use.
However, while it may be difficult to evaluate opaque models, organizations can be more risk-conscious with the data they use to train the LLMs. Is the data relevant to the objectives the organization is pursuing? Is it dependable, diverse, and balanced? Does it pose IP infringement questions or expose the organization to data privacy risks?
Assessing these factors will be an ever-more important task for organizations seeking to use GenAI. Ultimately, organizations will need to decide which risks they are comfortable with – potentially a task for the governance committee – and exclude data that could take them outside these boundaries. The threshold may vary according to the use case; there is little point in exposing the organization to additional danger if a GenAI use case is of limited value.
But also, different uses of GenAI come with different concerns. Generating creative content, for example, might carry elevated IP infringement dangers while, for a customer chatbot, bias might be more of a concern.
Another piece of the puzzle for organizations to solve is the question of how to assess GenAI outputs. Even where substantial work has gone into vetting the data used by a particular tool, it will still be important to question the responses that it generates – for example, to identify hallucinations.
Training staff to spot and test for hallucinations will be important. If the workforce is going to make use of GenAI tools, people need to know how to verify outputs and how to report suspect results so the model can be finetuned. Guardrails should also be in place to ensure GenAI is only used for intended tasks.
The potential of GenAI is too great to ignore. But pursuing that potential without properly understanding the inherent risks – many of which are only just beginning to emerge – is dangerously short-sighted. An approach that balances responsibility with open-mindedness and ambition will give organizations the best chance of walking that fine line.
All views expressed herein are those of the author and have been specifically developed and published in accordance with the principles of academic freedom. As such, such views are not necessarily held or endorsed by TONOMUS or its affiliates.