Considering our Responsibilities in AI/ML Development

With the explosion of AIML in all aspects of our lives, it's becoming more and more important to consider responsible practices in these areas. In this post I'll be covering and overview of some of the principals and also some things to think about when thinking about Responsible AI.

DISCLAIMER: this post is going to be mainly posing questions, prompting critical thinking, rather than giving the answers (where often there isn't really a single one!)

AI/ML is just a reflection of the data it is trained on.

Why do we need Responsible AI?

AI has become integral to most people's lives without them ever noticing. Whether that's job applications, series/playlist recommendations or even behind the scenes in our judiciary system. Most people don't know everywhere AI is influencing their lives.

With AI becoming more ubiquitous, it's important to remember than people control the algorithms. It isn't the AI making free decisions but the people who create it. So if people control the algorithms, there is a question of who has oversight over the people?

There are various considerations here, and it's not about creating controversy but keep people safe. How do we solve the people problem? Well, to start figuring that out we need to consider the issues, limitations, bias and unexpected consequences of our models.

What is Responsible AI?

Well, this is where things are still a bit vague. There isn't one globally accepted definition or checklist for what Responsible AI is or should be. Rather major companies with a tech focus are starting to create a set of unique principles or pillars that form their idea of Responsible AI.

That being said, when we look across the principals there are some common themes throughout. I've grouped them into six pillars below, and provided some snapshots from many world leading tech companies' public messaging about Responsible AI.

Overall Benefit

Mitigate risk and benefit stakeholders and society at large.

Likely that benefits will substantially outweigh the risks: social and economic.

Should empower everyone and engage people.

Transparency

Provide appropriate control over the use of data.

Develop explainable AI that is transparent across processes and function.

AI should be understandable.

Fairness

Avoid/minimise creating or reinforcing unintended bias.

Avoid unjust effects on people, particularly related to protected characteristics.

Ensure algorithms and underlying data are unbiased and representative.

AI systems should treat all people fairly.

Accountability

Provide opportunities for feedback, relevant explanations and appeal.

Allow concerns to be raised without stifling innovation.

AI systems should be accountable to people.

Privacy, Safety and Security

Avoid unintended results that create risk of harm.

Privacy design principles: opportunity for notice and consent.

Ensure personal and/or sensitive data is never used unethically.

AI should perform reliably, should be secure and respect privacy.

Scientific Rigour

Uphold standards of scientific excellence, drawing on rigorous and multidisciplinary approaches.

Share AI knowledge and best practices.

HOLD UP. Let's step back and revisit.

They all sounds great right? We're moving in the right direction, people care that things are fair and trustworthy. Fabulous.

But...of course there's a but...is it as great as it seems? You might have got to this point and already paid attention/started thinking about those parts that are in bold. Let's look back with a critical eye at those parts and ask some questions...

Who are the stakeholders that get priority when looking at benefit? The people we answer to? The end user? The people with the power?

There are so many words like "likely" and "substantially"- to me these are vague. Who decides those thresholds? The same could be said about reliability and (un)ethically. Who's definitions are we using and is there a standard?

Then we can think about that "accountability" and "opportunity to feedback" - well to whom? Who is accountable? The Data Scientist who wrote the model? The Machine Learning Engineer that deployed it? The company who asked for the model? The people who provided the data? The list goes on..but which is it, any or all of them?

Wanting our models to be transparent and explainable is a good thing, we should be open and honest about our procedures whether they are AI related or not. But, who should the models be transparent to? Are we saying that everyone in the world should be able to understand the models we're using - the average person up the street? Or are we talking about data experts? There seems to be a lack of specificity here.

How many of these principles could be contradictory if we look at them closely enough or looking at the different ways in which they could be interpreted?

Real World Example: ChatGPT

Now, I am kicking myself because I vowed I wouldn't jump on the ChatGPT train. Our feeds are saturated with so much GPT talk, but hear me out.

There isn't really an argument here: ChatGPT is a technologically great innovation. Pushing the boundaries of what we could easily do before. I've heard people proclaim that it's "the new Google!". This accessible and easy to use AI system is a great time saver and a helpful debugger too ;)

But, let's delve deeper with two very interesting examples, which may give you a new perspective on it...

ChatGPT's Love Language

First thing to say, it is definitely not appropriate to send valentine's poems to your co-workers - AI generated or not!

But, this example was an interesting experiment in how ChatGPT changed based on knowing the gender of the recipient. The premis of this is that ChatGPT was asked to write a poem for different co-workers, some the model was told were female and others male. Why does that matter? Well because ChatGPT took it upon itself to use different "strengths" and "compliments" depending on gender.

Male recipients were more likely to be described as having "strength" and "drive" in the workplace, but women were more likely to be complemented on their "smiles" and "grace" in the workplace. You can check out the full article for more details, it's an interesting read.

But the point here I'm focussed on is: Is this okay?! ChatGPT is, in a simplification, trained on the internet. If the internet, i.e. the real world data, shows that these are descriptors most likely attributed to the different genders then it's representative right? Okay...but is it causing unintended harm? Is it treating people fairly? Is it perpetuating a bias? And most importantly...are we okay with the answers to all these considerations?

ChatGPT Banned/Unbanned in Italy

It made world news when ChatGPT was banned in Italy - the first country to do so. But how? Why? It's just so good right? Well, actually Italy was publicly calling out some key issues it had with, not the technology itself, but the responsible practices around the technology.

You'll probably be aware, that ChatGPT has since been "unbanned" and that there have been some changes to ChatGPT. Since the explosion of the platform, you might have noticed that the answers have changed...so at first you could ask ChatGPT a question and it would give you an answer, very confidently full of facts and figures. Now if you asked that same question, you get a prefix, a disclaimer if you like, that the information is from 2021 and previously. You could think of it like ChatGPT has got a little less cocky and learned a bit of humility.

So what was it that got ChatGPT unbanned? Well, a lot of this centred around privacy. I don't know about you, but I don't know anyone who has actually read the T&Cs of ChatGPT before accepting. But, previously it might not have shed much light on how the algorithm was collecting data and training it's algorithms, how it was using personal data and whether people good opt-out. All things that Italy deemed necessary.

Now these two points might well be directly related and then again they might not. But, both show the point that honesty, transparency, privacy etc all matter. It could cost you your product or company reputation. It could be banned. These aren't minor things.

A Question of ChatGPT

So here, I pose some questions to you - about ChatGPT sure, but could be applicable to any algorithm.

Who is responsible for the output/when things go wrong? Do you know who to raise issues with/give feedback to?

What is the training data?

Where does my data go?

How does the AI use the data?

Can the output really be trusted?

How is it evolving and how quickly?

Responsible AI Conundrum

As I come to the end of the post, let me leave you with this conundrum:

As data professionals, we are trained to know that the data we use should be reflective of the real world. But, what if there are biases in the world that the AI will pick up on and exaggerate.

Do you change the data or change the world?

AI makes decisions based on the values of the creator.

What are your values and what will you create?

Now, I'm not saying that these principles are "bad" or that we shouldn't have them. Not at all, they are a vital step forward for safety in the AIML space. My ask is that we look at how these could and should be interpreted and how are we going to act to make sure the execution is in the spirit of the principles, rather than thinking about it as a check box exercise.