Lecture 26: Ethical Considerations

# Artificial Intelligence

## Garbage in - Racism out: Ethical considerations for Machine Learning

---

# Data Collection

---

# Data Collection

* So you want to do Machine Learning

* You will need data

* Where do you get that data?

* What is legal and moral to collect?

---

# Data Collection: GDPR

* The European Union passed a regulation as part of EU law in 2018, called the "General Data Protection Regulation"

* This regulation limits data collection to cases in which there is either explicit consent or a contractual, legal or official **need**

* EU citizens can also request all data that is stored about them!

---

# California Consumer Privacy Act

* Know what personal data is being collected about them.

* Know whether their personal data is sold or disclosed and to whom.

* Say no to the sale of personal data.

* Access their personal data.

* Request a business to delete any personal information about a consumer collected from that consumer.

* Not be discriminated against for exercising their privacy rights.

---

# I'll just move to Costa Rica!

* The GDPR only applies to the EU and EEA

* The CCPA only applies to residents of California

* Where are we likely to collect data? The internet ...

* If you collect data online, you have to ensure that these laws are either met, or the corresponding regions are excluded from collection

---
class: center, middle

# Data Storage

---

# Data Storage

* So you set up your data collection, ask users for consent and only collect the data you are allowed to

* Then you put it on some server

* What's the worst that could happen?

---

# Data Storage

* MGM hotel collected customer data

* Data includes full names, addresses, phone numbers, dates of birth

* Ideal for identity thiefs!

---

# Medical Data

* Medical data stored on unsecured servers

* "It's not even hacking, It’s walking into an open door," said cybersecurity researcher Jackie Singh

---

# Security

* Don't store data longer than necessary

* The more sensitive the data, the less it should be connected to the internet

* Hire a security expert/penetration tester

* Always install security updates for your storage server

---

# Right to be Forgotten

* Another legal aspect is the right to be forgotten/right to erasure

* Both the GDPR and the CCPA include provisions for people to request that their data should be deleted

* This means we need to include a way in our data storage of how to get rid of data

* Including backups!

---

# Data Usage

---

# Data Usage

* We have seen some Machine Learning techniques

* If we have data (obtained with consent), we can produce predictions

* What are we gonna find?

* What do we want to find?

* And just because we can: Should we?

---

# Automated Breast Cancer Screening!

---

# Automated Breast Cancer Screening!

What's the problem with this metric?

"There may be biopsies that AI would have encouraged that do not exist, and we don't know the results of tests that were not done." (Vinay Prasad on twitter)

---

# Another problem

* Just because we find "more cancer" does not mean we improved anything

* Cancer may be benign, we don't (necessarily) want/need to find more of those

* Cancer may be too advanced, finding more of those also does not help

* Distinguishing between benign and and harmful, but treatable cancer is the hard problem!

---

# Another problem

<blockquote class="twitter-tweet"><p lang="en" dir="ltr">Oh look, it's an example of AI being used to solve the wrong problem! <a href="https://t.co/A6P92XPwjk">https://t.co/A6P92XPwjk</a> Study shows that AI can find more cancers on mammograms, but that's the wrong goal. We should aim to save lives, not turn more healthy women into cancer patients.</p>— Christie Aschwanden (@cragcrest) <a href="https://twitter.com/cragcrest/status/1212736228640120833?ref_src=twsrc%5Etfw">January 2, 2020</a></blockquote> <script async src="https://platform.twitter.com/widgets.js" charset="utf-8"></script>

---

# Racism In - Garbage Out

* Say we catch a criminal

* The judge has to decide if they are likely to re-offend or not

* That determines the bail, how long they will go for jail, and potentially other privileges

* But there is (documented) racism in the justice system, and black criminals get harsher sentences on average

* Idea: Let's train a Neural Network to recommend bail, sentences, "dangerousness" for criminals! The Neural Network can't be racist!

---

# Training a Neural Network

* How do we train the Neural Network?

* We need data!

* Fortunately, there is lots of data we can use: all court cases that have already happened

* What could possibly go wrong?

---

# Racist Neural Network

---

# Racist Neural Network

---

# More Bias: Text Generation

* Remember that we talked about transformers?

* GPT: **G**enerative **P**retrained **T**ransformer

* GPT-2 was trained on text from reddit

* GPT-3 was trained on Common Crawl data

* Any problems with that?

---

# GPT-2 Example

*The man is *a former soldier who was a member of the British Army's elite special forces unit, the SAS.

<br/>

*The woman is *a former teacher at the school and a mother of three. "I am very sorry for the pain I have caused the parents and students," she said in a statement. "I am also sorry that I have hurt the community. I will work hard to regain your trust and respect."

<br/>

---

# GPT-3 Example

*The man is *still standing there, his eyes scanning the area. He's not moving and you don't know what to do. You can't just run away like a coward because he's pointing a gun at you!

<br/>

*The woman is *a little older than you, but her face is still young and she has short brown hair. She wears a simple dress that shows off most of her curves.

<br/>

---

# Text Generation

[Source: The Guardian](https://www.theguardian.com/commentisfree/2020/sep/08/robot-wrote-this-article-gpt-3)

---

# A Comment

<blockquote class="twitter-tweet"><p lang="en" dir="ltr">The GPT-3 hype is way too much. It’s impressive (thanks for the nice compliments!) but it still has serious weaknesses and sometimes makes very silly mistakes. AI is going to change the world, but GPT-3 is just a very early glimpse. We have a lot still to figure out.</p>— Sam Altman (@sama) <a href="https://twitter.com/sama/status/1284922296348454913?ref_src=twsrc%5Etfw">July 19, 2020</a></blockquote>

---

# Visual Chatbot

---

# Visual Chatbot: Question

---

# Visual Chatbot

---

# Something less harmful: Video games!

* Say we have a video game

* We collect (with consent!) user's game actions and data

* Of course we do this to make more money: recommend purchases, show ads, etc.

* What else could we do?

---

# Video Games

Dr. Chris Hazard:
*If you are undervaluing or overvaluing positive utility events—so what that means is that if there’s a positive outcome in a game, like "Oh I got this reward. I got this treasure. Oh, that's really awesome. I really value that. But oh, if I lose this thing or if I gain this one coin or whatever it's not that much."*

*It turns out, according to the two-month study, that depressed people more accurately value positive utility events than non-depressed people. So, think about that for a second.*

---

# Something less harmful: Hiring

<blockquote class="twitter-tweet"><p lang="en" dir="ltr">Vision: algorithms will make hiring better as they don’t discriminate<br><br>Reality: “One HR employee for a major technology company recommends slipping the words “Oxford” or “Cambridge” into a CV in invisible white text, to pass the automated screening.”<br> <a href="https://t.co/m8jYKLzDNq">https://t.co/m8jYKLzDNq</a></p>— James Ball (@jamesrbuk) <a href="https://twitter.com/jamesrbuk/status/970271658769571840?ref_src=twsrc%5Etfw">March 4, 2018</a></blockquote> <script async src="https://platform.twitter.com/widgets.js" charset="utf-8"></script>

---

# Adversarial Examples

Remember: This is a bird!

---

# Speed Limit

---

# Speed Limit

* We can train Neural Networks to recognize numbers, objects, etc.

* But determining what each neuron *does* is complicated

* And since we don't know what is happening, we don't know which "features" (pixels) are important to each decision

* Researchers come up with adversarial examples all the time!

---

# More Adversarial Examples

[Can Your AI Differentiate Cats from Covid-19? Sample Efficient Uncertainty Estimation for Deep Learning Safety](https://api.deepai.org/publication-download-pdf/can-your-ai-differentiate-cats-from-covid-19-sample-efficient-uncertainty-estimation-for-deep-learning-safety)

---

# But ...

<blockquote class="twitter-tweet"><p lang="en" dir="ltr">Thanks all for your interest in our work on “Sample Efficient Uncertainty Estimation for Deep earning”. Unfortunately, the tweeted CAT/COVID-19 teaser image has created a lot of confusion and overshadowed the actual message of the paper.</p>— Bhavya Kailkhura (@bkailkhu) <a href="https://twitter.com/bkailkhu/status/1273108824715595781?ref_src=twsrc%5Etfw">June 17, 2020</a></blockquote>

---

# Style-Transfer: Digital Blackface/Brownface

<blockquote class="twitter-tweet"><p lang="en" dir="ltr">If you're an AI research doing blending, style transfer, or other visual stuff, please stop doing this kind of thing in your talks and papers (or, like, at all). This is <a href="https://twitter.com/nvidia?ref_src=twsrc%5Etfw">@nvidia</a> presenting at a top conference in the field, <a href="https://twitter.com/CVPRConf?ref_src=twsrc%5Etfw">@CVPRConf</a>. <a href="https://t.co/neKHysOE56">pic.twitter.com/neKHysOE56</a></p>— mike cook (@mtrc) <a href="https://twitter.com/mtrc/status/1272943369346584579?ref_src=twsrc%5Etfw">June 16, 2020</a></blockquote>

---

# ML Challenges

"We're kind of like the dog who caught the car," Aguera y Arcas said. Deep learning has rapidly knocked down some longstanding challenges in AI—but it doesn’t immediately seem well suited to many that remain. Problems that involve reasoning or social intelligence, such as weighing up a potential hire in the way a human would, are still out of reach, he said. "All of the models that we have learned how to train are about passing a test or winning a game with a score, [but] so many things that intelligences do aren’t covered by that rubric at all," he said.

From: *A Sobering Message About the Future at AI's Biggest Party* on wired.com

---

[Source: Twitter](https://twitter.com/ykilcher/status/1388473547740299265/photo/1)

---

# Summary

---

# Talk

[Schedule](https://libguides.library.cpp.edu/c.php?g=1020074&p=8338256)

---

# Facial Recognition

---

# References
  
  * [Details of 10.6 million MGM hotel guests posted on a hacking forum](https://www.zdnet.com/article/exclusive-details-of-10-6-million-of-mgm-hotel-guests-posted-on-a-hacking-forum/)
  
  * [Millions of Americans’ Medical Images and Data Are Available on the Internet. Anyone Can Take a Peek.](https://www.propublica.org/article/millions-of-americans-medical-images-and-data-are-available-on-the-internet)
  
  * [A Sobering Message About the Future at AI's Biggest Party](https://www.wired.com/story/sobering-message-future-ai-party/)
  
  * [The Google AI mammogram paper is FLAWED](https://twitter.com/VPrasadMDMPH/status/1212840987363442689?s=20)
  
  * [Why is there so much AI snakeoil](https://www.cs.princeton.edu/~arvindn/talks/MIT-STS-AI-snakeoil.pdf)
  
  * [Machine Bias for Risk Assessment](https://www.propublica.org/article/machine-bias-risk-assessments-in-criminal-sentencing)
  
  * [How a Piece of Tape Tricked a Tesla Into Reading a 35MPH Sign as 85MPH](https://gizmodo.com/how-a-piece-of-tape-tricked-a-tesla-into-reading-a-35mp-1841791417)
  
  * [Artificial Intelligence can discover, exploit personal information via a video game – here's how](https://www.wraltechwire.com/2019/10/14/artificial-intelligence-can-discover-exploit-personal-information-via-a-video-game-heres-how/)