class: center, middle # Artificial Intelligence ## Garbage in - Racism out: Ethical considerations for Machine Learning --- class: center, middle # Data Collection --- # Data Collection * So you want to do Machine Learning * You will need data * Where do you get that data? * What is legal and moral to collect? --- # Data Collection: GDPR * The European Union passed a regulation as part of EU law in 2018, called the "General Data Protection Regulation" * This regulation limits data collection to cases in which there is either explicit consent or a contractual, legal or official **need** * EU citizens can also request all data that is stored about them! --- class: medium # California Consumer Privacy Act * Know what personal data is being collected about them. * Know whether their personal data is sold or disclosed and to whom. * Say no to the sale of personal data. * Access their personal data. * Request a business to delete any personal information about a consumer collected from that consumer. * Not be discriminated against for exercising their privacy rights. --- # But we are in Costa Rica! * The GDPR only applies to the EU and EEA * The CCPA only applies to residents of California * Where are we likely to collect data? The internet ... * If you collect data online, you have to ensure that these laws are either met, or the corresponding regions are excluded from collection --- class: center, middle # Data Usage --- # Data Usage * We have seen some Machine Learning techniques * If we have data (obtained with consent), we can produce predictions * What are we gonna find? * What do we want to find? * And just because we can: Should we? --- # Automated Breast Cancer Screening! <img src="/CI-2600/assets/img/AIMammography.png" width="60%"/> --- # Automated Breast Cancer Screening! <img src="/CI-2600/assets/img/AIMammographymetric.png" width="60%"/> What's the problem with this metric? -- "There may be biopsies that AI would have encouraged that do not exist, and we don't know the results of tests that were not done." (Vinay Prasad on twitter) --- # Another problem * Just because we find "more cancer" does not mean we improved anything * Cancer may be benign, we don't (necessarily) want/need to find more of those * Cancer may be too advanced, finding more of those also does not help * Distinguishing between benign and and harmful, but treatable cancer is the hard problem! --- # Another problem <blockquote class="twitter-tweet"><p lang="en" dir="ltr">Oh look, it's an example of AI being used to solve the wrong problem! <a href="https://t.co/A6P92XPwjk">https://t.co/A6P92XPwjk</a> Study shows that AI can find more cancers on mammograms, but that's the wrong goal. We should aim to save lives, not turn more healthy women into cancer patients.</p>— Christie Aschwanden (@cragcrest) <a href="https://twitter.com/cragcrest/status/1212736228640120833?ref_src=twsrc%5Etfw">January 2, 2020</a></blockquote> <script async src="https://platform.twitter.com/widgets.js" charset="utf-8"></script> --- class: medium # Racism In - Garbage Out * Say we catch a criminal * The judge has to decide if they are likely to re-offend or not * That determines the bail, how long they will go for jail, and potentially other privileges * But there is (documented) racism in the justice system, and black criminals get harsher sentences on average * Idea: Let's train a Neural Network to recommend bail, sentences, "dangerousness" for criminals! The Neural Network can't be racist! --- class: medium # Training a Neural Network * How do we train the Neural Network? * We need data! * Fortunately, there is lots of data we can use: all court cases that have already happened * What could possibly go wrong? --- # Racist Neural Network <img src="/CI-2600/assets/img/riskassessment.png" width="80%"/> --- # Racist Neural Network <img src="/CI-2600/assets/img/riskscores.png" width="65%"/> --- # Something less harmful: Video games! * Say we have a video game * We collect (with consent!) user's game actions and data * Of course we do this to make more money: recommend purchases, show ads, etc. * What else could we do? --- class: mediumt # Video Games <img src="/CI-2600/assets/img/videogamepersonal.png" width="80%"/> Dr. Chris Hazard: *If you are undervaluing or overvaluing positive utility events—so what that means is that if there’s a positive outcome in a game, like "Oh I got this reward. I got this treasure. Oh, that's really awesome. I really value that. But oh, if I lose this thing or if I gain this one coin or whatever it's not that much."* *It turns out, according to the two-month study, that depressed people more accurately value positive utility events than non-depressed people. So, think about that for a second.* --- # Something less harmful: Hiring <blockquote class="twitter-tweet"><p lang="en" dir="ltr">Vision: algorithms will make hiring better as they don’t discriminate<br><br>Reality: “One HR employee for a major technology company recommends slipping the words “Oxford” or “Cambridge” into a CV in invisible white text, to pass the automated screening.”<br> <a href="https://t.co/m8jYKLzDNq">https://t.co/m8jYKLzDNq</a></p>— James Ball (@jamesrbuk) <a href="https://twitter.com/jamesrbuk/status/970271658769571840?ref_src=twsrc%5Etfw">March 4, 2018</a></blockquote> <script async src="https://platform.twitter.com/widgets.js" charset="utf-8"></script> --- # Adversarial Examples <center> <img src="/CI-2600/assets/img/badcat.jpg" width="80%"/> </center> Remember: This is a bird! --- # Speed Limit <img src="/CI-2600/assets/img/35mph.png" width="50%"/> --- # Speed Limit * We can train Neural Networks to recognize numbers, objects, etc. * But determining what each neuron *does* is complicated * And since we don't know what is happening, we don't know which "features" (pixels) are important to each decision * Researchers come up with adversarial examples all the time! --- # More Adversarial Examples <blockquote class="twitter-tweet"><p lang="en" dir="ltr">Can Your AI Differentiate Cats from COVID-19?<br><br>There is a recent interest in using AI to detect COVID-19 from Chest X-Ray (CXR) images. Our work serve as a warning to practitioners by showing that overconfident and inaccurate predictions become severe in small-data regime. <a href="https://t.co/hMA7TQcuSZ">pic.twitter.com/hMA7TQcuSZ</a></p>— Bhavya Kailkhura (@bkailkhu) <a href="https://twitter.com/bkailkhu/status/1272619636710752256?ref_src=twsrc%5Etfw">June 15, 2020</a></blockquote> --- # Style-Transfer: Digital Blackface/Brownface <blockquote class="twitter-tweet"><p lang="en" dir="ltr">If you're an AI research doing blending, style transfer, or other visual stuff, please stop doing this kind of thing in your talks and papers (or, like, at all). This is <a href="https://twitter.com/nvidia?ref_src=twsrc%5Etfw">@nvidia</a> presenting at a top conference in the field, <a href="https://twitter.com/CVPRConf?ref_src=twsrc%5Etfw">@CVPRConf</a>. <a href="https://t.co/neKHysOE56">pic.twitter.com/neKHysOE56</a></p>— mike cook (@mtrc) <a href="https://twitter.com/mtrc/status/1272943369346584579?ref_src=twsrc%5Etfw">June 16, 2020</a></blockquote> --- # ML Challenges "We're kind of like the dog who caught the car," Aguera y Arcas said. Deep learning has rapidly knocked down some longstanding challenges in AI—but it doesn’t immediately seem well suited to many that remain. Problems that involve reasoning or social intelligence, such as weighing up a potential hire in the way a human would, are still out of reach, he said. "All of the models that we have learned how to train are about passing a test or winning a game with a score, [but] so many things that intelligences do aren’t covered by that rubric at all," he said. From: *A Sobering Message About the Future at AI's Biggest Party* on wired.com --- # Summary <center> <img src="/CI-2600/assets/img/MLprogresssummary.png" width="100%"/> </center> --- # Facial Recognition <iframe width="560" height="315" src="https://www.youtube.com/embed/jZjmlJPJgug" frameborder="0" allow="accelerometer; autoplay; encrypted-media; gyroscope; picture-in-picture" allowfullscreen></iframe> --- # Call for Participation * I have a master's student working on an AI agent for the board game Pandemic * He is currently running an experiment with human players * If you like board games and/or AI, please participate: http://pandemic.slothlab.info --- class: small # References * [Details of 10.6 million MGM hotel guests posted on a hacking forum](https://www.zdnet.com/article/exclusive-details-of-10-6-million-of-mgm-hotel-guests-posted-on-a-hacking-forum/) * [Millions of Americans’ Medical Images and Data Are Available on the Internet. Anyone Can Take a Peek.](https://www.propublica.org/article/millions-of-americans-medical-images-and-data-are-available-on-the-internet) * [A Sobering Message About the Future at AI's Biggest Party](https://www.wired.com/story/sobering-message-future-ai-party/) * [The Google AI mammogram paper is FLAWED](https://twitter.com/VPrasadMDMPH/status/1212840987363442689?s=20) * [Why is there so much AI snakeoil](https://www.cs.princeton.edu/~arvindn/talks/MIT-STS-AI-snakeoil.pdf) * [Machine Bias for Risk Assessment](https://www.propublica.org/article/machine-bias-risk-assessments-in-criminal-sentencing) * [How a Piece of Tape Tricked a Tesla Into Reading a 35MPH Sign as 85MPH](https://gizmodo.com/how-a-piece-of-tape-tricked-a-tesla-into-reading-a-35mp-1841791417) * [Artificial Intelligence can discover, exploit personal information via a video game – here's how](https://www.wraltechwire.com/2019/10/14/artificial-intelligence-can-discover-exploit-personal-information-via-a-video-game-heres-how/)