Herein lie some of my thoughts and resources about neural networks. Because I work for a company that builds models for computer vision, I have a bit of a professional bias towards image models, but I have tried to represent my knowledge/opinions about a broader range of subjects here.

What do you think about generative "AI"?

tl;dr - mostly dancing bearware, some novel uses in responsibility laundering

Resources

Image models

Stanford CS231n: Deep Learning for Computer Vision - excellent introductory course in computer vision (from kNN to VGGNet) focused on neural networks, with exercises done in Python (with numpy)
How to trick a neural network into thinking a panda is a vulture - excellent exploration by Julia Evans (with Python source code) of an adversarial attack on an image classifier
Multi-modal prompt injection image attacks against GPT-4V - "The fundamental problem here is this: Large Language Models are gullible...we need them to stay gullible. They’re useful because they follow our instructions. Trying to differentiate between “good” instructions and “bad” instructions is a very hard—currently intractable—problem." A very similar style of attack as one against the CLIP architecture published by OpenAI themselves.

Text models

For code

For everything else

Washington Post coverage of the data contained in the 'C4' dataset and how it influences the training of popular large models. Also allows users to check if arbitrary URLs are part of the dataset. (NOTE: C4 is not the only source of training text for the models being discussed, and the authors aren't doing a great job highlighting that, but it should still be pretty representative)
How well does ChatGPT speak Japanese? - an April 2023 evaluation of GPT-3.5 and GPT-4 performance on Japanese language assessments. Also includes an interesting comparison of the number of tokens required to represent the "Lord's Prayer" in multiple languages. I found the results of the latter particularly surprising.

Misc.

I gave a talk on the fundamentals of neural networks to Boston Python in March 2023
3blue1brown has an excellent series of lessons about the fundamentals of neural networks. Particularly interesting to me is the lesson on backpropagation for its excellent visualization of the process of adjusting neural network weights.

Writings by others

Academic works

"The Fallacy of AI Functionality" - "...fear of misspecified objectives, runaway feedback loops, and AI alignment presumes the existence of an industry that can get AI systems to execute on any clearly declared objectives, and that the main challenge is to choose and design an appropriate goal. Needless to say, if one thinks the danger of AI is that it will work too well, it is a necessary precondition that it works at all."
"Adversarial Reprogramming of Neural Networks" - "In each [of six cases], we reprogrammed the [classification] network [trained on ImageNet] to perform three different adversarial tasks: counting squares, MNIST classification, and CIFAR-10 classification… Our finding…[suggests] that the reprogramming across domains is likely [possible]."
"Universal and Transferable Adversarial Attacks on Aligned Language Models" - "For Harmful Behaviors, our approach achieves an attack success rate of 100% on Vicuna-7B and 88% on Llama-2-7B-Chat… we find that the adversarial examples also transfer to Pythia, Falcon, Guanaco, and surprisingly, to GPT-3.5 (87.9%) and GPT-4 (53.6%), PaLM-2 (66%), and Claude-2 (2.1%)."
"Mathematical Capabilities of ChatGPT" - in which ChatGPT and GPT4 largely fail to muster passing performance on a mathematical problem set, compared to a domain-specific model that achieves nearly 100% performance.
"Unmasking Clever Hans predictors and assessing what machines really learn" - "...it is important to comprehend the decision-making process itself...transparency of the what and why in a decision of a nonlinear machine becomes very effective for the essential task of judging whether the learned strategy is valid and generalizable or whether the model has based its decision on a spurious correlation in the training data"
"On the Dangers of Stochastic Parrots: Can Language Models Be Too Big? 🦜" - "LMs with extremely large numbers of parameters model their training data very closely and can be prompted to output specific information from that training data"
"Asleep at the Keyboard? Assessing the Security of GitHub Copilot's Code Contributions" - "In total, we produce 89 different scenarios for Copilot to complete, producing 1,689 programs. Of these, we found approximately 40% to be vulnerable."
"Do Users Write More Insecure Code with AI Assistants?" - "We observed that participants who had access to [codex-davinci-002] were more likely to introduce security vulnerabilities for the majority of programming tasks, yet also more likely to rate their insecure answers as secure compared to those in our control group."
"ChatGPT is fun, but it is not funny! Humor is still challenging Large Language Models" - "Over 90% of 1008 generated jokes were the same 25 Jokes."
"How is ChatGPT's behavior changing over time?" - "We find that the performance and behavior of both GPT-3.5 and GPT-4 can vary greatly over time."
"Are Emergent Abilities of Large Language Models a Mirage?" - "For a fixed task and a fixed model family, the researcher can choose a metric to create an emergent ability or choose a metric to ablate an emergent ability. Ergo, emergent abilities may be creations of the researcher’s choices, not a fundamental property of the model family on the specific task"
"Extracting Training Data from Large Language Models" - "We demonstrate our attack on GPT-2, a language model trained on scrapes of the public Internet, and are able to extract hundreds of verbatim text sequences from the model's training data...we find that larger models are more vulnerable than smaller models."
"Speak, Memory: An Archaeology of Books Known to ChatGPT/GPT-4" - "We find that these models have memorized books, both in the public domain and in copyright, and the capacity for memorization is tied to a book’s overall popularity on the web. This differential in memorization leads to differential in performance for downstream tasks, with better performance on popular books than on those not seen on the web"
"Who Answers It Better? An In-Depth Analysis of ChatGPT and Stack Overflow Answers to Software Engineering Questions" - "Our user study results show that users prefer ChatGPT answers 34.82% of the time. However, 77.27% of these preferences are incorrect answers"

Non-academic works

Lindsey Kuper's CSE232 syllabus section on LLM usage - "Aside from the fact that the resounding hollowness of the ChatGPT-produced prose has sucked away all of my zest for life…please understand that while you are welcome to use LLM-based tools in this course, you should be aware of their limitations."
Time: "OpenAI Used Kenyan Workers on Less Than $2 Per Hour to Make ChatGPT Less Toxic"
- The human labor that powers ChatGPT's reinforcement learning from human feedback (RLHF)
Donald Knuth: correspondence with Stephen Wolfram - "I myself shall certainly continue to leave such research to others, and to devote my time to developing concepts that are authentic and trustworthy. And I hope you do the same."
Douglas Hofstadter: "Gödel, Escher, Bach, and AI" - "I frankly am baffled by the allure, for so many unquestionably insightful people...of letting opaque computational systems perform intellectual tasks for them."
Ted Chiang: "ChatGPT Is a Blurry JPEG of the Web" - "Large language models identify statistical regularities in text...When we’re dealing with sequences of words, lossy compression looks smarter than lossless compression."
Ted Chiang: "Will A.I. Become the New McKinsey?" - "I’m not very convinced by claims that A.I. poses a danger to humanity because it might develop goals of its own and prevent us from turning it off. However, I do think that A.I. is dangerous inasmuch as it increases the power of capitalism."

Lawsuits

The legal status of generative models and their implications for intellectual property in the US is something I'm trying to keep an eye on. The cases given below are of particular interest to me.

ANDERSEN v. STABILITY AI LTD.

January 2023 coverage: initial complaint
Latest case proceedings:

Entry #133 in Andersen v. Stability AI Ltd., 3:23-cv-00201: Notice of Appearance
Entry #132 in Andersen v. Stability AI Ltd., 3:23-cv-00201: NOTICE of Change of Address by Angela Dunning (Dunning, Angela) (Filed on 12/5/2023) (Entered: 12/05/2023)
Entry #131 in Andersen v. Stability AI Ltd., 3:23-cv-00201: ORDER GRANTING 130 COOLEY LLP'S MOTION TO WITHDRAW AS COUNSEL FOR DEFENDANT MIDJOURNEY, INC. by Judge William H. Orrick. (jmd, COURT STAFF) (Filed on 12/1/2023) (Entered: 12/01/2023)

GETTY IMAGES (US), INC. v. STABILITY AI, INC.

Entry #33 in Getty Images (US), Inc. v. Stability AI, Inc., 1:23-cv-00135: Letter to The Honorable Gregory B. Williams from Michael Flynn regarding response to Getty Images (US), Inc.'s October 9, 2023 Discovery Dispute Letter - re 32 Letter. (Flynn, Michael) (Entered...
Minute entry from 2023-10-11 in Getty Images (US), Inc. v. Stability AI, Inc., 1:23-cv-00135: SO ORDERED, re 31 MOTION for Pro Hac Vice Appearance of Attorney Brian Liegel filed by Getty Images (US), Inc. Signed by Judge Gregory B. Williams on 10/11/23. (ntl)
Entry #32 in Getty Images (US), Inc. v. Stability AI, Inc., 1:23-cv-00135: Letter to The Honorable Gregory B. Williams from Robert M. Vrana regarding Discovery Dispute - re 30 Order,,,,,. (Attachments: # 1 Exhibit A, # 2 Exhibit B, # 3 Text of Proposed Order)(Vrana, Rober...

DOE 1 v. GITHUB, INC.

Entry #179 in DOE 1 v. GitHub, Inc., 4:22-cv-06823: Notice of Appearance
Entry #178 in DOE 1 v. GitHub, Inc., 4:22-cv-06823: Notice of Appearance
Entry #177 in DOE 1 v. GitHub, Inc., 4:22-cv-06823: Notice of Reference and Order re: Discovery Procedures; Order Denying Without Prejudice 173 Discovery Letter Brief. (dmrlc2, COURT STAFF) (Filed on 11/28/2023)

SILVERMAN v. OPENAI, INC.

July 2023 coverage: initial complaint
Latest case proceedings:

Minute entry from 2023-12-08 in Silverman v. OpenAI, Inc., 3:23-cv-03416: Motion Hearing
Entry #63 in Silverman v. OpenAI, Inc., 3:23-cv-03416: Minute Entry for proceedings held before Judge Araceli Martinez-Olguin: Motions to Dismiss Hearing held on 12/7/2023. Argument heard; matter submitted. Court to issue Order. Total Time in Court: 18...
Entry #62 in Silverman v. OpenAI, Inc., 3:23-cv-03416: Set/Reset Deadlines as to (32 in 3:23-cv-03416-AMO) MOTION to Dismiss., (33 in 3:23-cv-03223-AMO) MOTION to Dismiss, (This is a text-only entry generated by the court. There is no document associat...

MATA v. AVIANCA, INC. (closed)

Note: this case is not about machine learning textually, but is included in this list because it is a notable example of gross misuse of a language model by plaintiff's counsel to submit falsified documents to the court. This led to censure of plaintiff's counsel and dismissal of the case.

Neural Networks

Contents