Neural Networks

From jWiki
Revision as of 16:04, 2 November 2023 by Snoopj (talk | contribs) (→‎Academic works)
(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)
Jump to navigationJump to search

Herein lie some of my thoughts and resources about neural networks. Because I work for a company that builds models for computer vision, I have a bit of a professional bias towards image models, but I have tried to represent my knowledge/opinions about a broader range of subjects here.

What do you think about generative "AI"?

tl;dr - mostly dancing bearware, some novel uses in responsibility laundering

Resources

Image models

Text models

For code

For everything else

  • Washington Post coverage of the data contained in the 'C4' dataset and how it influences the training of popular large models. Also allows users to check if arbitrary URLs are part of the dataset. (NOTE: C4 is not the only source of training text for the models being discussed, and the authors aren't doing a great job highlighting that, but it should still be pretty representative)
  • How well does ChatGPT speak Japanese? - an April 2023 evaluation of GPT-3.5 and GPT-4 performance on Japanese language assessments. Also includes an interesting comparison of the number of tokens required to represent the "Lord's Prayer" in multiple languages. I found the results of the latter particularly surprising.

Misc.

  • I gave a talk on the fundamentals of neural networks to Boston Python in March 2023
  • 3blue1brown has an excellent series of lessons about the fundamentals of neural networks. Particularly interesting to me is the lesson on backpropagation for its excellent visualization of the process of adjusting neural network weights.

Writings by others

Academic works

Non-academic works

Lawsuits

The legal status of generative models and their implications for intellectual property in the US is something I'm trying to keep an eye on. The cases given below are of particular interest to me.

ANDERSEN v. STABILITY AI LTD.

Entry #131 in Andersen v. Stability AI Ltd., 3:23-cv-00201
Order on Motion to Withdraw as Attorney
Entry #130 in Andersen v. Stability AI Ltd., 3:23-cv-00201
MOTION to Withdraw as Attorney Cooley LLP's Withdraw as Counsel filed by Midjourney, Inc.. Motion Hearing set for 1/10/2024 02:00 PM in San Francisco, Courtroom 02, 17th Floor before Judge Will...
Entry #129 in Andersen v. Stability AI Ltd., 3:23-cv-00201
AMENDED COMPLAINT FIRST against DeviantArt, Inc., Midjourney, Inc., Stability AI Ltd., Stability AI, Inc., Runway AI, Inc.. Filed by Karla Ortiz, Kelly McKernan, Sarah Andersen, Gregory Manchess, A...


GETTY IMAGES (US), INC. v. STABILITY AI, INC.

Entry #33 in Getty Images (US), Inc. v. Stability AI, Inc., 1:23-cv-00135
Letter to The Honorable Gregory B. Williams from Michael Flynn regarding response to Getty Images (US), Inc.'s October 9, 2023 Discovery Dispute Letter - re 32 Letter. (Flynn, Michael) (Entered...
Minute entry from 2023-10-11 in Getty Images (US), Inc. v. Stability AI, Inc., 1:23-cv-00135
SO ORDERED, re 31 MOTION for Pro Hac Vice Appearance of Attorney Brian Liegel filed by Getty Images (US), Inc. Signed by Judge Gregory B. Williams on 10/11/23. (ntl)
Entry #32 in Getty Images (US), Inc. v. Stability AI, Inc., 1:23-cv-00135
Letter to The Honorable Gregory B. Williams from Robert M. Vrana regarding Discovery Dispute - re 30 Order,,,,,. (Attachments: # 1 Exhibit A, # 2 Exhibit B, # 3 Text of Proposed Order)(Vrana, Rober...

DOE 1 v. GITHUB, INC.

Entry #177 in DOE 1 v. GitHub, Inc., 4:22-cv-06823
Notice of Reference and Order re: Discovery Procedures; Order Denying Without Prejudice 173 Discovery Letter Brief. (dmrlc2, COURT STAFF) (Filed on 11/28/2023)
Minute entry from 2023-11-21 in DOE 1 v. GitHub, Inc., 4:22-cv-06823
CASE REFERRED to Magistrate Judge Donna M. Ryu for Discovery (mkl, COURT STAFF) (Filed on 11/21/2023)
Entry #176 in DOE 1 v. GitHub, Inc., 4:22-cv-06823
NOTICE by GitHub, Inc., Microsoft Corporation NOTICE OF SUPPLEMENTAL AUTHORITY RELEVANT TO DEFENDANTS GITHUB AND MICROSOFTS MOTIONS TO DISMISS PORTIONS OF THE FIRST AMENDED COMPLAINT IN CONSOLIDATE...

SILVERMAN v. OPENAI, INC.

Entry #60 in Silverman v. OpenAI, Inc., 3:23-cv-03416
Transcript of Proceedings held on 11/08/23, before Judge Araceli Martinez-Olguin. Court Reporter/Transcriber Echo Reporting, Inc., telephone number echoreporting@yahoo.com. Per General Order No. 59...
Entry #59 in Silverman v. OpenAI, Inc., 3:23-cv-03416
STATEMENT OF RECENT DECISION pursuant to Civil Local Rule 7-3.d filed byOpenAI GP, L.L.C., OpenAI OpCo, L.L.C., OpenAI Startup Fund GP I, L.L.C., OpenAI Startup Fund I, L.P., OpenAI Startup Fund Ma...
Entry #58 in Silverman v. OpenAI, Inc., 3:23-cv-03416
PRETRIAL ORDER NO. 1. Signed by Judge Araceli Martinez-Olguin on 11/9/2023. (ads, COURT STAFF) (Filed on 11/9/2023) (Entered: 11/09/2023)

MATA v. AVIANCA, INC. (closed)

Note: this case is not about machine learning textually, but is included in this list because it is a notable example of gross misuse of a language model by plaintiff's counsel to submit falsified documents to the court. This led to censure of plaintiff's counsel and dismissal of the case.