Neural Networks

From jWiki
Jump to navigationJump to search

Herein lie some of my thoughts and resources about neural networks. Because I work for a company that builds models for computer vision, I have a bit of a professional bias towards image models, but I have tried to represent my knowledge/opinions about a broader range of subjects here.

What do you think about generative "AI"?

tl;dr - mostly dancing bearware, some novel uses in responsibility laundering

Resources

Image models

Text models

For code

For everything else

  • Washington Post coverage of the data contained in the 'C4' dataset and how it influences the training of popular large models. Also allows users to check if arbitrary URLs are part of the dataset. (NOTE: C4 is not the only source of training text for the models being discussed, and the authors aren't doing a great job highlighting that, but it should still be pretty representative)
  • How well does ChatGPT speak Japanese? - an April 2023 evaluation of GPT-3.5 and GPT-4 performance on Japanese language assessments. Also includes an interesting comparison of the number of tokens required to represent the "Lord's Prayer" in multiple languages. I found the results of the latter particularly surprising.

Misc.

  • I gave a talk on the fundamentals of neural networks to Boston Python in March 2023
  • 3blue1brown has an excellent series of lessons about the fundamentals of neural networks. Particularly interesting to me is the lesson on backpropagation for its excellent visualization of the process of adjusting neural network weights.

Dumping ground

These references are totally unclassified

Writings by others

Academic works

Non-academic works

Lawsuits

The legal status of generative models and their implications for intellectual property in the US is something I'm trying to keep an eye on. The cases given below are of particular interest to me.

The New York Times Company v. MICROSOFT CORPORATION

Entry #124 in The New York Times Company v. Microsoft Corporation, 1:23-cv-11195
LETTER MOTION to Compel The New York Times Company to Produce Documents addressed to Judge Sidney H. Stein from Joseph R. Wetzel dated May 23, 2024. Document filed by OAI Corporation, LLC, OpenAI G...
Entry #123 in The New York Times Company v. Microsoft Corporation, 1:23-cv-11195
LETTER RESPONSE to Motion addressed to Judge Sidney H. Stein from Michelle Ybarra and Joseph R. Wetzel and Allyson R. Bennett dated May 22, 2024 re: 117 LETTER MOTION for Conference Concerning Plai...
Entry #122 in The New York Times Company v. Microsoft Corporation, 1:23-cv-11195
LETTER RESPONSE to Motion addressed to Judge Sidney H. Stein from Jared B. Briant dated 05/22/2024 re: 117 LETTER MOTION for Conference Concerning Plaintiff's Request for Bi-monthly Status Conf...

Andersen v. Stability AI Ltd.

Entry #216 in Andersen v. Stability AI Ltd., 3:23-cv-00201
Order AND ~Util - Add and Terminate Attorneys
Entry #215 in Andersen v. Stability AI Ltd., 3:23-cv-00201
Order AND ~Util - Add and Terminate Attorneys
Entry #214 in Andersen v. Stability AI Ltd., 3:23-cv-00201
Order AND ~Util - Add and Terminate Attorneys


Getty Images (US), Inc. v. Stability AI, Inc.

Entry #39 in Getty Images (US), Inc. v. Stability AI, Inc., 1:23-cv-00135
NOTICE requesting Clerk to remove Nicole M. Jantzi, Paul M. Schoenhard, Michael C. Keats and Amir R. Ghavi as co-counsel. Reason for request: no longer associated with the case. (Flynn, Michael) (E...
Entry #38 in Getty Images (US), Inc. v. Stability AI, Inc., 1:23-cv-00135
NOTICE to Take Deposition of Peter O'Donoghue on February 22, 2024 filed by Getty Images (US), Inc..(Vrana, Robert) (Entered: 02/13/2024)
Entry #37 in Getty Images (US), Inc. v. Stability AI, Inc., 1:23-cv-00135
NOTICE OF SERVICE of (1) Stability AI Ltd.'s Second Supplemental Objections and Responses to Plaintiff's Jurisdictional Interrogatories Nos. 2 and 12; and (2) Stability AI, Inc.'s Secon...

Doe 1 v. GitHub, Inc.

Minute entry from 2024-05-09 in DOE 1 v. GitHub, Inc., 4:22-cv-06823
Order AND Set Motion and Deadlines/Hearings
Entry #252 in DOE 1 v. GitHub, Inc., 4:22-cv-06823
CLERKS NOTICE SETTING ZOOM HEARING. Notice is hereby given to all parties that a motion hearing on 248 Joint Discovery Letter Brief is set for 7/11/2024 01:00 PM in Oakland, - Videoconference Only...
Entry #251 in DOE 1 v. GitHub, Inc., 4:22-cv-06823
NOTICE by OAI CORPORATION,, OPENAI GLOBAL, LLC, OPENAI HOLDCO, LLC, OPENAI HOLDINGS, LLC, OPENAI INVESTMENT LLC, OPENAI STARTUP FUND SPV GP I, L.L.C., OPENAI STARTUP FUND SPV I, L.P, OPENAI, L.L.C....

Silverman v. OpenAI, Inc.

Entry #70 in Silverman v. OpenAI, Inc., 3:23-cv-03416
PRETRIAL ORDER as Modified. Signed by Judge Araceli Martinez-Olguin on 2/16/2024. (ads, COURT STAFF) (Filed on 2/16/2024) (Entered: 02/16/2024)
Entry #69 in Silverman v. OpenAI, Inc., 3:23-cv-03416
Order as Modified by Judge Araceli Martinez-Olguin granting (60) Stipulation Consolidating Cases in case 3:23-cv-03223-AMO. Associated Cases: 3:23-cv-03223-AMO, 3:23-cv-03416-AMO, 3:23-cv-04625-AMO...
Entry #68 in Silverman v. OpenAI, Inc., 3:23-cv-03416
Order by Judge Araceli Martinez-Olguin granting in part and denying in part 32 Motion to Dismiss. Cross-posted in 23-cv-03223. (amolc2, COURTSTAFF) (Filed on 2/12/2024) (Entered: 02/12/2024)

Kadrey v. Meta Platforms, Inc.

  • Similar suit to Silverman v. OpenAI, same parties etc.
  • Notable for a prominent dismissal of the class-action nature of the case, as the blatantly copied copyrighted works in the training data are not the works of the plaintiffs.
  • Latest [1]:
Entry #101 in Kadrey v. Meta Platforms, Inc., 3:23-cv-03417
ORDER by Magistrate Judge Thomas S. Hixson granting 100 Stipulation. (rmm2, COURT STAFF) (Filed on 4/10/2024) (Entered: 04/10/2024)
Entry #100 in Kadrey v. Meta Platforms, Inc., 3:23-cv-03417
STIPULATION WITH PROPOSED ORDER JOINT MOTION FOR ENTRY OF PROPOSED STIPULATED ORDER RE: DISCOVERY OF ELECTRONICALLY STORED INFORMATION filed by John Blase, Michael Chabon, Ta-Nehisi Coates, Junot D...
Entry #99 in Kadrey v. Meta Platforms, Inc., 3:23-cv-03417
NOTICE of Change of Address by Joseph R. Saveri (Saveri, Joseph) (Filed on 4/5/2024) (Entered: 04/05/2024)

Authors Guild v. OpenAI Inc.

Entry #150 in Authors Guild v. OpenAI Inc., 1:23-cv-08292
LETTER RESPONSE in Opposition to Motion addressed to Judge Sidney H. Stein from Paven Malhotra dated 05/17/2024 re: (122 in 1:23-cv-10211-SHS) LETTER MOTION for Conference Concerning Plaintiffs&#39...
Entry #149 in Authors Guild v. OpenAI Inc., 1:23-cv-08292
LETTER RESPONSE to Motion addressed to Judge Sidney H. Stein from Annette L. Hurst dated 05/17/2024 re: (147 in 1:23-cv-08292-SHS, 122 in 1:23-cv-10211-SHS) LETTER MOTION for Conference Concerning...
Entry #148 in Authors Guild v. OpenAI Inc., 1:23-cv-08292
STIPULATED PROTECTIVE ORDER...regarding procedures to be followed that shall govern the handling of confidential material...SO STIPULATED AND AGREED. The protective order may be amended for good ca...

Sancton v. OpenAI Inc. et al

[ Document 21]
ORDER granting #17 Motion for Alejandra Christina Salinas to Appear Pro Hac Vice (HEREBY ORDERED by Judge Sidney H. Stein)(Text Only Order) (lab)
2023-11-30 08:00:00
[ Document 20]
ORDER granting #14 Motion for Rohit Dwarka Nath to Appear Pro Hac Vice (HEREBY ORDERED by Judge Sidney H. Stein)(Text Only Order) (lab)
2023-11-30 08:00:00
[ Document 19]
ORDER granting #16 Motion for Justin Adatto Nelson to Appear Pro Hac Vice (HEREBY ORDERED by Judge Sidney H. Stein)(Text Only Order) (lab)
2023-11-30 08:00:00

Mata v. Avianca, Inc. (closed)

Note: this case is not about machine learning textually, but is included in this list because it is a notable example of gross misuse of a language model by plaintiff's counsel to submit falsified documents to the court. This led to censure of plaintiff's counsel and dismissal of the case.