Neural Networks

From jWiki
Jump to navigationJump to search

Herein lie some of my thoughts and resources about neural networks. Because I work for a company that builds models for computer vision, I have a bit of a professional bias towards image models, but I have tried to represent my knowledge/opinions about a broader range of subjects here.

What do you think about generative "AI"?

tl;dr - mostly dancing bearware, some novel uses in responsibility laundering

Resources

Image models

Text models

For code

For everything else

  • Washington Post coverage of the data contained in the 'C4' dataset and how it influences the training of popular large models. Also allows users to check if arbitrary URLs are part of the dataset. (NOTE: C4 is not the only source of training text for the models being discussed, and the authors aren't doing a great job highlighting that, but it should still be pretty representative)
  • How well does ChatGPT speak Japanese? - an April 2023 evaluation of GPT-3.5 and GPT-4 performance on Japanese language assessments. Also includes an interesting comparison of the number of tokens required to represent the "Lord's Prayer" in multiple languages. I found the results of the latter particularly surprising.

Misc.

  • I gave a talk on the fundamentals of neural networks to Boston Python in March 2023
  • 3blue1brown has an excellent series of lessons about the fundamentals of neural networks. Particularly interesting to me is the lesson on backpropagation for its excellent visualization of the process of adjusting neural network weights.

Dumping ground

These references are totally unclassified

Writings by others

Academic works

Non-academic works

Lawsuits

The legal status of generative models and their implications for intellectual property in the US is something I'm trying to keep an eye on. The cases given below are of particular interest to me.

The New York Times Company v. MICROSOFT CORPORATION

Entry #98 in The New York Times Company v. Microsoft Corporation, 1:23-cv-11195
ORDER granting 94 Motion for Michelle S. Ybarra to Appear Pro Hac Vice (HEREBY ORDERED by Judge Sidney H. Stein)(Text Only Order) (lab) (Entered: 04/25/2024)
Entry #97 in The New York Times Company v. Microsoft Corporation, 1:23-cv-11195
ORDER granting 93 Motion for Nicholas S. Goldberg to Appear Pro Hac Vice (HEREBY ORDERED by Judge Sidney H. Stein)(Text Only Order) (lab) (Entered: 04/25/2024)
Entry #96 in The New York Times Company v. Microsoft Corporation, 1:23-cv-11195
ORDER granting 92 Motion for Robert Addy Van Nest Jr to Appear Pro Hac Vice (HEREBY ORDERED by Judge Sidney H. Stein)(Text Only Order) (lab) (Entered: 04/25/2024)

Andersen v. Stability AI Ltd.

Entry #190 in Andersen v. Stability AI Ltd., 3:23-cv-00201
NOTICE of Appearance by Luke P. Apfeld (Apfeld, Luke) (Filed on 4/19/2024) (Entered: 04/19/2024)
Entry #189 in Andersen v. Stability AI Ltd., 3:23-cv-00201
RESPONSE re 165 Request for Judicial Notice by Runway AI, Inc.. (Malhotra, Paven) (Filed on 4/18/2024) (Entered: 04/18/2024)
Entry #188 in Andersen v. Stability AI Ltd., 3:23-cv-00201
REPLY (re 164 MOTION to Dismiss Plaintiffs First Amended Complaint ) filed byRunway AI, Inc.. (Malhotra, Paven) (Filed on 4/18/2024) (Entered: 04/18/2024)


Getty Images (US), Inc. v. Stability AI, Inc.

Entry #38 in Getty Images (US), Inc. v. Stability AI, Inc., 1:23-cv-00135
NOTICE to Take Deposition of Peter O'Donoghue on February 22, 2024 filed by Getty Images (US), Inc..(Vrana, Robert) (Entered: 02/13/2024)
Entry #37 in Getty Images (US), Inc. v. Stability AI, Inc., 1:23-cv-00135
NOTICE OF SERVICE of (1) Stability AI Ltd.'s Second Supplemental Objections and Responses to Plaintiff's Jurisdictional Interrogatories Nos. 2 and 12; and (2) Stability AI, Inc.'s Secon...
Entry #36 in Getty Images (US), Inc. v. Stability AI, Inc., 1:23-cv-00135
ORAL ORDER: Defendants' Motion to Dismiss or Transfer (D.I. 16 ) is DISMISSED without prejudice to re-file after the conclusion of jurisdictional discovery. Ordered by Judge Jennifer L. Hall on...

Doe 1 v. GitHub, Inc.

Entry #248 in DOE 1 v. GitHub, Inc., 4:22-cv-06823
Discovery Letter Brief
Entry #247 in DOE 1 v. GitHub, Inc., 4:22-cv-06823
Joint Discovery Letter Brief filed by J. DOE 1, J. DOE 2, J. Doe 3, J. Doe 4, J. Doe 5. (Saveri, Joseph) (Filed on 4/17/2024) (Entered: 04/17/2024)
Entry #246 in DOE 1 v. GitHub, Inc., 4:22-cv-06823
Order by Judge Jon S. Tigar denying 218 Motion for Reconsideration re 218 MOTION for Reconsideration re 189 Order on Motion to Dismiss, filed by J. Doe 5, J. DOE 1, J. Doe 4,...

Silverman v. OpenAI, Inc.

Entry #70 in Silverman v. OpenAI, Inc., 3:23-cv-03416
PRETRIAL ORDER as Modified. Signed by Judge Araceli Martinez-Olguin on 2/16/2024. (ads, COURT STAFF) (Filed on 2/16/2024) (Entered: 02/16/2024)
Entry #69 in Silverman v. OpenAI, Inc., 3:23-cv-03416
Order as Modified by Judge Araceli Martinez-Olguin granting (60) Stipulation Consolidating Cases in case 3:23-cv-03223-AMO. Associated Cases: 3:23-cv-03223-AMO, 3:23-cv-03416-AMO, 3:23-cv-04625-AMO...
Entry #68 in Silverman v. OpenAI, Inc., 3:23-cv-03416
Order by Judge Araceli Martinez-Olguin granting in part and denying in part 32 Motion to Dismiss. Cross-posted in 23-cv-03223. (amolc2, COURTSTAFF) (Filed on 2/12/2024) (Entered: 02/12/2024)

Kadrey v. Meta Platforms, Inc.

  • Similar suit to Silverman v. OpenAI, same parties etc.
  • Notable for a prominent dismissal of the class-action nature of the case, as the blatantly copied copyrighted works in the training data are not the works of the plaintiffs.
  • Latest [1]:
Entry #101 in Kadrey v. Meta Platforms, Inc., 3:23-cv-03417
ORDER by Magistrate Judge Thomas S. Hixson granting 100 Stipulation. (rmm2, COURT STAFF) (Filed on 4/10/2024) (Entered: 04/10/2024)
Entry #100 in Kadrey v. Meta Platforms, Inc., 3:23-cv-03417
STIPULATION WITH PROPOSED ORDER JOINT MOTION FOR ENTRY OF PROPOSED STIPULATED ORDER RE: DISCOVERY OF ELECTRONICALLY STORED INFORMATION filed by John Blase, Michael Chabon, Ta-Nehisi Coates, Junot D...
Entry #99 in Kadrey v. Meta Platforms, Inc., 3:23-cv-03417
NOTICE of Change of Address by Joseph R. Saveri (Saveri, Joseph) (Filed on 4/5/2024) (Entered: 04/05/2024)

Authors Guild v. OpenAI Inc.

Minute entry from 2024-04-25 in Authors Guild v. OpenAI Inc., 1:23-cv-08292
Order on Motion to Appear Pro Hac Vice
Entry #129 in Authors Guild v. OpenAI Inc., 1:23-cv-08292
ORDER granting (123) Motion for Michelle S. Ybarra to Appear Pro Hac Vice in case 1:23-cv-08292-SHS (HEREBY ORDERED by Judge Sidney H. Stein)(Text Only Order) Filed In Associated Cases: 1:23-cv-082...
Entry #128 in Authors Guild v. OpenAI Inc., 1:23-cv-08292
NOTICE OF APPEARANCE by Nicholas S Goldberg on behalf of OAI Corporation, LLC, OpenAI GP, LLC, OpenAI Global LLC, OpenAI Holdings, LLC, OpenAI LLC, OpenAI OpCo LLC, OpenAI, Inc., OAI Corporation LL...

Sancton v. OpenAI Inc. et al

[ Document 21]
ORDER granting #17 Motion for Alejandra Christina Salinas to Appear Pro Hac Vice (HEREBY ORDERED by Judge Sidney H. Stein)(Text Only Order) (lab)
2023-11-30 08:00:00
[ Document 20]
ORDER granting #14 Motion for Rohit Dwarka Nath to Appear Pro Hac Vice (HEREBY ORDERED by Judge Sidney H. Stein)(Text Only Order) (lab)
2023-11-30 08:00:00
[ Document 19]
ORDER granting #16 Motion for Justin Adatto Nelson to Appear Pro Hac Vice (HEREBY ORDERED by Judge Sidney H. Stein)(Text Only Order) (lab)
2023-11-30 08:00:00

Mata v. Avianca, Inc. (closed)

Note: this case is not about machine learning textually, but is included in this list because it is a notable example of gross misuse of a language model by plaintiff's counsel to submit falsified documents to the court. This led to censure of plaintiff's counsel and dismissal of the case.