Neural Networks

From jWiki
Jump to navigationJump to search

Herein lie some of my thoughts and resources about neural networks. Because I work for a company that builds models for computer vision, I have a bit of a professional bias towards image models, but I have tried to represent my knowledge/opinions about a broader range of subjects here.

What do you think about generative "AI"?

tl;dr - mostly dancing bearware, some novel uses in responsibility laundering

Resources

Image models

Text models

For code

For everything else

  • Washington Post coverage of the data contained in the 'C4' dataset and how it influences the training of popular large models. Also allows users to check if arbitrary URLs are part of the dataset. (NOTE: C4 is not the only source of training text for the models being discussed, and the authors aren't doing a great job highlighting that, but it should still be pretty representative)
  • How well does ChatGPT speak Japanese? - an April 2023 evaluation of GPT-3.5 and GPT-4 performance on Japanese language assessments. Also includes an interesting comparison of the number of tokens required to represent the "Lord's Prayer" in multiple languages. I found the results of the latter particularly surprising.

Misc.

  • I gave a talk on the fundamentals of neural networks to Boston Python in March 2023
  • 3blue1brown has an excellent series of lessons about the fundamentals of neural networks. Particularly interesting to me is the lesson on backpropagation for its excellent visualization of the process of adjusting neural network weights.

Dumping ground

These references are totally unclassified

Writings by others

Academic works

Non-academic works

Lawsuits

The legal status of generative models and their implications for intellectual property in the US is something I'm trying to keep an eye on. The cases given below are of particular interest to me.

The New York Times Company v. MICROSOFT CORPORATION

Entry #56 in The New York Times Company v. Microsoft Corporation, 1:23-cv-11195
Appear Pro Hac Vice
Entry #55 in The New York Times Company v. Microsoft Corporation, 1:23-cv-11195
RULE 7.1 CORPORATE DISCLOSURE STATEMENT. Identifying Other Affiliate OpenAI GP, L.L.C., Other Affiliate OpenAI Global, L.L.C., Other Affiliate OAI Corporation, L.L.C., Other Affiliate OpenAI Holdin...
Entry #54 in The New York Times Company v. Microsoft Corporation, 1:23-cv-11195
LETTER MOTION for Oral Argument addressed to Judge Sidney H. Stein from Andrew M. Gass dated February 26, 2024. Document filed by OAI Corporation, LLC, OpenAI GP, LLC, OpenAI Global LLC, OpenAI Hol...

Andersen v. Stability AI Ltd.

Entry #172 in Andersen v. Stability AI Ltd., 3:23-cv-00201
Order by Judge William H. Orrick granting 168 Motion for Pro Hac Vice by Amira Perryman. (jmd, COURT STAFF) (Filed on 2/12/2024) (Entered: 02/12/2024)
Entry #171 in Andersen v. Stability AI Ltd., 3:23-cv-00201
Order by Judge William H. Orrick granting 167 Motion for Pro Hac Vice by Vishakha S. Joshi. (jmd, COURT STAFF) (Filed on 2/12/2024) (Entered: 02/12/2024)
Entry #170 in Andersen v. Stability AI Ltd., 3:23-cv-00201
Request for Judicial Notice re 169 MOTION to Dismiss Correcting Bar Number of A. Dunning (212047) in ECF 160 and ECF 161 filed byMidjourney, Inc.. (Attachments: # 1 Declaration of Angela L. Dunning...


Getty Images (US), Inc. v. Stability AI, Inc.

Entry #38 in Getty Images (US), Inc. v. Stability AI, Inc., 1:23-cv-00135
NOTICE to Take Deposition of Peter O'Donoghue on February 22, 2024 filed by Getty Images (US), Inc..(Vrana, Robert) (Entered: 02/13/2024)
Entry #37 in Getty Images (US), Inc. v. Stability AI, Inc., 1:23-cv-00135
NOTICE OF SERVICE of (1) Stability AI Ltd.'s Second Supplemental Objections and Responses to Plaintiff's Jurisdictional Interrogatories Nos. 2 and 12; and (2) Stability AI, Inc.'s Secon...
Entry #36 in Getty Images (US), Inc. v. Stability AI, Inc., 1:23-cv-00135
ORAL ORDER: Defendants' Motion to Dismiss or Transfer (D.I. 16 ) is DISMISSED without prejudice to re-file after the conclusion of jurisdictional discovery. Ordered by Judge Jennifer L. Hall on...

Doe 1 v. GitHub, Inc.

Entry #221 in DOE 1 v. GitHub, Inc., 4:22-cv-06823
Notice (Other)
Entry #220 in DOE 1 v. GitHub, Inc., 4:22-cv-06823
Pro Hac Vice
Entry #219 in DOE 1 v. GitHub, Inc., 4:22-cv-06823
MOTION to Dismiss Second Amended Complaint filed by OAI CORPORATION,, OPENAI GLOBAL, LLC, OPENAI HOLDCO, LLC, OPENAI HOLDINGS, LLC, OPENAI INVESTMENT, OPENAI INVESTMENT LLC, OPENAI STARTUP FUND SPV...

Silverman v. OpenAI, Inc.

Entry #70 in Silverman v. OpenAI, Inc., 3:23-cv-03416
PRETRIAL ORDER as Modified. Signed by Judge Araceli Martinez-Olguin on 2/16/2024. (ads, COURT STAFF) (Filed on 2/16/2024) (Entered: 02/16/2024)
Entry #69 in Silverman v. OpenAI, Inc., 3:23-cv-03416
Order as Modified by Judge Araceli Martinez-Olguin granting (60) Stipulation Consolidating Cases in case 3:23-cv-03223-AMO. Associated Cases: 3:23-cv-03223-AMO, 3:23-cv-03416-AMO, 3:23-cv-04625-AMO...
Entry #68 in Silverman v. OpenAI, Inc., 3:23-cv-03416
Order by Judge Araceli Martinez-Olguin granting in part and denying in part 32 Motion to Dismiss. Cross-posted in 23-cv-03223. (amolc2, COURTSTAFF) (Filed on 2/12/2024) (Entered: 02/12/2024)

Kadrey v. Meta Platforms, Inc.

  • Similar suit to Silverman v. OpenAI, same parties etc.
  • Notable for a prominent dismissal of the class-action nature of the case, as the blatantly copied copyrighted works in the training data are not the works of the plaintiffs.
  • Latest [1]:
Entry #97 in Kadrey v. Meta Platforms, Inc., 3:23-cv-03417
NOTICE of Appearance by Angela Dunning (Dunning, Angela) (Filed on 2/20/2024) (Entered: 02/20/2024)
Entry #96 in Kadrey v. Meta Platforms, Inc., 3:23-cv-03417
Transcript of Proceedings held on 01/25/24, before Judge Vince Chhabria. Court Reporter/Transcriber Echo Reporting, Inc., telephone number echoreporting@yahoo.com. Tape Number: 1:47 - 2:09. Per Gen...
Entry #95 in Kadrey v. Meta Platforms, Inc., 3:23-cv-03417
TRANSCRIPT ORDER for proceedings held on 01/25/2024 before Judge Vince Chhabria by Meta Platforms, Inc., for Recorded Proceeding - San Francisco. (Ghajar, Bobby) (Filed on 1/29/2024) (Entered: 01/2...

Authors Guild v. OpenAI Inc.

Sancton v. OpenAI Inc. et al

[ Document 21]
ORDER granting #17 Motion for Alejandra Christina Salinas to Appear Pro Hac Vice (HEREBY ORDERED by Judge Sidney H. Stein)(Text Only Order) (lab)
2023-11-30 08:00:00
[ Document 20]
ORDER granting #14 Motion for Rohit Dwarka Nath to Appear Pro Hac Vice (HEREBY ORDERED by Judge Sidney H. Stein)(Text Only Order) (lab)
2023-11-30 08:00:00
[ Document 19]
ORDER granting #16 Motion for Justin Adatto Nelson to Appear Pro Hac Vice (HEREBY ORDERED by Judge Sidney H. Stein)(Text Only Order) (lab)
2023-11-30 08:00:00

Mata v. Avianca, Inc. (closed)

Note: this case is not about machine learning textually, but is included in this list because it is a notable example of gross misuse of a language model by plaintiff's counsel to submit falsified documents to the court. This led to censure of plaintiff's counsel and dismissal of the case.