Neural Networks

From jWiki
Revision as of 13:14, 21 May 2024 by Snoopj (talk | contribs) (→‎Non-academic works: Add reference to original SolidGoldMagikarp posting)
(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)
Jump to navigationJump to search

Herein lie some of my thoughts and resources about neural networks. Because I work for a company that builds models for computer vision, I have a bit of a professional bias towards image models, but I have tried to represent my knowledge/opinions about a broader range of subjects here.

What do you think about generative "AI"?

tl;dr - mostly dancing bearware, some novel uses in responsibility laundering

Resources

Image models

Text models

For code

For everything else

  • Washington Post coverage of the data contained in the 'C4' dataset and how it influences the training of popular large models. Also allows users to check if arbitrary URLs are part of the dataset. (NOTE: C4 is not the only source of training text for the models being discussed, and the authors aren't doing a great job highlighting that, but it should still be pretty representative)
  • How well does ChatGPT speak Japanese? - an April 2023 evaluation of GPT-3.5 and GPT-4 performance on Japanese language assessments. Also includes an interesting comparison of the number of tokens required to represent the "Lord's Prayer" in multiple languages. I found the results of the latter particularly surprising.

Misc.

  • I gave a talk on the fundamentals of neural networks to Boston Python in March 2023
  • 3blue1brown has an excellent series of lessons about the fundamentals of neural networks. Particularly interesting to me is the lesson on backpropagation for its excellent visualization of the process of adjusting neural network weights.

Dumping ground

These references are totally unclassified

Writings by others

Academic works

Non-academic works

Lawsuits

The legal status of generative models and their implications for intellectual property in the US is something I'm trying to keep an eye on. The cases given below are of particular interest to me.

The New York Times Company v. MICROSOFT CORPORATION

Entry #160 in The New York Times Company v. Microsoft Corporation, 1:23-cv-11195
NOTICE of Supplemental Authority re: 51 MOTION to Dismiss .. Document filed by OAI Corporation, LLC, OpenAI GP, LLC, OpenAI Global LLC, OpenAI Holdings, LLC, OpenAI LLC, OpenAI LP, OpenAI OpCo LLC,...
Minute entry from 2024-07-12 in The New York Times Company v. Microsoft Corporation, 1:23-cv-11195
>>>NOTICE REGARDING PRO HAC VICE MOTION. Regarding Document No. 159 MOTION for Elizabeth M.C. Scheibel to Appear Pro Hac Vice . Filing fee $ 200.00, receipt number ANYSDC-29594436. Motion...
Minute entry from 2024-07-12 in The New York Times Company v. Microsoft Corporation, 1:23-cv-11195
>>>NOTICE REGARDING PRO HAC VICE MOTION. Regarding Document No. 158 MOTION for Carrie A. Beyer to Appear Pro Hac Vice . Filing fee $ 200.00, receipt number ANYSDC-29594410. Motion and supp...

Andersen v. Stability AI Ltd.

Entry #219 in Andersen v. Stability AI Ltd., 3:23-cv-00201
Notice (Other)
Entry #218 in Andersen v. Stability AI Ltd., 3:23-cv-00201
Transcript Order
Entry #217 in Andersen v. Stability AI Ltd., 3:23-cv-00201
TRANSCRIPT ORDER for proceedings held on 5/8/2024 before Judge William H. Orrick for Court Reporter Beth Krupa (Krupa, Beth) (Filed on 6/18/2024) (Entered: 06/18/2024)


Getty Images (US), Inc. v. Stability AI, Inc.

Entry #42 in Getty Images (US), Inc. v. Stability AI, Inc., 1:23-cv-00135
SECOND AMENDED COMPLAINT against Stability AI, Inc., Stability AI, Ltd., Stability AI US Services Corporation - filed by Getty Images (US), Inc. (Attachments: # 1 Exhibits A-B)(ceg) (Entered: 07/08...
Entry #41 in Getty Images (US), Inc. v. Stability AI, Inc., 1:23-cv-00135
ORDER granting 40 Unopposed Motion for Leave to File Second Amended Complaint. Signed by Judge Jennifer L. Hall on 7/8/2024. (ceg) (Entered: 07/08/2024)
Entry #40 in Getty Images (US), Inc. v. Stability AI, Inc., 1:23-cv-00135
Unopposed MOTION for Leave to File Second Amended Complaint - filed by Getty Images (US), Inc.. (Attachments: # 1 Exhibit A - Proposed Second Amended Complaint, # 2 Exhibit B - Redline)(Mercer, Tam...

Doe 1 v. GitHub, Inc.

Minute entry from 2024-07-12 in DOE 1 v. GitHub, Inc., 4:22-cv-06823
Reset Hearing. Updated Joint Case Management Statement due by 9/27/2024. Further Case Management Conference set for 10/4/2024 at 01:30 PM - Videoconference Only. This proceeding will be held via a...
Minute entry from 2024-07-12 in DOE 1 v. GitHub, Inc., 4:22-cv-06823
Order
Minute entry from 2024-07-12 in DOE 1 v. GitHub, Inc., 4:22-cv-06823
Set Deadlines/Hearings AND ~Util - Teleconference Zoom

Silverman v. OpenAI, Inc.

Entry #70 in Silverman v. OpenAI, Inc., 3:23-cv-03416
PRETRIAL ORDER as Modified. Signed by Judge Araceli Martinez-Olguin on 2/16/2024. (ads, COURT STAFF) (Filed on 2/16/2024) (Entered: 02/16/2024)
Entry #69 in Silverman v. OpenAI, Inc., 3:23-cv-03416
Order as Modified by Judge Araceli Martinez-Olguin granting (60) Stipulation Consolidating Cases in case 3:23-cv-03223-AMO. Associated Cases: 3:23-cv-03223-AMO, 3:23-cv-03416-AMO, 3:23-cv-04625-AMO...
Entry #68 in Silverman v. OpenAI, Inc., 3:23-cv-03416
Order by Judge Araceli Martinez-Olguin granting in part and denying in part 32 Motion to Dismiss. Cross-posted in 23-cv-03223. (amolc2, COURTSTAFF) (Filed on 2/12/2024) (Entered: 02/12/2024)

Kadrey v. Meta Platforms, Inc.

  • Similar suit to Silverman v. OpenAI, same parties etc.
  • Notable for a prominent dismissal of the class-action nature of the case, as the blatantly copied copyrighted works in the training data are not the works of the plaintiffs.
  • Latest [1]:
Entry #107 in Kadrey v. Meta Platforms, Inc., 3:23-cv-03417
Order by Judge Vince Chhabria granting 106 Stipulation RE: VOLUNTARY DISMISSAL AND CONSOLIDATION. (bxs, COURT STAFF) (Filed on 7/5/2024) (Entered: 07/05/2024)
Entry #106 in Kadrey v. Meta Platforms, Inc., 3:23-cv-03417
STIPULATION WITH PROPOSED ORDER filed by Michael Chabon. (Sweatman, Alexander) (Filed on 7/1/2024) (Entered: 07/01/2024)
Entry #105 in Kadrey v. Meta Platforms, Inc., 3:23-cv-03417
Joint Discovery Letter Brief filed by Meta Platforms, Inc.. (Attachments: # 1 Declaration of T. Dettmers, # 2 Declaration of L. Zettlemoyer, # 3 Declaration of K. Hartnett Declaration (redacted), #...

Authors Guild v. OpenAI Inc.

Entry #163 in Authors Guild v. OpenAI Inc., 1:23-cv-08292
NOTICE OF APPEARANCE by Elizabeth Mead Cavert Scheibel on behalf of Microsoft Corporation, Microsoft Corporation. Filed In Associated Cases: 1:23-cv-08292-SHS, 1:23-cv-10211-SHS, 1:24-cv-00084-SHS....
Entry #162 in Authors Guild v. OpenAI Inc., 1:23-cv-08292
NOTICE OF APPEARANCE by Carrie A. Beyer on behalf of Microsoft Corporation, Microsoft Corporation. Filed In Associated Cases: 1:23-cv-08292-SHS, 1:23-cv-10211-SHS, 1:24-cv-00084-SHS.(Beyer, Carrie)...
Entry #161 in Authors Guild v. OpenAI Inc., 1:23-cv-08292
NOTICE OF APPEARANCE by Kirstin Stoll-DeBell on behalf of Microsoft Corporation, Microsoft Corporation. Filed In Associated Cases: 1:23-cv-08292-SHS, 1:23-cv-10211-SHS, 1:24-cv-00084-SHS.(Stoll-DeB...

Sancton v. OpenAI Inc. et al

[ Document 21]
ORDER granting #17 Motion for Alejandra Christina Salinas to Appear Pro Hac Vice (HEREBY ORDERED by Judge Sidney H. Stein)(Text Only Order) (lab)
2023-11-30 08:00:00
[ Document 20]
ORDER granting #14 Motion for Rohit Dwarka Nath to Appear Pro Hac Vice (HEREBY ORDERED by Judge Sidney H. Stein)(Text Only Order) (lab)
2023-11-30 08:00:00
[ Document 19]
ORDER granting #16 Motion for Justin Adatto Nelson to Appear Pro Hac Vice (HEREBY ORDERED by Judge Sidney H. Stein)(Text Only Order) (lab)
2023-11-30 08:00:00

Mata v. Avianca, Inc. (closed)

Note: this case is not about machine learning textually, but is included in this list because it is a notable example of gross misuse of a language model by plaintiff's counsel to submit falsified documents to the court. This led to censure of plaintiff's counsel and dismissal of the case.