Neural Networks

From jWiki
Revision as of 13:14, 21 May 2024 by Snoopj (talk | contribs) (→‎Non-academic works: Add reference to original SolidGoldMagikarp posting)
(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)
Jump to navigationJump to search

Herein lie some of my thoughts and resources about neural networks. Because I work for a company that builds models for computer vision, I have a bit of a professional bias towards image models, but I have tried to represent my knowledge/opinions about a broader range of subjects here.

What do you think about generative "AI"?

tl;dr - mostly dancing bearware, some novel uses in responsibility laundering

Resources

Image models

Text models

For code

For everything else

  • Washington Post coverage of the data contained in the 'C4' dataset and how it influences the training of popular large models. Also allows users to check if arbitrary URLs are part of the dataset. (NOTE: C4 is not the only source of training text for the models being discussed, and the authors aren't doing a great job highlighting that, but it should still be pretty representative)
  • How well does ChatGPT speak Japanese? - an April 2023 evaluation of GPT-3.5 and GPT-4 performance on Japanese language assessments. Also includes an interesting comparison of the number of tokens required to represent the "Lord's Prayer" in multiple languages. I found the results of the latter particularly surprising.

Misc.

  • I gave a talk on the fundamentals of neural networks to Boston Python in March 2023
  • 3blue1brown has an excellent series of lessons about the fundamentals of neural networks. Particularly interesting to me is the lesson on backpropagation for its excellent visualization of the process of adjusting neural network weights.

Dumping ground

These references are totally unclassified

Writings by others

Academic works

Non-academic works

Lawsuits

The legal status of generative models and their implications for intellectual property in the US is something I'm trying to keep an eye on. The cases given below are of particular interest to me.

The New York Times Company v. MICROSOFT CORPORATION

Entry #160 in The New York Times Company v. Microsoft Corporation, 1:23-cv-11195
NOTICE of Supplemental Authority re: 51 MOTION to Dismiss .. Document filed by OAI Corporation, LLC, OpenAI GP, LLC, OpenAI Global LLC, OpenAI Holdings, LLC, OpenAI LLC, OpenAI LP, OpenAI OpCo LLC,...
Minute entry from 2024-07-12 in The New York Times Company v. Microsoft Corporation, 1:23-cv-11195
>>>NOTICE REGARDING PRO HAC VICE MOTION. Regarding Document No. 159 MOTION for Elizabeth M.C. Scheibel to Appear Pro Hac Vice . Filing fee $ 200.00, receipt number ANYSDC-29594436. Motion...
Minute entry from 2024-07-12 in The New York Times Company v. Microsoft Corporation, 1:23-cv-11195
>>>NOTICE REGARDING PRO HAC VICE MOTION. Regarding Document No. 158 MOTION for Carrie A. Beyer to Appear Pro Hac Vice . Filing fee $ 200.00, receipt number ANYSDC-29594410. Motion and supp...

Andersen v. Stability AI Ltd.

Entry #220 in Andersen v. Stability AI Ltd., 3:23-cv-00201
STATEMENT OF RECENT DECISION pursuant to Civil Local Rule 7-3.d filed byMidjourney, Inc.. (Attachments: # 1 Exhibit 1)(Dunning, Angela) (Filed on 7/17/2024) (Entered: 07/17/2024)
Entry #219 in Andersen v. Stability AI Ltd., 3:23-cv-00201
NOTICE of Withdraw of Counsel by Runway AI, Inc. (Malave, Celina) (Filed on 7/12/2024) Modified on 7/15/2024 (kmm2, COURT STAFF). (Entered: 07/12/2024)
Entry #218 in Andersen v. Stability AI Ltd., 3:23-cv-00201
TRANSCRIPT ORDER for proceedings held on 5/8/2024 before Judge William H. Orrick for Court Reporter Beth Krupa (Krupa, Beth) (Filed on 6/21/2024) (Entered: 06/21/2024)


Getty Images (US), Inc. v. Stability AI, Inc.

Minute entry from 2024-07-22 in Getty Images (US), Inc. v. Stability AI, Inc., 1:23-cv-00135
SO ORDERED
Minute entry from 2024-07-22 in Getty Images (US), Inc. v. Stability AI, Inc., 1:23-cv-00135
DEFICIENCY NOTICE issued by the Court to Stability AI US Services Corporation: Pursuant to Fed. R. Civ. P. 7.1 (b)(1), A party must: (1) file the disclosure statement with its first appearance, ple...
Minute entry from 2024-07-22 in Getty Images (US), Inc. v. Stability AI, Inc., 1:23-cv-00135
SO ORDERED, re 43 Stipulation and Order to Extend Time for Defendants to respond to the Second Amended Complaint to July 29, 2024. (*Reset Answer Deadlines: Stability AI US Services Corporation ans...

Doe 1 v. GitHub, Inc.

Entry #267 in DOE 1 v. GitHub, Inc., 4:22-cv-06823
ANSWER to Amended Complaint by GitHub, Inc.. (Hurst, Annette) (Filed on 7/22/2024) (Entered: 07/22/2024)
Entry #266 in DOE 1 v. GitHub, Inc., 4:22-cv-06823
ANSWER to Amended Complaint by Microsoft Corporation. (Hurst, Annette) (Filed on 7/22/2024) (Entered: 07/22/2024)
Entry #265 in DOE 1 v. GitHub, Inc., 4:22-cv-06823
ANSWER to Second Amended Complaint by OAI CORPORATION,, OPENAI GLOBAL, LLC, OPENAI HOLDCO, LLC, OPENAI HOLDINGS, LLC, OPENAI STARTUP FUND SPV GP I, L.L.C., OPENAI STARTUP FUND SPV I, L.P, OPENAI, L...

Silverman v. OpenAI, Inc.

Entry #70 in Silverman v. OpenAI, Inc., 3:23-cv-03416
PRETRIAL ORDER as Modified. Signed by Judge Araceli Martinez-Olguin on 2/16/2024. (ads, COURT STAFF) (Filed on 2/16/2024) (Entered: 02/16/2024)
Entry #69 in Silverman v. OpenAI, Inc., 3:23-cv-03416
Order as Modified by Judge Araceli Martinez-Olguin granting (60) Stipulation Consolidating Cases in case 3:23-cv-03223-AMO. Associated Cases: 3:23-cv-03223-AMO, 3:23-cv-03416-AMO, 3:23-cv-04625-AMO...
Entry #68 in Silverman v. OpenAI, Inc., 3:23-cv-03416
Order by Judge Araceli Martinez-Olguin granting in part and denying in part 32 Motion to Dismiss. Cross-posted in 23-cv-03223. (amolc2, COURTSTAFF) (Filed on 2/12/2024) (Entered: 02/12/2024)

Kadrey v. Meta Platforms, Inc.

  • Similar suit to Silverman v. OpenAI, same parties etc.
  • Notable for a prominent dismissal of the class-action nature of the case, as the blatantly copied copyrighted works in the training data are not the works of the plaintiffs.
  • Latest [1]:
Entry #107 in Kadrey v. Meta Platforms, Inc., 3:23-cv-03417
Order by Judge Vince Chhabria granting 106 Stipulation RE: VOLUNTARY DISMISSAL AND CONSOLIDATION. (bxs, COURT STAFF) (Filed on 7/5/2024) (Entered: 07/05/2024)
Entry #106 in Kadrey v. Meta Platforms, Inc., 3:23-cv-03417
STIPULATION WITH PROPOSED ORDER filed by Michael Chabon. (Sweatman, Alexander) (Filed on 7/1/2024) (Entered: 07/01/2024)
Entry #105 in Kadrey v. Meta Platforms, Inc., 3:23-cv-03417
Joint Discovery Letter Brief filed by Meta Platforms, Inc.. (Attachments: # 1 Declaration of T. Dettmers, # 2 Declaration of L. Zettlemoyer, # 3 Declaration of K. Hartnett Declaration (redacted), #...

Authors Guild v. OpenAI Inc.

Entry #169 in Authors Guild v. OpenAI Inc., 1:23-cv-08292
LETTER MOTION to Compel Microsoft to produce [Redacted] addressed to Judge Sidney H. Stein from Rachel Geman, Rohit Nath, and Scott J. Sholder dated July 22, 2024. Document filed by Authors...
Entry #168 in Authors Guild v. OpenAI Inc., 1:23-cv-08292
***EX-PARTE*** LETTER MOTION to Compel Microsoft to produce addressed to Judge Sidney H. Stein from Rachel Geman, Rohit Nath, and Scott J. Sholder dated July 22, 2024. Document filed by Authors Gui...
Entry #167 in Authors Guild v. OpenAI Inc., 1:23-cv-08292
MOTION to Seal Letter Motion to Compel. Document filed by Authors Guild, David Baldacci, Mary Bly, Sylvia Day, Jonathan Franzen, John Grisham, Elin Hilderbrand, Christina Baker Kline, Victor LaVall...

Sancton v. OpenAI Inc. et al

[ Document 21]
ORDER granting #17 Motion for Alejandra Christina Salinas to Appear Pro Hac Vice (HEREBY ORDERED by Judge Sidney H. Stein)(Text Only Order) (lab)
2023-11-30 08:00:00
[ Document 20]
ORDER granting #14 Motion for Rohit Dwarka Nath to Appear Pro Hac Vice (HEREBY ORDERED by Judge Sidney H. Stein)(Text Only Order) (lab)
2023-11-30 08:00:00
[ Document 19]
ORDER granting #16 Motion for Justin Adatto Nelson to Appear Pro Hac Vice (HEREBY ORDERED by Judge Sidney H. Stein)(Text Only Order) (lab)
2023-11-30 08:00:00

Mata v. Avianca, Inc. (closed)

Note: this case is not about machine learning textually, but is included in this list because it is a notable example of gross misuse of a language model by plaintiff's counsel to submit falsified documents to the court. This led to censure of plaintiff's counsel and dismissal of the case.