Precision Recall Cover: A Method to Assess Generative Models

Simple item page
dc.contributor.advisorUrner, Ruth
dc.contributor.authorCheema, Fasil Tariq
dc.date.accessioned2023-12-08T14:37:44Z
dc.date.available2023-12-08T14:37:44Z
dc.date.issued2023-12-08
dc.date.updated2023-12-08T14:37:43Z
dc.degree.disciplineComputer Science
dc.degree.levelMaster's
dc.degree.nameMSc - Master of Science
dc.description.abstractGenerative modelling has seen enormous practical advances over the past few years from LLMs like ChatGPT to image generation. However, evaluating the quality of a generative system is often still based on subjective human inspection. To overcome this, very recently, the research community has turned to exploring formal evaluation metrics and methods. In this work, we propose a novel evaluation method based on a two-way nearest neighbor test. We define a new measure of mutual coverage for two probability distributions. From this, we derive an empirical analogue and show analytically that it exhibits favorable theoretical properties while it is also straightforward to compute. We show that, while algorithmically simple, our derived method is also statistically sound. We complement our analysis with a systematic experimental evaluation and comparison to other recently proposed measures. Using a wide array of experiments, we demonstrate our algorithm’s strengths over other existing methods and confirm our results from the theoretical analysis.
dc.identifier.urihttps://hdl.handle.net/10315/41705
dc.languageen
dc.rightsAuthor owns copyright, except where explicitly noted. Please contact the author directly with licensing requests.
dc.subjectArtificial intelligence
dc.subjectComputer science
dc.subject.keywordsGenerative models
dc.subject.keywordsMachine learning
dc.subject.keywordsStatistical learning theory
dc.subject.keywordsArtificial intelligence
dc.subject.keywordsEvaluation methods
dc.subject.keywordsGANs
dc.titlePrecision Recall Cover: A Method to Assess Generative Models
dc.typeElectronic Thesis or Dissertation

Files

Original bundle
Now showing 1 - 1 of 1
Thumbnail Image
Name:
ThesisSubmitted.pdf
Size:
4.42 MB
Format:
Adobe Portable Document Format
Download
License bundle
Now showing 1 - 2 of 2
No Thumbnail Available
Name:
license.txt
Size:
1.87 KB
Format:
Plain Text
Description:
Download
No Thumbnail Available
Name:
YorkU_ETDlicense.txt
Size:
3.39 KB
Format:
Plain Text
Description:
Download

Collections

Computer Science

All items in the YorkSpace institutional repository are protected by copyright, with all rights reserved except where explicitly noted.