Fine Granularity is Critical for Intelligent Neural Network Pruning

Simple item page
dc.contributor.advisorZylberberg, Joel
dc.contributor.authorHeyman, Andrew Baldwin
dc.date.accessioned2023-12-08T14:28:09Z
dc.date.available2023-12-08T14:28:09Z
dc.date.issued2023-12-08
dc.date.updated2023-12-08T14:28:09Z
dc.degree.disciplineComputer Science
dc.degree.levelMaster's
dc.degree.nameMSc - Master of Science
dc.description.abstractNeural network pruning is a popular approach to reducing the computational costs of training and/or deploying a network, and aims to do so while minimizing accuracy loss. Pruning methods that remove individual weights (fine granularity) yield better ratios of accuracy to parameter count, while methods that preserve some or all of a network’s structure (coarser granularity, e.g. pruning channels from a CNN) take better advantage of hardware and software optimized for dense matrix computations. We compare intelligent iterative pruning using several different criteria sampled from the literature against random pruning at initialization across multiple granularities on two different image classification architectures and tasks. We find that the advantage of intelligent pruning (with any criterion) over random pruning decreases dramatically as granularity becomes coarser. Our results suggest that, compared to coarse pruning, fine pruning combined with efficient implementation of the resulting networks is a more promising direction for improving accuracy-to-cost ratios.
dc.identifier.urihttps://hdl.handle.net/10315/41637
dc.languageen
dc.rightsAuthor owns copyright, except where explicitly noted. Please contact the author directly with licensing requests.
dc.subjectComputer science
dc.subjectArtificial intelligence
dc.subject.keywordsNeural networks
dc.subject.keywordsNeural network pruning
dc.subject.keywordsSparse neural networks
dc.subject.keywordsLottery ticket hypothesis
dc.titleFine Granularity is Critical for Intelligent Neural Network Pruning
dc.typeElectronic Thesis or Dissertation

Files

Original bundle
Now showing 1 - 1 of 1
Thumbnail Image
Name:
Heyman_Alex_VB_2023_Masters.pdf
Size:
702.11 KB
Format:
Adobe Portable Document Format
Download
License bundle
Now showing 1 - 2 of 2
No Thumbnail Available
Name:
license.txt
Size:
1.87 KB
Format:
Plain Text
Description:
Download
No Thumbnail Available
Name:
YorkU_ETDlicense.txt
Size:
3.39 KB
Format:
Plain Text
Description:
Download

Collections

Computer Science

All items in the YorkSpace institutional repository are protected by copyright, with all rights reserved except where explicitly noted.