Dual Fixed-Size Ordinally Forgetting Encoding (FOFE) For Natural Language Processing

dc.contributor.advisorJiang, Hui
dc.contributor.authorWatcharawittayakul, Sedtawut
dc.date.accessioned2020-05-11T12:53:27Z
dc.date.available2020-05-11T12:53:27Z
dc.date.copyright2019-11
dc.date.issued2020-05-11
dc.date.updated2020-05-11T12:53:27Z
dc.degree.disciplineComputer Science
dc.degree.levelMaster's
dc.degree.nameMSc - Master of Science
dc.description.abstractIn this thesis, we propose a new approach to employ fixed-size ordinally-forgetting encoding (FOFE) on Natural Language Processing (NLP) tasks, called dual-FOFE. The main idea behind dual-FOFE is that it allows the encoding to be done with two different forgetting factors; this would resolve the original FOFEs dilemma in choosing between the benefits offered by having either small or large values for its single forgetting factor. For this research, we have conducted our experiments on two prominent NLP tasks, namely, language modelling and machine reading comprehension. Our experiment results shown that the dual-FOFE provide a definite improvement over the original FOFE by approximately 11% in perplexity (PPL) for language modelling task and 8% in Exact Match (EM) score for machine reading comprehension task.
dc.identifier.urihttps://hdl.handle.net/10315/37464
dc.languageen
dc.rightsAuthor owns copyright, except where explicitly noted. Please contact the author directly with licensing requests.
dc.subjectComputer science
dc.subject.keywordsNatural language processing
dc.subject.keywordsDeep learning
dc.titleDual Fixed-Size Ordinally Forgetting Encoding (FOFE) For Natural Language Processing
dc.typeElectronic Thesis or Dissertation

Files

Original bundle
Now showing 1 - 1 of 1
Loading...
Thumbnail Image
Name:
Watcharawittayakul_Sedtawut_2019_Masters.pdf
Size:
15.73 MB
Format:
Adobe Portable Document Format
License bundle
Now showing 1 - 2 of 2
No Thumbnail Available
Name:
license.txt
Size:
1.83 KB
Format:
Plain Text
Description:
No Thumbnail Available
Name:
YorkU_ETDlicense.txt
Size:
3.36 KB
Format:
Plain Text
Description: