Fixed Size Ordinally-Forgetting Encoding and its Applications

Simple item page
dc.contributor.advisorJiang, Hui
dc.creatorXu, Mingbin
dc.date.accessioned2018-03-01T14:10:56Z
dc.date.available2018-03-01T14:10:56Z
dc.date.copyright2017-09-14
dc.date.issued2018-03-01
dc.date.updated2018-03-01T14:10:56Z
dc.degree.disciplineComputer Science
dc.degree.levelMaster's
dc.degree.nameMSc - Master of Science
dc.description.abstractIn this thesis, we propose the new Fixed-size Ordinally-Forgetting Encoding (FOFE) method, which can almost uniquely encode any variable-length sequence of words into a fixed-size representation. FOFE can model the word order in a sequence using a simple ordinally-forgetting mechanism according to the positions of words. We address two fundamental problems in natural language processing, namely, Language Modeling (LM) and Named Entity Recognition (NER). We have applied FOFE to FeedForward Neural Network Language Models (FFNN-LMs). Experimental results have shown that without using any recurrent feedbacks, FOFE-FFNN-LMs significantly outperform not only the standard fixed-input FFNN-LMs but also some popular Recurrent Neural Network Language Models (RNN-LMs). Instead of treating NER as a sequence labeling problem, we propose a new local detection approach, which relies on FOFE to fully encode each sentence fragment and its left/right contexts into a fixed-size representation. This local detection approach has shown many advantages over the traditional sequence labeling methods. Our method has yielded pretty strong performance in all tasks we have examined.
dc.identifier.urihttp://hdl.handle.net/10315/34398
dc.language.isoen
dc.rightsAuthor owns copyright, except where explicitly noted. Please contact the author directly with licensing requests.
dc.subjectComputer science
dc.subject.keywordsFOFE
dc.subject.keywordsDeep learning
dc.subject.keywordsNatural language processing (NLP)
dc.subject.keywordsLanguage model (LM)
dc.subject.keywordsNamed entity recognition (NER)
dc.titleFixed Size Ordinally-Forgetting Encoding and its Applications
dc.typeElectronic Thesis or Dissertation

Files

Original bundle
Now showing 1 - 1 of 1
Thumbnail Image
Name:
Xu_Mingbin_2017_Masters.pdf
Size:
1.03 MB
Format:
Adobe Portable Document Format
Download
License bundle
Now showing 1 - 2 of 2
No Thumbnail Available
Name:
license.txt
Size:
1.83 KB
Format:
Plain Text
Description:
Download
No Thumbnail Available
Name:
YorkU_ETDlicense.txt
Size:
3.38 KB
Format:
Plain Text
Description:
Download

Collections

Computer Science and Engineering

All items in the YorkSpace institutional repository are protected by copyright, with all rights reserved except where explicitly noted.