Instruction-Tuning For Chart Comprehension And Reasoning

Simple item page
dc.contributor.advisorPrince, Emmanuel Hoque
dc.contributor.authorShah Mohammadi, Mehrad
dc.date.accessioned2025-04-10T10:49:19Z
dc.date.available2025-04-10T10:49:19Z
dc.date.copyright2024-11-15
dc.date.issued2025-04-10
dc.date.updated2025-04-10T10:49:18Z
dc.degree.disciplineInformation Systems and Technology
dc.degree.levelMaster's
dc.degree.nameMA - Master of Arts
dc.description.abstractCharts provide visual representations of data and are widely used for analyzing information, addressing queries, and conveying insights to others. Various chart-related downstream tasks have emerged recently, such as questionanswering and summarization. A common strategy to solve these tasks is to fine-tune various models originally trained on vision tasks language. However, such task-specific models are not capable of solving a wide range of chartrelated tasks, constraining their real-world applicability. To overcome these challenges, we introduce ChartInstruct: a novel chart-specific vision-language Instruction-following dataset comprising if instructions generated with distinct charts. We then present two distinct systems for instruction tuning on such datasets: (1) an end-to-end model (2) a pipeline model employing a two-step approach. Evaluation shows that our instruction-tuning approach supports a wide array of real-world chart comprehension and reasoning scenarios, thereby expanding the scope and applicability of our models to new kinds of tasks.
dc.identifier.urihttps://hdl.handle.net/10315/42807
dc.languageen
dc.rightsAuthor owns copyright, except where explicitly noted. Please contact the author directly with licensing requests.
dc.subject.keywordsNatural language processing
dc.subject.keywordsMultimodal learning
dc.subject.keywordsCharts
dc.subject.keywordsComputer vision
dc.subject.keywordsDeep learning
dc.titleInstruction-Tuning For Chart Comprehension And Reasoning
dc.typeElectronic Thesis or Dissertation

Files

Original bundle
Now showing 1 - 1 of 1
Thumbnail Image
Name:
Shah_Mohammadi_Mehrad_2024_MA.pdf
Size:
5.22 MB
Format:
Adobe Portable Document Format
Download
License bundle
Now showing 1 - 2 of 2
No Thumbnail Available
Name:
license.txt
Size:
1.87 KB
Format:
Plain Text
Description:
Download
No Thumbnail Available
Name:
YorkU_ETDlicense.txt
Size:
3.39 KB
Format:
Plain Text
Description:
Download

Collections

Information Systems and Technology

All items in the YorkSpace institutional repository are protected by copyright, with all rights reserved except where explicitly noted.