Aijun AnDehnen, Nicholas Alexander2025-07-232025-07-232025-04-152025-07-23https://hdl.handle.net/10315/43017This thesis addresses the semantic gap between visual perception and functional significance of symbols used in road vehicles. It presents a novel approach that enables users to identify and understand automotive symbols by describing what they visually perceive, translating visual descriptions into practical implications. A system combining a property graph representation of visual components and semantic relationships with a language model-powered natural language interface is developed. This method explicitly models relationships between visual elements and interpretations, differing from end-to-end vision-language models. Evaluations, using automated metrics and human assessment, demonstrate performance exceeding baseline large language models, with a BERTscore F1 of 0.765, compared to the best baseline's 0.597. Analysis of visual symbol queries reveals human description tendencies, favoring intuitive analogies and basic shapes. Contributions include a symbol decomposition methodology, an advanced property graph schema, natural language query processing, and evidence supporting structured knowledge representation for symbol description-recognition, applicable beyond automotive interfaces.Author owns copyright, except where explicitly noted. Please contact the author directly with licensing requests.Computer scienceElectronic Thesis or Dissertation2025-07-23Automotive symbolsSymbol recognitionSemantic gapProperty graphsKnowledge representationLanguage modelsNatural language interfacesVisual descriptionSymbol decompositionSemantic interpretation