Visualizations and text are commonly used together in various applications, ranging from communicative documents to interactive tools for analyzing and exploring data. However, much about the relationship between visualization and text remains unexplored. This talk will focus on three problems related to communicating the connections between the two representations and ways to surface the connections to address the problem.

I will begin by discussing how readers integrate information between visualizations and text when the two representations emphasize different aspects of the underlying data and how we can help authors write chart-caption pairs with matched emphasis. Then, I will shift focus to the cognitive burden arising from trying to link information in visualizations and text and present an interactive document reader that extends existing PDF documents based on automatically extracted references between visualizations and text. Next, I will discuss transparency in chart question answering systems and present a method of generating visual explanations that significantly improve transparency and achieve levels of trust close to human-generated explanations. Finally, I will end with a note on potential future directions of related research.

Dae Hyun Kim is a postdoctoral researcher at KAIST Interaction Lab, working with Professor Juho Kim. His work lies in the intersection of Data Visualization, Human-Computer Interaction and Natural Language Processing. He has specifically focused on understanding the relationship between data visualizations and text in communication, and building tools to facilitate the process. He has recently been exploring various ways of involving large language models along this direction as well as communication in other contexts, such as metaverse and AI application planning. He has published his work in Computer Science venues such as CHI, UIST and VIS.

Dae Hyun received a PhD in Computer Science from Stanford University and a Bachelor of Science in Computer Science from California Institute of Technology. He is a recipient of a Samsung Scholarship for his PhD program and a Presidential Science Scholarship for his Bachelor of Science. For more information, see his website at dhkim16.github.io.