Deep Learning and the Future of Search: Objects, Apps, and Beyond
Web search has matured over the past 20 years, but it still leans on the traditions of textual document retrieval. You give me a textual query, and I’ll give you some textual documents that are related to it. Most audio and visual search technologies, just 10 years younger, rely on hand-engineered feature extraction systems to replace the role of words in documents. Recent sweeping advances in perceptual artificial intelligence are now making it possible for search systems to radically extend the space of queries and documents. We can now search by photograph or by voice to find physical objects or moments reachable within interactive media.
In this talk, you will learn about the key concepts behind cutting-edge, cross-modality search engines. You will learn how to train semantic embeddings which map items onto vectors in an abstract space where distances and directions are significant. You will learn strategies for turning piles of data into a useful index that can be efficiently queried. Examples shown will draw on interactive media indexing and retrieval systems produced by the Design Reasoning Lab at UC Santa Cruz which map out the space of interesting content within apps and games by direct interacting with them.
Adam Smith is an Assistant Professor of Computational Media at UC Santa Cruz where he directs the Design Reasoning Lab, working with graduate students from art, design, and engineering. He was previously a postdoctoral researcher in the Center for Game Science at the University of Washington where he created design tools for widely deployed educational games, an area he continued to work in as a consultant.
Adam was also a research engineer at Microsoft where he developed technologies that gave non-coders the ability to generate small, useful programs from just one or two input-output examples. His research broadly applies AI techniques in the game design process to problems such as automatic content creation and machine playtesting or quality assurance. Additionally, his personal mobile app development hobby has reached an audience of over ten million users.
SEMI, Milpitas, CA
673 S Milpitas Blvd, Milpitas, CA 95035
View Map & Directions