Artificial intelligence (AI) technologies are emerging and affecting our lives in many aspects. However, the majority of individuals are merely users of AI with little capability to adapt AI to their own long-tail tasks, preferences, and interests that are not covered by the existing AI solutions. Democratizing AI to empower the individuals to create, configure, and extend AI-powered systems requires a new human-centered approach to guide the users to provide more effective inputs to the AI systems, and to enable the systems to better understand and to generalize from inputs in the user’s natural style.
In this talk, Toby Jia-Jun Li of Carnegie Mellon University will present his research on designing, developing, and studying (1) a new smartphone interactive task learning agent that empowers end-users to teach new tasks, concepts, and automation rules using the combination of natural language instructions and demonstrations on existing third-party app GUIs; and (2) a new developer tool that allows developers without natural language processing expertise to bootstrap a task-oriented conversational bot from existing mobile apps using the task model extracted from the app interaction traces.
These projects exemplify his approach of combining human-centered methods with AI techniques to create new novel interfaces and system capabilities that facilitate effective human-AI collaboration, lowering the barriers to customizing, extending, and developing AI applications for end-users and non-expert developers.
Toby Jia-Jun Li is a final year Ph.D. candidate in the Human-Computer Interaction Institute at Carnegie Mellon University, advised by Prof. Brad A. Myers. He also works closely with Prof. Tom M. Mitchell.
Toby works at the intersection of HCI, applied AI, and human factors in software engineering, where he uses human-centered methods to design, build, and study interactive systems to empower end-users and novice programmers to create, configure, and extend AI-powered computing systems.
He has authored more than 15 publications in premier academic venues across HCI, NLP, and systems (e.g., CHI, UIST, CSCW, ACL, MobiSys, VL/HCC), including three award-winning papers. His work has been supported by a Yahoo! Fellowship through the InMind project, by J.P. Morgan, and by the NSF. You can learn more about Toby here.
Contact Ginny Watterson for Zoom link.