東華大學圖書館 |

Language: English

Help

回圖書館首頁

手機版館藏查詢

Back

Switch To: Labeled | MARC Mode | ISBD

Linked to FindBook

Google Book

Amazon

博客來

Multimodal Representations for Vision, Language, and Embodied Ai.

Record Type:	Electronic resources : Monograph/item
Title/Author:	Multimodal Representations for Vision, Language, and Embodied Ai./
Author:	Chen, Kevin.
Published:	Ann Arbor : ProQuest Dissertations & Theses, : 2021,
Description:	103 p.
Notes:	Source: Dissertations Abstracts International, Volume: 83-03, Section: B.
Contained By:	Dissertations Abstracts International83-03B.
Subject:	Robots. -
Online resource:	http://pqdd.sinica.edu.tw/twdaoapp/servlet/advanced?query=28688427
ISBN:	9798544204770

Multimodal Representations for Vision, Language, and Embodied Ai.
Chen, Kevin.

Multimodal Representations for Vision, Language, and Embodied Ai. - Ann Arbor : ProQuest Dissertations & Theses, 2021 - 103 p.

Source: Dissertations Abstracts International, Volume: 83-03, Section: B.

Thesis (Ph.D.)--Stanford University, 2021.

This item must not be sold to any third party vendors.

Recent years have seen incredible growth and advances in artificial intelligence research. Much of this progress has primarily been made on three fronts: computer vision, natural language processing, and robotics. For example, image recognition is widely considered the holy grail of computer vision, whereas language modeling and translation have been fundamental tasks in natural language processing. However, many practical applications and tasks require going beyond solving these domain-specific problems and instead require solving problems which involve all three of the domains together. An autonomous system not only needs to be able to recognize objects in an image, but also interpret natural language descriptions or commands and understand how they might relate to its perceived visual observations. Furthermore, a robot needs to utilize this information for decision-making and determining which physical actions to take in order to complete a task. In the first part of this dissertation, I present a method for learning how to relate natural language and 3D shapes such that the system can draw connections about words like "round" described in a text description with the geometric attributes of round in a 3D object. To relate the two modalities, we rely a cross-modal embedding space for multimodal reasoning and learn this space without fine-grained, attribute-level categorical annotations. By learning how to relate these two modalities, we can perform tasks such as text-to-shape retrieval and shape manipulation, and also enable new tasks such as text-to-shape generation. In the second part of this dissertation, we allow the agent to be embodied and explore a task which relies on all three domains (computer vision, natural language, and robotics): robot navigation by following natural language instructions. Rather than relying on a fixed dataset of images or 3D objects, the agent is now situated in a physical environment and captures its own visual observations of the space using an onboard camera. To draw connections between vision, language, and robot physical state, we propose a system that performs planning and control using a topological map. This fundamental abstraction allows the agent to relate parts of the language instruction with relevant spatial regions of the environment and to relate a stream of visual observations with physical movements and actions.

ISBN: 9798544204770Subjects--Topical Terms:

529507
Robots.

Multimodal Representations for Vision, Language, and Embodied Ai.
LDR:03414nmm a2200313 4500 001 2348628
005 20220912135621.5
008 241004s2021 ||||||||||||||||| ||eng d
020 $a 9798544204770
035 $a (MiAaPQ)AAI28688427
035 $a (MiAaPQ)STANFORDqw400zc0878
035 $a AAI28688427
040 $a MiAaPQ $c MiAaPQ
100 1 $a Chen, Kevin. $3 3687998
245 1 0 $a Multimodal Representations for Vision, Language, and Embodied Ai.
260 1 $a Ann Arbor : $b ProQuest Dissertations & Theses, $c 2021
300 $a 103 p.
500 $a Source: Dissertations Abstracts International, Volume: 83-03, Section: B.
500 $a Advisor: Savarese, Silvio.
502 $a Thesis (Ph.D.)--Stanford University, 2021.
506 $a This item must not be sold to any third party vendors.
520 $a Recent years have seen incredible growth and advances in artificial intelligence research. Much of this progress has primarily been made on three fronts: computer vision, natural language processing, and robotics. For example, image recognition is widely considered the holy grail of computer vision, whereas language modeling and translation have been fundamental tasks in natural language processing. However, many practical applications and tasks require going beyond solving these domain-specific problems and instead require solving problems which involve all three of the domains together. An autonomous system not only needs to be able to recognize objects in an image, but also interpret natural language descriptions or commands and understand how they might relate to its perceived visual observations. Furthermore, a robot needs to utilize this information for decision-making and determining which physical actions to take in order to complete a task. In the first part of this dissertation, I present a method for learning how to relate natural language and 3D shapes such that the system can draw connections about words like "round" described in a text description with the geometric attributes of round in a 3D object. To relate the two modalities, we rely a cross-modal embedding space for multimodal reasoning and learn this space without fine-grained, attribute-level categorical annotations. By learning how to relate these two modalities, we can perform tasks such as text-to-shape retrieval and shape manipulation, and also enable new tasks such as text-to-shape generation. In the second part of this dissertation, we allow the agent to be embodied and explore a task which relies on all three domains (computer vision, natural language, and robotics): robot navigation by following natural language instructions. Rather than relying on a fixed dataset of images or 3D objects, the agent is now situated in a physical environment and captures its own visual observations of the space using an onboard camera. To draw connections between vision, language, and robot physical state, we propose a system that performs planning and control using a topological map. This fundamental abstraction allows the agent to relate parts of the language instruction with relevant spatial regions of the environment and to relate a stream of visual observations with physical movements and actions.
590 $a School code: 0212.
650 4 $a Robots. $3 529507
650 4 $a Behavior. $3 532476
650 4 $a Natural language processing. $3 1073412
650 4 $a Success. $3 518195
650 4 $a Localization. $3 3560711
650 4 $a Graph representations. $3 3560730
650 4 $a Language instruction. $3 3686602
650 4 $a Neural networks. $3 677449
650 4 $a Robotics. $3 519753
650 4 $a Language. $3 643551
650 4 $a Datasets. $3 3541416
650 4 $a Experiments. $3 525909
650 4 $a Learning. $3 516521
690 $a 0771
690 $a 0679
710 2 $a Stanford University. $3 754827
773 0 $t Dissertations Abstracts International $g 83-03B.
790 $a 0212
791 $a Ph.D.
792 $a 2021
793 $a English
856 4 0 $u http://pqdd.sinica.edu.tw/twdaoapp/servlet/advanced?query=28688427