Talking to Robots: Learning to Ground Human
Language in Robotic Perception

Cynthia Matuszek
University of Washington

1:00pm Monday, 7 April 2014, ITE325b, UMBC

Advances in computation, sensing, and hardware are enabling robots to perform an increasing variety of tasks in ever less constrained settings. It is now possible to imagine robots that can operate in traditionally human-centric settings. However, such robots need the flexibility to take instructions and learn about tasks from nonspecialists using language and other natural modalities. At the same time, learning to process natural language about the physical world is difficult without a robot’s sensors and actuators. Combining these areas to create useful robotic systems is a fundamentally multidisciplinary problem, requiring advances in natural language processing, machine learning, robotics, and human-robot interaction. In this talk, I describe my work on learning natural language from end users in a physical context; such language allows a person to communicate their needs in a natural, unscripted way. I demonstrate that this approach can enable a robot to follow directions, learn about novel objects in the world, and perform simple tasks such as navigating an unfamiliar map or putting away objects.

Cynthia Matuszek is a Ph.D. candidate in the University of Washington Computer Science and Engineering department, where she is a member of both the Robotics and State Estimation lab and the Language, Interaction, and Learning group. She earned a B.S. in Computer Science from the University of Texas at Austin, and M.Sc. from the University of Washington. She is published in the areas of artificial intelligence, robotics, ubiquitous computing, and human-robot interaction.

UMBC ACM Student Chapter

Tech Talk: What can a Humanoid Robot Learn?

Professor Tim Oates, UMBC

10:00am-11:00am Friday, 4 April 2014, ITE346

We hope everyone had a fun spring break! We are back with another talk in our UMBC ACM techTalk series. Professor Tim Oates, who is also one of the faculty advisers of the ACM student chapter, will talk about “What can a Humanoid Robot Learn?”. Dr. Oates will split the talk into two sessions. In the first half, he will introduce the topic and talk about current research being pursued in the area of humanoid robots. Whereas, the second half of the talk will be an interactive session focusing on ideating challenges and future research directions. To make the discussion interactive, Dr. Oates encourages you to spend a few minutes beforehand to think about what you would do if you had access to a humanoid robot for research purposes.

Abstract: Robots and AI have a long history together, both in the popular culture and in research.  In this talk I will review some of my past work at the juncture of robotics, AI, and machine learning, as well as ongoing work with collaborators at UMCP along the same lines.  With those collaborators, I wrote a proposal to buy a few humanoid robots that was funded, so I’ll next describe the robots that we’ve bought.  Finally, I’d like to have an open discussion about my ideas for research using these robots, and ideas that those in the audience might have as well.  My goal is to get as many people as is practical involved with the robots, which are relatively expensive and thus not a common resource.  If you’re coming to the talk, spend a few minutes beforehand thinking about what you would do if you had access to a humanoid robot for research purposes.

Dr. Tim Oates is an Oros Familty Professor of Computer Science at the University of Maryland Baltimore County. He received B.S. degrees in Computer Science and Electrical Engineering from North Carolina State University in 1989, and M.S. and Ph.D. degrees from the University of Massachusetts Amherst in 1997 and 2000, respectively. Prior to coming to UMBC in the Fall of 2001, he spent a year as a postdoc in the Artificial Intelligence Lab at the Massachusetts Institute of Technology. In 2004 Dr. Oates won a prestigious NSF CAREER award. He is an author or co-author of more than 100 peer reviewed papers and is a member of the Association for Computing Machinery and the Association for the Advancement of Artificial Intelligence. His research interests include pattern discovery in time series, grammatical inference, graph mining, statistical natural language processing, robotics, and language acquisition.

RSVP for the talk at http://my.umbc.edu/events/23881

UMBC ACM Student Chapter

Tech Talk: What can a Humanoid Robot Learn?

Professor Tim Oates, UMBC

10:00am-11:00am Friday, 4 April 2014, ITE346

We hope everyone had a fun spring break! We are back with another talk in our UMBC ACM techTalk series. Professor Tim Oates, who is also one of the faculty advisers of the ACM student chapter, will talk about “What can a Humanoid Robot Learn?”. Dr. Oates will split the talk into two sessions. In the first half, he will introduce the topic and talk about current research being pursued in the area of humanoid robots. Whereas, the second half of the talk will be an interactive session focusing on ideating challenges and future research directions. To make the discussion interactive, Dr. Oates encourages you to spend a few minutes beforehand to think about what you would do if you had access to a humanoid robot for research purposes.

Abstract: Robots and AI have a long history together, both in the popular culture and in research.  In this talk I will review some of my past work at the juncture of robotics, AI, and machine learning, as well as ongoing work with collaborators at UMCP along the same lines.  With those collaborators, I wrote a proposal to buy a few humanoid robots that was funded, so I’ll next describe the robots that we’ve bought.  Finally, I’d like to have an open discussion about my ideas for research using these robots, and ideas that those in the audience might have as well.  My goal is to get as many people as is practical involved with the robots, which are relatively expensive and thus not a common resource.  If you’re coming to the talk, spend a few minutes beforehand thinking about what you would do if you had access to a humanoid robot for research purposes.

Dr. Tim Oates is an Oros Familty Professor of Computer Science at the University of Maryland Baltimore County. He received B.S. degrees in Computer Science and Electrical Engineering from North Carolina State University in 1989, and M.S. and Ph.D. degrees from the University of Massachusetts Amherst in 1997 and 2000, respectively. Prior to coming to UMBC in the Fall of 2001, he spent a year as a postdoc in the Artificial Intelligence Lab at the Massachusetts Institute of Technology. In 2004 Dr. Oates won a prestigious NSF CAREER award. He is an author or co-author of more than 100 peer reviewed papers and is a member of the Association for Computing Machinery and the Association for the Advancement of Artificial Intelligence. His research interests include pattern discovery in time series, grammatical inference, graph mining, statistical natural language processing, robotics, and language acquisition.

RSVP for the talk at http://my.umbc.edu/events/23881

Three students in UMBC’s College of Engineering and Information Technology (COEIT) Department of Computer Science and Electrical Engineering (CSEE) have been selected for major scholarships to pursue studies in cybersecurity-related fields under UMBC’s participation in the National Science Foundation’s (NSF) Scholarship for Service (SFS) Federal CyberCorps program. As SFS Scholars, students receive full tuition, fees, annual reimbursement of professional development expenses ($3,000), a nine-month stipend ($20,000 for undergraduates, $25,000 for MS/MPS students, and $30,000 for PhD students) for up to two years (three years for PhD or BS/MS), and assistance with federal cybersecurity internships and career placement. To be eligible for SFS funding, applicants must be full-time students at UMBC committed to cybersecurity-related research and education activities, have an excellent academic record, and not have any significant outside employment obligations.

The first-round awardees for AY14-15 are:

• Anastasia Raffucci, BS (CMPE)
• Jackson Schmandt, Ph.D. (CMPE)
• Brooke Young, BS (CMSC)

These new awardees will join existing UMBC SFS Scholars Oliver Kubik (BS, CMSC), Mary Mathews (PhD, CMSC), Nathan Price, MS (CMPE). Punlada Muangrat BS (IS), Alex Cooke BS (IS), and Denis Danilin MS (IS).

ADDITIONAL SLOTS AVAILABLE! Applications for SFS support for AY14-15 will be accepted through 12PM (noon) on Friday, May 2, 2014 with notice of awards made by May 23, 2014. All levels of study are welcome to apply — however, students must be admitted to (or already enrolled at) UMBC first before applying for SFS. For information about SFS (including application information and instructions) and other US government-related cybersecurity scholarships available, please visit the CISA website.

The CyberCorps program produces highly-qualified professionals to meet the United States government’s increasing need to protect American’s cyber infrastructure. While in the program at UMBC, SFS CyberCorps Scholars participate in special SFS program activities, have opportunities to engage in mentored research opportunities both at UMBC and its partners from industry and government, and must complete a paid summer internship for the federal government. Upon graduation, each student must work for the government (for pay) for one year for each year of scholarship received. CSEE Drs. Alan Sherman and Richard Forno direct the program under a five-year $2.5 million NSF grant received in 2012.

Three students in UMBC’s College of Engineering and Information Technology (COEIT) Department of Computer Science and Electrical Engineering (CSEE) have been selected for major scholarships to pursue studies in cybersecurity-related fields under UMBC’s participation in the National Science Foundation’s (NSF) Scholarship for Service (SFS) Federal CyberCorps program. As SFS Scholars, students receive full tuition, fees, annual reimbursement of professional development expenses ($3,000), a nine-month stipend ($20,000 for undergraduates, $25,000 for MS/MPS students, and $30,000 for PhD students) for up to two years (three years for PhD or BS/MS), and assistance with federal cybersecurity internships and career placement. To be eligible for SFS funding, applicants must be full-time students at UMBC committed to cybersecurity-related research and education activities, have an excellent academic record, and not have any significant outside employment obligations.

The first-round awardees for AY14-15 are:

• Anastasia Raffucci, BS (CMPE)
• Jackson Schmandt, Ph.D. (CMPE)
• Brooke Young, BS (CMSC)

These new awardees will join existing UMBC SFS Scholars Oliver Kubik (BS, CMSC), Mary Mathews (PhD, CMSC), Nathan Price, MS (CMPE). Punlada Muangrat BS (IS), Alex Cooke BS (IS), and Denis Danilin MS (IS).

ADDITIONAL SLOTS AVAILABLE! Applications for SFS support for AY14-15 will be accepted through 12PM (noon) on Friday, May 2, 2014 with notice of awards made by May 23, 2014. All levels of study are welcome to apply — however, students must be admitted to (or already enrolled at) UMBC first before applying for SFS. For information about SFS (including application information and instructions) and other US government-related cybersecurity scholarships available, please visit the CISA website.

The CyberCorps program produces highly-qualified professionals to meet the United States government’s increasing need to protect American’s cyber infrastructure. While in the program at UMBC, SFS CyberCorps Scholars participate in special SFS program activities, have opportunities to engage in mentored research opportunities both at UMBC and its partners from industry and government, and must complete a paid summer internship for the federal government. Upon graduation, each student must work for the government (for pay) for one year for each year of scholarship received. CSEE Drs. Alan Sherman and Richard Forno direct the program under a five-year $2.5 million NSF grant received in 2012.

TL;DR: UMBC’s IEEE student branch will hold free weekly workshops on Arduino starting next week. Register at http://bit.ly/OVzA1f.

The Arduino micro-controller is a great device for anyone who wants to learn more about technology. It is used in a variety of fields in research and academia as well as by hobbyists. Arduino can be used for projects ranging from quadcopters to thermal cyclers  and even wearable electronics. Here is more information about the Arduino.

The workshops are intended for ANY student from ANY major and only require a basic understanding of basic programming concepts such as if-statements and loops. The majority of the code will be provided.

The Arduino workshops are offered on a weekly basis at two levels: Level 1 for those new to Arduino and Level 2 for students who have completed Level 1 or have a basic familiarity with Arduino. The Level 1 workshop is offered on Tuesday and Wednesday from 7:00pm to 9:00pm starting the week of April 1st. You may select to participate in either our Tuesday lecture or Wednesday lecture. The Level 2 workshop is only offered on Thursday from 7:00pm to 9:00pm starting the week of April 1st as well. For both workshops, you will need to bring your laptop and you must able to download the Arduino IDE.

Register at http://bit.ly/OVzA1f. Seats are limited. Contact Sekar Kulandaivel (Sorry, you need javascript to view this email address. ) with questions about registration, downloading the IDE, or anything else.

TL;DR: UMBC’s IEEE student branch will hold free weekly workshops on Arduino starting next week. Register at http://bit.ly/OVzA1f.

The Arduino micro-controller is a great device for anyone who wants to learn more about technology. It is used in a variety of fields in research and academia as well as by hobbyists. Arduino can be used for projects ranging from quadcopters to thermal cyclers  and even wearable electronics. Here is more information about the Arduino.

The workshops are intended for ANY student from ANY major and only require a basic understanding of basic programming concepts such as if-statements and loops. The majority of the code will be provided.

The Arduino workshops are offered on a weekly basis at two levels: Level 1 for those new to Arduino and Level 2 for students who have completed Level 1 or have a basic familiarity with Arduino. The Level 1 workshop is offered on Tuesday and Wednesday from 7:00pm to 9:00pm starting the week of April 1st. You may select to participate in either our Tuesday lecture or Wednesday lecture. The Level 2 workshop is only offered on Thursday from 7:00pm to 9:00pm starting the week of April 1st as well. For both workshops, you will need to bring your laptop and you must able to download the Arduino IDE.

Register at http://bit.ly/OVzA1f. Seats are limited. Contact Sekar Kulandaivel (Sorry, you need javascript to view this email address. ) with questions about registration, downloading the IDE, or anything else.

The next Council of Computing Majors (CCM) will be at Noon this Wednesday, March 26th, in Math/Psychology 008. One of our group members, Patrick McElvaney, will give a presentation about a new CCM project: using the Raspberry Pi and photo-receptors to analyze water in the Chesapeake watershed. His talk will provide an overview of the installation of sensing equipment, specific details on how the photo-receptors can differentiate between organic and inorganic compounds, and information on how other students can get involved.

The CCM is a student organization for undergraduate computer science and computer engineering majors as well as other students interested in computing. If you are interested in starting your own project, please come and speak with any of the officers after the meeting.

The next Council of Computing Majors (CCM) will be at Noon this Wednesday, March 26th, in Math/Psychology 008. One of our group members, Patrick McElvaney, will give a presentation about a new CCM project: using the Raspberry Pi and photo-receptors to analyze water in the Chesapeake watershed. His talk will provide an overview of the installation of sensing equipment, specific details on how the photo-receptors can differentiate between organic and inorganic compounds, and information on how other students can get involved.

The CCM is a student organization for undergraduate computer science and computer engineering majors as well as other students interested in computing. If you are interested in starting your own project, please come and speak with any of the officers after the meeting.

CSEE Professor Marie desJardins has been named as UMBC’s Presidential Teaching Professor for 2014-17. Dr. desJardins joined the UMBC faculty in 2001 after earning her Ph.D. in computer science from UC Berkeley and spending ten years at SRI International as a research scientist. She has made significant contributions to the fields of artificial intelligence and machine learning, published over 100 peer-reviewed papers, brought over six million dollars to UMBC as PI or co-PI in external grant funding, and held leadership positions in the top professional organizations in her field.

Dr. desJardins is an outstanding teacher, earning praise from students in courses from freshman-level courses for non-majors to specialized graduate-level seminars. She was named one of UMBC’s “Professors Not to Miss” in 2011 and is one of the first cohort of Hrabowski Academic Innovation Fellows. She is also well known for mentoring students at all levels, having graduated ten Ph.D. and 22 M.S. students, mentored over 50 undergraduates in research and served on the dissertation and thesis committees of more than 30 other students. She was recently recognized for mentoring by the National Center for Women & Information Technology, who selected her as one of four awardees of the 2014 NCWIT Undergraduate Research Mentoring Award.

Within the department and university, Dr. desJardins’s commitment to teaching and student success goes far beyond the classroom. She has served as the computer science undergraduate program director and led an effort to redesign the introductory computing course to better serve new students. In addition to mentoring her own graduate students, she is the Faculty Advisor for the Women in Science and Engineering Graduate Association and a member of the Center for Women in Technology Advisory Board. She is regularly invited to participate on panels, give presentations in the Honors Forum and other campus events, and to run workshops for graduate students and junior faculty.

Outside of UMBC, Dr. desJardins has built an international reputation as an advocate for high-quality education, mentoring, and diversity at all levels of the profession. She has been chair, mentor, reviewer, and/or panelist of the AAAI/SIGART Doctoral Consortium for the last 14 years; this event has provided valuable feedback and mentoring to hundreds of computing graduate students during that time. She co-founded the AAAI Educational Advances in Artificial Intelligence annual symposium. She regularly publishes articles on her research and innovations in computing education, including tools and techniques for classroom teaching, new courses, and analyses of the state of computer science education at the high school level.

Dr. desJardins is also a nationally recognized leader in computer science education and has received multiple NSF awards to support her work in this area. She gives frequent presentations around the state and the country on high school computer science education and preparing a diverse population of students to succeed in computing careers. She is a founding member of the Maryland chapter of the Computer Science Teachers Association, and has organized several professional development workshops for high school teachers. Her NSF-funded CE21-Maryland (Computing Education for the 21st Century) grant explored the landscape of high school CS education in Maryland, culminating in a statewide summit for educators, administrators, and community members that was held at UMBC in May 2013. A recent NSF CE-21 grant will result in curriculum creation and professional development for 100 Maryland high school teachers, focused on the new CS Principles course that is scheduled to become a new AP offering in 2016. Other funded grants in the educational arena include her Hrabowski Innovation Fund award to create the ACTIVE Center, an NSF TUES award that is developing a new freshman-level computing course, an NSF T-SITE grant to build a community of transfer scholars in IT/engineering, as well as multiple smaller awards to run workshops and support graduate student development.

We congratulate Professor desJardins for her selection as Presidential Teaching Professor and look forward to the Presidential Faculty and Staff Awards ceremony on Wednesday, April 2 in the University Center Ballroom. Not only is she an outstanding and dedicated classroom teacher, her contributions to research, teaching, mentoring, and educational innovations have been broad and sustained.