MS Defense
Computer Science and Electrical Engineering

Social Media Data Analytics Applied to Hurricane Sandy

Han Dong

12:30-2:30 Monday, 29 July 2013, ITE 325b

Social media websites are an integral part of many people’s lives in delivering news and other emergency information. This is especially true during natural disasters. Furthermore, the role of social media websites is becoming more important due to the cost of recent natural disasters. These online platforms are usually the first to deliver emergency news to a wide variety of people due to the significantly large number of users registered. During disasters, extracting useful information from this pool of social media data can be useful in understanding the sentiment of the public; this information can then be used to improve decision making. In this work, I am presenting a system that automates the process of collecting and analyzing social media data from Twitter. I also explore a variety of visualizations that can be generated by the system in order to understand the public sentiment. I demonstrate an example of utilizing this system on the Hurricane Sandy disaster from October 26, 2012 to October 30, 2012. Finally, a statistical analysis is performed to explore the causality correlation between an approaching hurricane and the sentiment of the public.

As a result of the large amount of data collected by this system; scalable machine learning algorithms are needed for analysis. Boosting is a popular and powerful ensemble method in the area of supervised machine learning algorithms due to its theoretical convergence guarantees, simple implementation and ability to use different learning algorithms to produce a classifier with high accuracy. A novel parallel implementation of the multiclass version of Boosting (AdaBoost.MH) is proposed and our experimental results show that the parallel implementation achieves classification error percentages similar to serial implementation with fewer execution iterations. By distributing the tasks, the number of Boosting iterations decreased linearly at least up to 16 computational threads.

Committee: Professors Milton Halem (chair), Yelena Yesha, John Dorband and Shujia Zhou

As a beginner in any specific web tech, your first concern is "what is this and how does it work?" As you level up, those concerns change. You may still learn about about how it works as you dive deeper, but other concerns will emerge the more you know, the more you use it, and the more work of others you are exposed to.

CSS

Beginner

• How do I make stuff look how I want?

Intermediate

• How can I keep this under control?
• Can/should I use frameworks/libraries?
• How do I deal with cross-browser problems?
• Should I be using a preprocessor?

Expert

• How can I write less of this?
• Can/should I create frameworks/libraries?
• Is testing possible?
• How can I improve the tooling?

JavaScript

Beginner

• How do I make stuff happen?

Intermediate

• How can I organize this?
• What are the best practices regarding speed and efficiency?
• What jobs are best left to other web tech?

Expert

• How can I write less of this?
• Are Preprocessors appropriate for this?
• How can I keep this really fast?
• How does a team write this?
• How do we make less mistakes?

HTML

Beginner

• What tags do I use for what?

Intermediate

• How can I write this really efficiently?
• Am I being as semantic as I can be?
• Am I being as accessible as I can be?

Expert

• What tags do I use for what? lol.

Back End Languages

Beginner

• What is the syntax?
• How do I perform basic loops and logic?

Intermediate

• How do I organize this?

Expert

• How can I keep this fast?
• How can I keep this secure?
• Am I using the right languages for the right jobs?

Databases

Beginner

• How can I store and get stuff?

Intermediate

• How do I structure this intelligently?

Expert

• How can I scale this?
• How can I predict our future needs?
• What types of databases are right for what kind of data?

What does it mean?

I'm not sure exactly. It's just interesting to think about. Here's some maybes:

From an education perspective it would be interesting to know how many people there are out there at each level and how interested they are in getting help on those questions.

From a hiring perspective it may help identify the relative skill level of hires by seeing what kind of things they are concerned about at the moment in a particular area.

From a self improvement perspective it may help you understand what is in store for you as you level up.

How Web Tech Concerns Change is a post from CSS-Tricks