A wonderful lesson--Sentiment Analysis and Opinion Mining

Hello, my friends. This is my second post about this course. From the forth lecture, I've learned so much and I can't wait to share it with you.

The fourth lecture is about sentiment analysis and opinion mining. Before the class, Prof. Rosanna showed us a picture, on which the right answer is B. However, she wanted us to answer the wrong one, because she could consider who came late through this approach. When we said C, the one who came late just could not understand why most of people said the incorrect one, so they became nervous and suspicious of themselves. It is obvious that  when we need to make a decision we often seek out the opinions of others. 

From some examples of product, book and micro-blogging, I've developed a clearer understanding about the basic terms such as entity, attributes, opinion holder(the person who express opinions on an entity form a review, a rating, a twitter message, etc). And the concept of sentiment analysis, which is the field of study involving information retrieval and computational linguistics that analyzes people’s opinions, sentiments, evaluations, appraisals, attitudes, and emotions towards entities such as products, services, organizations, individuals, issues, events, topics, and their attributes.

These useful information can be used in many applications. For instance, how people perceive an entity, event monitoring, collect customers’ opinions and information to make decisions on what to buy, understand what do people of different places think about the same thing, learning analytics etc. Then we feel some different descriptions of opinions. Despite some opinions are implicit, we can also know the meaning of them.

The thing attracted me most is a online demo Rosanna showed us on the class. It is a tool can develop and analyze twitter message based on natural language processing. You can switch along the time line. For example, if I select the day, 18th, Jan, 2012. Base on that, the twitter message generate up to that point of time, so how do the people feel about different actors, actress and movies, you can see that there is a trend of development about the opinion, the very strong opinion can be developed among the users, because the twitter users will exchange more message more frequently and their opinion also converge. 

Along the development of the time, the opinion of the majority of population, the time of the Oscar should be around. Analyzing the day 26th,2012, people may have very strong opinion, we can see the sudden burst of the opinion. It is based on NLP technic, you just count your computer program, just count about the opinion, word are associated to the actors, actress and movies. 

So how can this kind of opinion minings possible. Technics like NLP and classification, test, analysis and polarity etc. I learn some different levels of sentiment classification. They are document level, sentence Level and attribute level. In addition, two methods for sentiment classification are introduced and some limitations are also pointed out. The first one is dictionary-based approach, which focus on checking whether +ve or –ve words appear in the document/sentences to be classified (+ve stands for good, excellent, durable etc. -ve stands for poor expensive etc.). The second one is called supervised learning. We use the documents’ sentiment class to train a classifier. 

Another important thing we should do is identifying the aspects of the entity commented on by the people who write the comments. For the sentence below, we can find the aspects or nouns are food, drinks, waiters, the interior of the restaurant. ‘The food in this restaurant is very delicious. There is a large variety of food and drinks. The waiters are very polite and helpful. The interior of the restaurant is also nicely decorated…’ So we should do the aspect-level analysis to extract nouns that appear frequently. In Python NLTK, we can do something like this:

We can then extract the nouns(NN) and choose the most frequently seen ones to be the aspects.

After extracting the nouns, we also need to associate the opinion words with the aspects. There are two methods to find them. First one is finding out the noun that is closest to the opinion word. And second one is developing rules to extract pairs of opinion words and aspects. For example, find the common patterns of words in the different comments.

At the end of the class, Prof. Rosanna showed us some excellent previous students’ group projects, which really gave me lots of ideas about our own project in this term. And I will try my best to complete it and share more wonderful ideas with all of you!

SAP can do more on social media analytics

I think social media adds virtual relationships into our lives, supplementing the traditional ways in which people are connected. Social networks, dating sites are all powerful tools that bring together people from different places in a virtual world. Unable to meet each other in person as we are, strangers separated by space can communicate perfectly using mobile phones or the Internet. The Six Degree of Separation has actually been reduced to only two- a person and a connected computer.

Through the donation logo, which mentioned in class, we know that some people will donate while someone not, and we can record their response time to reflect their intentions. Also the average amount of money donated per donor can reflect supportive strength on this issue.

We can use Python to analyze the content we collect, but we should also understand human behavior, user preferences and reactions, trends and problems at the same time. Only in this way can we build better social media services to support user interactions and information sharing. 

Besides Python, there is another perfect tool named SAP Social Media Analytics can also analyze the content. For instance, as a mobile phone operator, T Mobile want to know the needs of customers on the social media , interact with them, build a strong loyal customer base and help them solve problems. SAP Social Media Analytics helps T Mobile to realize their dream.

They have such a huge focus on their customers’ requirement on  T-mobile, and their people love their customers. Social media is huge for them. Their strategy is listen, engage, and resolve. They need tools to help them to realize these objectives. They need to understand what customers’ thought from listeners’ perspective. They need to be able to interact with them and build  loyal customer base and resolve their issues. They use respond with SAP product to customers on social media. From that, I love SAP Social Analytics, because it is easy to use and it gives me a whole listed picture in one chart. You will find that we’re able to realize even further to the customers and we’re able to speak with them. Not only about their problems at present but their previous issues. 

Their work is more complicated before, when they use just Facebook or Twitter to engage no tracking, no information. But now, it is like work has changed radically. Since they use SAP Cloud for Social, they have saved 15% improvement on their productivity. One of the unexpected advantages is be able to take potential customers to promoters and they can do that by understanding the thought that customers are having about new products and new services. Finally, they can complete this  change. For example, they want to market a new Samsung device, they can gather customers’s thought on the product and flag that and follow up with them when the devices are available and tell them where they can purchase them.

All in all, I think SAP is a perfect tool that can analyze the content we need. I will learn more about Python language and the tools like this to build better social media services to support user interactions and information sharing.