2014年10月25日星期六

Link Analysis--Relative importance between you and me



When analyzing a network, we are often interested in understanding the relative importance of each vertex. For example, identifying the most influential people, schools, 
countries, or scientific publications with the greatest impact. When we search for a given topic through searching engine, we usually focus on the page that is ranked at top one. This is because we want to find out the most authoritative Web page. In addition, the following factors are also considerable.
- Pages that contain the largest number of occurrence of the keywords
- Pages that are written by people/organizations that are trustworthy, or are experts in the topic
- Pages that are read by many people
- Pages that are linked to by many other pages (having a lot of in-links)

There are two popular Link Analysis techniques:

• HITS (Hyperlink-Induced Topic Search)
: there are two types of vertices in a network (hubs and authorities)
. Hubs contain list of inks to other pages while authorities contain useful information of a given topic.
PageRank:
Endorsement from more important vertices should be considered more valuable or credible.
HITS used in Twitter’s “Who to follow”, page ranking, recommendation Systems and PageRank used in part of Google’s search engine. Both algorithms rank pages by analyzing their in-coming and out-going links. And another algorithm named EigenRumor can identify blog posts in a blog community. The following chart shows the different application fields and algorithms among HITS, PageRank and EigenRumor.

In my last blog, I use the software "NodeXL Excel Template" to draw a sociagram of my blog network and know who is more active in sub groups of the whole class. And this time, I also use it to import the data of YouTube User's Network and use the function of Graph Metrics to analyze all kinds of information about this network. From the graph, we choose the mode named Harel-Koren Fast Mul to see the followers clearly. 

The users named UNC-Chapel Hill and themrfinneth followed many users. And we can see that tedtalksdirector followed tededucation, tedtalks, tedyouth, tedpartners, and tedfellowstalks. After we enter the webpage of “tedtalksdirector” on YouTube platform, we really can find this five users. It indicates that tedtalksdirector is more like a hub and from this hub we can find related users that tedtalksdirector followed.

There is one thing I have to mention. Because one of my friends shared a wonderful tool named "AlchemyAPI" with us. I think it is really powerful and useful. So I can't wait for using it to find more interesting things and added them in this blog. Firstly, I also test out the AlchemyLanguage API demo on the webpage based on this blog. I chose one of the functions, which can extract some keywords from my blog and calculate the relevance and sentiment value on what I’ve mentioned. The web use colorful squares to show the positive, neutral, negative and some mixed words. 
And then, I tried the function of AlchemyVision Face Detection, which can automatically detect faces and identify people within images. I selected an image, which was taken after our first group meeting. Then I uploaded it on the webpage and the result is quite funny. It shows us different ages of each group member. Actually, Wenwen is older than me, so I think the system may detect the gesture of victory of her. I guess this gesture can make people look very young? And the same to you, your ages are less than 18. Are you a student in high school? haha...



26 条评论:

  1. Hi,Yang.Your blog is great! Can't imagine you draw these graphs all by yourself.I guess you learned very well.

    回复删除
    回复
    1. Hi, Selina~Thanks for your comment. Yeah, it really took me a long time to draw these graphs. But I think it is worth doing that, because the process of reviewing the lecture can help me understand the core issues even deeper. Can't wait for reading your blogs and sharing ideas with U.

      删除
  2. Your blog includes detailed analyzing parts by social graphs and very interesting last part of which I think all the girls are below 18 ha. I'm deeply impressed by your hardwork, thank you!

    回复删除
    回复
    1. Hi, Tianyu~Thanks for your comment! After we have learned these lectures, I think the most important thing is putting the theoretic knowlege into real practice and you will find something different.

      删除
  3. Face recognition seems very interesting. I am still very young, haha

    回复删除
    回复
    1. Yeah, we are all very young! haha...Well, I know some basic knowledge of the technic of Face Recognition. For example, we can estimate a person's age form his wrinkle depth, color of hair etc. But the deeper knowledge of algorithm needs to be done by learning the lesson called machine learning, right?

      删除
  4. Wow!!! It so interesting!!! I am the youngest!!!

    回复删除
    回复
    1. Hi Shishi~ Thanks for your wonderful sharing of AlchemyAPI!! haha...I think I will use the gesture of victory next time. But I also find that the machine judges Cheng Fang as a man...OMG, so I think the algorithm of Face Recognition should be optimized in the future.

      删除
  5. very interesting and useful.
    I think you must spend much time on preparing it so the blog looks so substantial.

    回复删除
    回复
    1. Hi Guilin~Tanks for your comment! I recommend this online demo to you. Please click http://www.alchemyapi.com/products/demo/. And btw, thank you for helping me with my pythone assignment and hope we can share more with each other :)

      删除
  6. Awesome post and you deserve an A+++ for it. Your post reviewed all detailed knowledge in respective lecture. Furthermore, you leverage the knowledge gained from the class and apply to real world using practical tool, the result is fantastic. The overall look and the design of the post are also gorgeous. I think you can consider of designer career in the future which is in a large demand in nowadays mobile internal industry. By the way, thanks for the younger age than my actual age, so glad to see that :)

    回复删除
    回复
    1. Wow, tanks for your great comment! After learning Rosanna's whole class, I gained lots of knowledge about social media analytics like NLP, content analysis, sentiment analysis and opinion mining. And the Python programing assignments let us apply the knowledge into the practice. Additionally, I'm really interested in UI design, and I hope I can find an appropriate work in the future. Prof. Rosanna and you really taught me a lot of useful things. Thank you again my friend! :)

      删除
  7. Thank you for introducing all these useful tools.
    It is so interesting to apply what we learnt in real life example.

    回复删除
    回复
    1. Hi, Chu~Tanks for your comment. There are lots of analyzing tools nowadays and in my first blog I introduced another wonderful tool named SAP Social Media Analytics, which can timely analyze the feedbacks of customers on social media and we can deeply understand the opinions and orientation of customers.

      删除
  8. I am quite happy that I am under 18 in this face detection app. In fact, I have download several mobile apps in my cell phone. However, for different photos of the same guys, these apps always show different results, including different ages, different genders or even different races. To be honest, I am quite interesting in the algorithms behind these apps and why they will show quite different results in different photos.

    回复删除
    回复
    1. Yeah.. I think you are very happy because the system considers you as the youngest person. Each app may use different facial recognition technologies, so the result can be a little bit different. Hope you can find more interesting things in this field and can share with us.

      删除
  9. I am deeply impressed by your hard-working and u show all of us a good example to write blogs.and is that the truth the system will identify the age a little bit younger to satisfy we girls?haha

    回复删除
    回复
    1. Hi, my friend~Thanks for your comment! I think it may be my last blog, so I want to write more useful and interesting things that we can share with each other. And I really think you are quite humorous..haha, because others may think about improving the accuracy of Face Recognition Algorithm, but you focus on computers can also have an insight into human emotion. Tanks for your great idea :)

      删除
  10. Oh...I read the comments of this blog first, and I thought there must be a very interesting app recommended by you because they are all very happy...Then when I found the picture...I am 35-44 years old? And it's not enough, I am male...T-T It's not interesting at all, it's so cruel...

    回复删除
  11. It is the greatest blog I have ever read,you really did a good job in managing to turn the theory of the class into something useful!!!

    回复删除
  12. You apply what you learn in class into application. It really inspires me to take another look about studying and practicing.

    回复删除
  13. hi dear ,as I said last time ,your blog never disappoints me.This time I am happy to see a new tool from your post AlchemyAPI I'm planning to use it in our project .Thanks a lot

    回复删除
  14. It is the most excellent blog I have seen since we joined in this class. Not only have you review all the social media technology we learn from Prof.Rosanna, but do each experiment hand by hand and come up with your new idea. I admire your hard-working and creative. Thanks for your sharing and it my honor to be your group mate!

    回复删除
  15. Wow! AlchemyVision Face Detection is quite impressive. They must be using deep machine learning. In fact, a machine learning API is a good idea because they create a easy way to do machine learning at scale without having a deep understanding of machine learning techniques. You just give a picture, you ask for facedetection and you get the result. Thank you for sharing.

    回复删除
  16. Thank you for introducing alchemy api to me. It is a cost effective way to do data analytic. Alchemy have another useful api.

    AlchemyTagger works in conjunction with your WordPress-powered blog, analyzing your writing and suggesting new categorization tags for your posts.

    AlchemyCmd is a command-line tool for performing natural language processing and text analysis on Linux/Unix systems.

    回复删除
  17. I think your expression about link analysis is quite specific and with some pictures, I can easily understand and form some direct impression in my mind. the whole internet net is made up of websites. and all the contents are shared in the way of giving hyperlink.

    回复删除