Showing posts with label social network analysis. Show all posts
Showing posts with label social network analysis. Show all posts

Sunday, November 11, 2012

Analysis of the statistics blogosphere

My analysis of the statistics blogosphere for the Coursera Social Networking Analysis class is up. The Python code and the data are up at my github repository. Enjoy!

Included are most of the Python code I used to obtain blog content, some of my attempts to automate the building of the network (I ended up using a manual process in the end), and my analysis. I also included the data. (You can probably see some of your own content.)

Here's what I learned/got reminded of the most:

  • Doing projects like this is hard when you have other responsibilities, and you usually end up paring down your ambitions toward the end
  • Data collection and curation was, as usual, the most difficult process
  • Network analysis is fun, but I have a ways to go to know where to start first, what questions to ask, and so forth (these are the things you learn with experience)
  • The measures that seem to be the most revealing are not always obvious -- in this network, it was the number of shortest paths compared to a random graph
  • Andrew Gelman's blog is central (but you probably don't need a formal analysis to tell you that)
  • There's a lot of great content about statistics, data analysis, data science, and statistical computing out there. I've relied on blog posts for a lot of my work, and I've found even more great stuff. It's a firehose of information.

Monday, November 5, 2012

Snapshot of the statistics blogosphere

stats_blogs

This was generated during my social network analysis project. I haven’t finished yet, but I did want to show the cute picture. The statistics blogosphere is like a school of jellyfish.

Monday, October 29, 2012

The most valuable thing about my little stat blog network project

So, I decided to construct the linking graph through blogrolls, and finally settled on using a manual process. The best part of this project is really finding out for myself all the great content out there!

Monday, October 22, 2012

SNA class proposal

I’ve been taking several classes through Coursera (nothing against the other platforms; I took two of the original three classes via Stanford and just stuck with the platform). The latest one is Social Network Analysis, which has a programming project. Here is what I have posted as a proposal:

Ok, I've been thinking about the programming project idea some, and at first I was thinking of analyzing the statistics blogging community, mostly because I belong to it and I wanted to see what comes out. The analysis below can be done for any sort of community. I've developed this idea a little further and wanted to record it here for two reasons. First, I simply need to write it down to get it out of my head and in such a way that the public can understand it. Second, I'd like feedback.

As it turns out, I took the NLP class in the spring and think there's some overlap that can be exploited. (This comes up nicely in the Mining the Social Web and Programming Collective Intelligence books.) There are measures of content similarity, such as cosine similarity, which are simple to compute and reasonably work well to see how similar content is. Content can then be clustered based on similarity. So, then, I have the following questions:

  • What are the communities, and do they relate to clusters of content similarity?
  • If so, who are the "brokers" between different communities, and what do they blog about? There are a couple of aggregators, such as StatBlogs and R-Bloggers, that I imagine would glue together several communities (that's their purpose and value), but I imagine there are a few others that are aggregator-like + commentary as well. Original content generators, like mine, will probably be on the edges.
  • Is it better to threshold edges based on a number of mentions, or use an edge weight based on the number of mentions?
  • If I have time, I may try to do some sort of topic or named entity extraction, and get an automated way of seeing what these different communities are talking about.