The Refactoring

Lately I’ve been working on something that I’m calling The Refactoring.

Not that long ago, I wrote about my Twitter Conversation Networks (read the rationale behind them) and I was blown away by the popularity and the amazing feedback that I got. So I tried to incorporate it all in.

Here’s the thing, though, it was an experiment and frankly kind of a hack. I didn’t have a clear idea of where I’d go with it, I was just playing with things and seeing what would happen. So I rapidly realized that it wouldn’t scale as it was, and I needed to go back to the beginning and start again from scratch, building more complex data structures and making it as modular as possible.

In tutorials and the lab, I’m often telling students not to get too attached to their code. In fact, when one of them was reluctant to delete a load I told him about this and he looked at me in horror. But it was easy for me to throw it out and start over, because the first thing was just a prototype and after seeing how people used it I have so many more ideas, that will be so much easier when I design it from the ground up with scalability and potential in mind. In fact, I’ve effectively created a Twitter data-mining library that will make it easy to extract information from your network, which is also used to generate the data for my List Visualizations. This will (eventually!) be open-sourced.

The thing no-one tells you about being a grad student, though, is how little time you get to spend coding. Between TA-ing (especially marking), this course I’m taking with the copious amount of homework, reading papers, and making presentations… I’m not where I want to be. However, I made significant progress on it yesterday and here are some of the things that are going to be possible, as a result of The Refactoring.

• Weak-tie and strong-tie graphs, so you’ll be able to say I just want to see connections where there are a minimum of two mentions, or three. I’m interested to see how this will change the graph for a user, as we progressively increase the threshold.
• Your full graph (including weak ties) but only stronger ties for those people you’re connected to – this should help filter out noise as the graphs get very busy. You’ll be able to have an increment, that will increase the threshold as you get further from your central node. This means that it will be possible to go to a greater depth.
• Related to this, this should make it easy to show connections amongst your first degree nodes, but not connections to people you are not connected to.
• Reciprocal connection only graphs.

What else would you like to see about your network?

I really hope to finish this tomorrow and will need test subjects. You can indicate below if you want to be one, comment including your Twitter handle and what kind of graph you want (i.e. specify tie-strength for central node, tie-strength for first-degree and increment – if any).

The first five people to comment will definitely get graphs. Thereafter, I’ll do my best but it is the end of the semester so I cannot promise anything on the time frame, sorry.