You can change around the axis etc however you like. Suggestion:
Color: unique colors
Size: Number of Cliques
Y-axis: Followers
X-axis: Following
Now press the “play” button at the bottom left of the chart. You’ll see the bubbles change in size. I know it looks like a year, but it’s not – 1903 means number of cliques of size 3, 1904 is number of cliques of size 4, etc (haven’t figured out how to fix this yet, if anyone knows please let me know in the comments – thanks!)
Abstract for a talk I’m giving tomorrow some time between 1300h and 1500h at the University of Ottawa. Let me know if you want to attend (slides will be up later).
Follower / Following networks are essentially meaningless on Twitter due to the prevalence of spam. However by creating the graphs of conversation networks it is possible to create a better picture of more meaningful connections – the other users that interact / are interacted with by a given user. For power users, however, these graphs can be extremely busy, making it difficult to pick out the most important conversations and connections.
One potential way to summarize the most important connections in a network is to pull out cliques – completely connected sub-graphs. These cliques may represent part of a users core network, or a suggestion of new users to interact with by generating those cliques that a user is connected to. For example, user A might be in a clique with users B, C, D and users B, C, D may be in a clique with a 5th user, E. This suggests that user A might well be interested to interact with user E, as well. This may also help us determine tie strength as well, as a clique is likely an indication of a stronger tie strength than just a singly connected node.
Last night I went to my first Girl Geek Dinner. It was awesome. I don’t think I’d even realized how male-dominated my existence was, until I was in a room full of women (and maybe 2 guys). Yes, there was some talk of handbags, but everyone seemed interested in everyone else, there was so much talk of the projects we’re working on and when people stood up to do the open mic, it was great to hear about what they’re doing.
I really felt that this was a community of women who gather together – not to compete – just to connect and cheer each other on. That’s amazing.
Last call for Open Mic, I was brave enough to do it! Earlier in the evening I was all, nooo could not do that, too scary but I did it. And the other women I was sitting with were literally cheering me on. Wow.
I talked about WISE, and how we are always looking for great speakers (afterward a couple of people connected with me and now February’s event is covered, it’s going to be on entrepreneurship – more soon!). And I mentioned what I was working on and invited suggestions from people as to what they’d like to see about their Twitter network.
There was in fact no dinner last night, so once people had dispersed I went for sushi with Terri from CU:WISE and her boyfriend John. There was some competition for biggest geek (it will come to no surprise to anyone who knows me that I lost) and we talked about programming and open source (another blog post, coming soon) and it was really great.
So what did I come away with? A stack of business cards, speakers for WISE in February, a bit more confidence after speaking in front of people with a microphone and total impatience for the next event. Now, if you’ll excuse me, I’m off to follow all the awesome people I met on Twitter.
If you’re not a computer scientist, probably you think of cliques like in the movie Mean Girls. A bunch of people who talk the same and look the same and act the same…
Credit: flikr / Xpectro
To computer scientists (well, not all of them – I’m more inclined to think about Mean Girls to be honest) cliques are a part of graph theory. Essentially, it’s a group of nodes within a graph that are all connected to each other (Wikipedia).
So what? Big who cares, right? Graph theory is super boring!
I’m not big on the maths side of computer science, been there, got the t-shirt, and now I want to make stuff! But recently I’ve been working on applying graph theory to the Twitter Conversation Networks I make in order to produce my graphs. Why? Because this will allow me to pull out the sub-networks that you’re a part of, and the cliques that you’re connected to. Maybe if you know this you’ll find other people you’re potentially interested in following or (even better!) talking to.
For instance, here are my cliques of size 4 (the maximum size of the cliques found) within my 2 degree network:
There are more for size 3, but you get the picture.
This is a step towards making simpler graphs, where I will graph only the cliques in someone’s network – this will make it easier to see your important connections, I think. For now, if you want one, you can have a mapping and the result of clique finding for a minimum size of 3 or more.
How it’s done
I use the GraphML generated by my Twitter datamining library, that can also be used to produce visualizations.
A Python script generates a mapping and a list of connections
I implemented clique finding using Haskell – this means it runs really quickly and I don’t have to worry about memory because of lazy evaluation.
Minor optimization in the Haskell code, but significant due to the sparse nature of the graphs: I only try and find connections for nodes that have the minimum number of connections. Should be able to further optimize so that the same cliques aren’t found multiple times.
As a student, you don’t really learn how to delegate. It’s one of those crucial life skills that doesn’t seem to make it onto the curriculum, and when we think about it – how would it?
There are opportunities, but you have to go out and find them. Volunteer to team lead on a group project. Join a student organization and take on an exec position.
The thing is, if you delegate writing some code in a group project to someone who’s a terrible programmer, they let the whole team down. You often have no control over who the people in your group are, let alone whether they are competent. The danger is, first time around, you don’t realize this – you think they know roughly the same things you do. Some might, some won’t, particularly if your program has a lot of flexibility in course choices and supports joint programs. The second time around, you might have learned from this experience but you can’t necessarily change it. You might know that some people aren’t competent, but you don’t have the time to discover who is or who isn’t, and in a group project you can’t just kick people out because they don’t know what they are doing.
Group project experiences:
As team leader in a group project I organized the task divisions. One guy completely screwed his up, let us all down and our project was kinda a bust. This was the case for every other group as well, but that didn’t help the feeling of crushing failure that I’d delegated badly.
In another group project, one guy was such a muppet he wrote a loading screen that came on screen for 10 seconds and delayed the launch of the application until it had gone. One of my friends ended up having to rewrite all of his code for that and everything else (later, when I had to code a loading screen, it was so simple I couldn’t believe he’d managed to get it so very wrong). Another of my friends was excluded from her sub-group in this project, as the other (male) members of that group took the tasks that had been assigned to that sub-group and did them all. When she finally told me, I flipped out and emailed the prof – he was really nice and took care of things. She said it wasn’t sexist, but I think it was because there’s no way they would have done that to another guy – even one as incompetent as loading screen guy, and my friend is a talented programmer.
Problems with group projects in University:
People you work with don’t necessarily know what they’re doing.
You don’t necessarily respect each other. If I’m at a company, hopefully I respect people because they’ve gone through the same hiring process as I have. I should be able to assume they’ve proven their abilities. This is not the case at university.
In the real world, there are significant benefits to being liked that do not apply here (you don’t usually get graded on how easy you are to work with).
There is no clear hierarchy – even if you’re the group leader, that doesn’t mean you have years of experience and have proven yourself, it’s just luck of the draw or being the person that takes charge.
All this leads to it being hard to delegate, and the same issues apply in a student organization – perhaps even more so.
People may not think you have a right to ask them to do anything.
There may be no significant downside (to them) in not doing it.
They may be disorganized, and forget.
They may think the task you’ve asked them to do is not important, but ignore the request instead of refusing.
Trying to delegate more, I find three common scenarios.
I ask someone to do something, and it happens. These people are gold dust – keep them onside.
I ask someone to do something, and they do not respond. I chase it up, but in the end do it myself / ask someone more reliable.
I ask someone to do something, and they do not respond. It is a small task, which means I don’t necessarily remember to follow it up. It just doesn’t get done. Down the road, this causes problems.
In the worse case, delegating takes more time than it would to do it myself. In the best case, I save myself time and give someone who deserves it more responsibility. In the best case, I leverage and more gets done.
The question: how to maximize the best-case scenario, and minimize the worst-case?
So these have been on a bit of a back burner lately, with the end of the semester and associated craziness. However I had a suggestion from Treena that’s been sitting in my inbox for a while. She suggested I try @erinblaskie‘s lists, here’s hoping they show more of what I think this visualization will be useful for bringing out (lists that represent actual mini-communities, rather than just grouping people you follow).
First up: metinreallife. After I graphed this for the first time, I removed an outlier who was following/followed by a ton of people, causing every other point to clump together at the bottom left of the axes. removing it improved things somewhat, as you can see below. It’s noticeable that the more engaged people in the list (in terms of conversations) are not those with the most followers. You can get to the interactive version by clicking on the image (for any of the graphs below).
Next: interesting. There were no conversations in this, though, so I decided not to graph it.
Third: askerinlive. Again, there were few conversations in this, so I didn’t graph it.
I’m really looking for lists that represent communities, and perhaps a better way to go about this is to look for lists with more people following them. Erin’s most followed list is one for Ottawa, but that has 500 people in it. Intuitively, I’m looking for lists with a good ratio of people following them to the people in the list.
Let’s try geekylikeme: Following 155, followers 18. This one is better, but the outliers make it really hard to read. I wonder if it’s better to do it by ratio of followers/following plotted against number of mentions. I’d like to try this on a logarithmic scale, but ManyEyes does not support it. Really, I want more control over the graph which does not appear to be supported.
The last list I’m going to try is entrepreneurship, Following 107, followers 14. Again, the points with more influence within the list (more conversation) are clustered in the bottom left corner.
What have I learned from this?
Outliers are rarely the most influential in a list. Interaction is probably limited by followers/following – when very popular interaction will be low proportionally out of necessity.
I’m not looking for lists of celebrities (or wannabe celebrities), I’m looking for lists that represent communities. Thus the GGDOttawa list is the best I’ve found so far.
ManyEyes does not give me all the functionality I want, for example logarithmic scales, and it’s hard to remove outlying data-points (lots of clicks). Going to try Google Widgets next.
Lately, things have been somewhat chaotic. I don’t like it. It makes me stressed, overwhelmed, and unproductive.
Credit: flikr / kevindooley
On Saturday, Treena and I headed out of town for breakfast and a chat. We caught up, and I was talking about how I had just hit this point where I was so overwhelmed I was having a hard time being productive. I’m trying to get a grip; I’ve managed to delegate something that was causing me a giant headache and I’ve been trying to do more things that make me happy rather than I feel I should do (this means I’ve finally caught up on this season of Ugly Betty – love that show).
Credit flikr / flashcurd
However, it’s not enough. At the end of the semester… I think the picture below captures it. It’s like when the snow melts and everything you’ve done all semester needs to be done and final. There’s a cascade of stress, as anything that takes longer than anticipated slides into everything else…
Credit: flikr / mint imperial
Treena tells me (I’m paraphrasing here):
In production, you always schedule at 80% of capacity just in case.
I try to say that I do, it’s just more has gone wrong than the 20% allocated for. Maybe I’m right – I mean, over 4 hours a week spent on physio at the start of term… there’s 20% right there. Having to remark a whole assignment? That’s 20% and it’s happened twice.
Then later, I think about it some more, and realize – I don’t know what my capacity is anymore. Some weeks I’ll work 80 hours and be OK with that. Last week I didn’t achieve anywhere near that (I tried to, but I was having terrible problems focusing). I’ve hit the point where my cup is overflowing – and not in a good way.
Credit: flikr / 96dpi
So Treena is right – I’ve not been scheduling at 80% of capacity. The fact that I don’t even know what my capacity is anymore, tells me I’ve really screwed things up – I’ve sprinted and crashed. I need to be doing 50 hours every week, not 80 hours one week and 20 the next. I shouldn’t be at the point where what needs to happen this week makes me want to curl into a ball and cry.
Credit: flikr / hufse
It’s probably too late for this semester. If I can get through this week, it’s over. For next semester, what can I do to learn what my 80% is and schedule for that?
Working Saturdays – don’t do it. Saturday lunchtime labs screw up my whole weekend, they’ve often overran as well as assignments have been due on Sundays and I feel compelled to stay longer to help.
TA-ing period. No TA-ing in French (it’s much more stressful for me), and if I TA at all it will be a “proper” CS course as the obligatory courses for non-CS students are much harder.
Delegate before it’s panicking me. I arranged for someone to take charge of something last week that I probably should have arranged a month ago.
Better sleep schedule. I was up early for the first chunk of the semester, but then I work late into the evening and it spills over to the following day when I sleep late… need to avoid this and keep on a more regular schedule – especially since the morning is often my most productive time.
Do things that make me happy. Read more novels. Go do things I enjoy. Spend time with my boyfriend. I’m 24 – it’s too young to do nothing but work.
Email. Takes too much time. Unsubscribe from everything I can. This will include Twitter notifications. I should make a custom Twitter landing page indicating that I don’t check the notifications and that I mostly follow back people who talk to me, so send me an @ message saying hello. Once the end of the semester is over (no more panicked emails from students) I should be able to check it just once a day. Try and move to inbox zero.
Courses. Take a course that I enjoy and am interested in. This semester’s course was one I had to take, which definitely made me less motivated. That kind of workload in something I’m more passionate about would not be as big a problem. Spend more time at the beginning of the semester going to a few courses and picking the one that I will enjoy most – this will pay off later.
Some tasks get bigger the longer you put them off. Last week, I spent several hours trying to clear my email. On my desk, there’s a pile of paper 6 inches high. At this point, they become so large I need to set aside a lot of time to deal with them. This makes them much more intimidating, and I put them off even longer… it’s a vicious cycle. Try not to get into it in the first place.
This is everything I can think of for now, but as I try to find my 80% no doubt more will come up. How about you? How do you find your 80%?
Clear email… it’s got completely out of hand (worse than ever before) – made a lot of progress on this, but still lots left
Social Capital, Trust Agents and the New Tribe event
Students and Startups
An Introduction to Processing – make slide-deck.
Marking (again)
Gym 4 times
Recreate conversational graphs from re-factored code – Nearly done
Implement clique-finding algorithm for Twitter Conversation Graphs
Re-read 7 Habits – Got started, part way through
Get started on giant pile of papers.
For next week, I fly back to the UK on Sunday so I’m going to have to focus on the stuff that must be done by then and not crowd my task list with things that are important, but can wait until I’m there/get back.
Recreate conversational graphs from re-factored code
Implement clique finding algorithm for Twitter Conversation Graphs
Visualize results of clique finding
Create presentation for class on Cliques/Conversation Graphs