A few years ago, it was not uncommon to hear search people talk about The Dark Web (also known as Dark Net or the Deep Web). Basically, the Dark Web is made up pages or files that are unreachable by search engines. Examples of this sort of content include information that sits behind login, or pages without inbound links. Some estimates put the Dark Web at about 400 times the size of the Surface Web (content indexed by search engines).
Google has invested tremendous resources in lighting up, or indexing these hidden pieces of content. An example of their efforts was their 2008 announcement that they could now index the contents of .PDF files, a “feat that requires and an immense amount of processing power.”
Why would Google do this? The short answer is fear. The longer that Google allowed such a massive amount of content to go unindexed, the more exposed they were to potential competitors able to light up and organize this information.
Flash forward to 2011. We are now witnessing a similar dynamic, but the battlefield has shifted from search to social. Just as Google had early dominance in lighting up a portion of the web, Facebook has early dominance in lighting up a portion of the world’s social graph. But much like the Dark Web, there exists network upon network not yet graphed by Facebook, waiting to be mapped, organized, and optimized for communication.
This is the unlit social graph, and this is where Facebook is vulnerable.
Let’s talk examples.
For years I have been looking for a solution to the pick-up basketball problem. I have a large-ish network of people that I play hoops with in San Francisco. This network has not yet been lit up by any online service. While most of these people are on Facebook, they are hard to organize as I don’t know many of their last names. And furthermore, even if I did know their last names, I would feel awkward friending them on Facebook, as they’re not really my friends.
It’s a network, but it’s not a friend network, not a professional network, and not a work network. This particular network is a place based network, aligned around various basketball courts in San Francisco.
Over the years, we’ve tried to light up our hoops network, but haven’t been able to find the right tools. We tried Google Groups, but it got overrun by spammers. A Facebook Group set up for this purpose never really got traction.
We tried GroupMe, but the push aspect wasn’t appropriate, it needed to be pull – find ballers when you’re ready to play, not when others are ready to play.
Just last week, I read about a company called Sportaneous that is trying to solve this problem.
Ubiquitous smartphones and always on access to umbrella social graphs are suddenly making these sort of tools possible.
And the opportunity is far larger than pick-up basketball, or even sports. Every school is a network, every employer is a network, every bar is a network, every office building is a network, every hobby is a network, every neighborhood is a network, and at an extreme level, every shared interest is a network, regardless of location.
This doesn’t even get at the disposable, or elastic networks as discussed by companies like Nearverse and Color – people that happen to just be nearby each other for a snapshot of time.
All of these networks share two common characteristics. 1) They are not yet graphed in a mainstream way by Facebook, Twitter, LinkedIn, Jive, or any other dominant, online social service; and 2) They are all mappable with a smartphone.
Much like Google moved decisively to index huge swathes of the Dark Web, Facebook will almost certainly look at ways to better map some of the relationships discussed in this article.
But that doesn’t mean that there aren’t huge opportunities for thousands of startups.
As an entrepreneur, there is a lot to be excited about right now. But for me it doesn’t get more exciting than finding and mapping the unlit social graph.















Welcome to ————— – http://www.2kuu.com are specialized in offering all kinds of top brand shoes, jeans, t- shirts, bikini, swimwear, jacket, jerseys, watches, purses, handbags, belts, wallets , sunglasses and hats etc.Accept paypal ,All the prices list on our website include shipping cost,insurance, tax etc..$30 nike shoes,air jordan shoes,nike shox shoes,gucci shoes$35 true religion jeans, ed hardy jeans,coogi jeans,afflictio n jeans, Laguna Beach Jeans$14 ed hardy T-shirts,Coogi T-shirts,Christ ian Audigier T-shirts,Gucci T-shirts,Polo T-shirts.$35 coach handbag,gucci handbag,prada handbag,chanel handbag.
Lawrence you nicely highlight that ‘social networks’ have been around forever and how the current suite of popular tools fail to really capture their needs, however, you missed an 3 characteristic and that is that they really isn’t a burning need to be linked through technology for a ‘location’ based social network.
@David Cooper Thanks David. “Burning need” is awfully tricky to predict.
@David Cooper I agree with David. Apparently it is earth-shattering news that people are actually capable of such location-based activities as walking around at dusk and talking with neighbors on their front porches, meeting guys for pickup games of b-ball at the playground, visiting the supermarket, etc. without a suite of web-based applications, a social network, a huge untamed database of visitors, tweets, coupons and a bloody GPS to map it all out. Not only is there no burning need, there’s not even the faintest need at all. At a certain point (and that point has been reached) it’s time to put away your electronic devices and get a breath of fresh air. Apparently there’s not enough oxygen in the conference rooms where otherwise thoughtful people are caught up trying to tattoo a dragon on a mouse so the cat can find it.
@nigel2011 @David Cooper If I may, Nigel: you tend to forget that most kids now live with their smartphone in hand, being connected is as natural for them as breathing, they don’t have a need for a breath of fresh air. Social media are part of their everyday routine and they don’t feel overwhelmed by technology. Therefore, also they can chat and play games with friends from their neighbourhood without a need for social media, they might as well use it, it’s no effort.
@nigel2011 @David Cooper “put away your electronic devices and get a breath of fresh air.”
This is absolutely good advice. I try and do it from time to time myself.
Lawrence, friends -
We at Blipboard couldn’t agree more. We’re delivering a new kind of social network that connects you with all the people around you whom you don’t know yet, but should. Blipboard leverages the combination of your interests (via tags) and your location to connect you with local networks relevant to your interests, wherever you go.
Check out Blipboard – we’d love to hear your thoughts on it.
http://blipboard.com/user/register?invitecode=peter2011
Thanks,
- Peter
@Peter Schurman Peter, checked it out last night. Looks like a good premise. Are you in SF?
@Lawrence Coburn Hi Lawrence – Thanks for your interest. Yes, we are here in SF. – Peter <peter@blipboard.com>
@Peter Schurman the link and code are not working, submitted my request for an invitation, I’d like to get on the Blipboard :)
@neenz Thanks for pointing this out. We’ve corrected the issue and this link should now work to get you all the way in:
http://blipboard.com/user/register?invitecode=peter2011
@Peter Schurman Thank you Peter, I’m in! I also invited more folks from Hawaii so I can connect :)
Lawrence – Great article. What you point to as unlit networks, work, shared interests, pick-up basketball, I think can largely be classified as “shared experiences.” Whether you work in the same place (LinkedIn/Yobongo), have shared a snapshot in place-time (Color), or both watched the latest Bulls-Celtics game on TV (Hot Potato), you are connected in some way that is not yet captured.
I think group messaging startups like Beluga, GroupMe, and us (Foxfly) are aiming at capturing this. I think GroupMe’s joinable groups is a good start, Beluga has group location/time settings, and we are also planning to rollout local groups.
- Robert Hsiung
@foxfly Robert, I absolutely think the group messaging startups are emerging, with momentum, because of this opportunity.
This is an earnest enough piece, but could there be any further evidence of an insane bubble in webland than this? I don’t play street ball but I’ve never known anyone who does who had trouble finding a pickup game. There doesn’t need to be a web-based solution to this “problem” any more than there needs to be one for finding a gas station or a cold beer. Let me provide an elegant solution: 1. Go to basketball court after school or work, with basketball. 2. Dribble and start shooting. 3. Spend 3 seconds organizing game with others who happen to show up.
What’s next? A web-based solution to the problem of how to locate the nearest roll of toilet tissue? I don’t mean to be crass but the article’s premise is positively ludicrous.
@nigel2011 “3. Spend 3 seconds organizing the game with others who happen to show up.”
This is where your advice fails :) Smartphones and networks provide a better way.
Lawrence is spot on here as these corners of the Web are seeing the light of day. I co-founded a network for hobbyists that presents a set of tools that makes way more sense to these individuals than a FB group or any of the other solutions mentioned in the article.
I think these micro networks are going to see more and more attention as the bigger generic networks lack specific experience features. We at Collectdat.com are more than happy to carve out our little corner of the social graph and I am sure others will be eager as well.
@Kevin Scott nicely put.
The question that I’ve been asking myself, is, what’s going to happen from an SEO perspective with iFrame Facebook Page tabs now that FBML has been deprecated?
If an organization creates its own Facebook apps that point to public, 520px wide Web pages which, in turn, are presented within Page tab iFrames, then it’s not only the Facebook Page that gets indexed. Google will crawl and index the public Web pages — so organic search will point to the “narrow” pages.
So, does one noindex the framed pages? Does one have a “view in Facebook” button that redirects to the Facebook page? (the button, of course, would also be visible in the framed Web page within the Facebook Page and therefore be self-referential)
@Steve Chipman Make sure to use 301 redirects. Also, beware the duplicate content penalty.
H’m… make it GPS attached, and put up a message, ‘Pickup game at these coordinates at X hour. ‘ People could ‘subscribe’ to that coordinate- be it a court, or a bar, or whatever, and voila- flash game, mob, party, etc.
People could select from a list of coordinates to ‘follow’ a’ la Twitter, and light up a whole new ballgame.
@Lorie Johnson now you’re talking my language
Great article and discussion about the unlit social graph. For me, as a social media practioner and advocator for all things social, it’s more evidence that Facebook, Twitter, and the likes are not the utopia we’re led to believe. While hundreds of millions are on some social network, hundreds of millions more connect and socialize in other ways — that community is powerful, that community needs to be reached, that community can further change the world. I too am excited about the challenge of exploring, discovering, and mapping the unlit social graph. Let’s start at the parks! :)
Great article – spot on! This is exactly what INCLIQ is all about – you hit the nail on the head. Check out INCLIQ, the beta program is opening up.
@INCLIQ – Are you in? you guys should fix that “reserve your username!” bit until you can actually reserve your username. Ugly first impression.
Hey Lawrence, Fubles is the online service you are looking for. It organizes networks of people who share a passion for a sport and who want to find more pals to play with. http://www.fubles.com/
thanks for the heads up
Well INCLIQ, from a site that talks about social freedom, I don’t expect a box telling you to reserve your username that signs you up to a newsletter from mailchimp. I think facebook started with alot more honesty than that. I will be following diaspora https://joindiaspora.com/) as I have unlimited more faith in the developers than you guys after finding out about you 5 minutes ago!
@Rob Went I found that to be annoying as well. The old bait and switch.
Very interesting article. I’d like to add http://www.webdoc.com to the list of services that has the potential to light up some of these dark networks. We designed it around the idea of bringing people together in a spontaneous way around things they have in common or find of interest such as people, places, events.
webdocs are rich two-way conversations that are simple to create and can be shared and embedded all over the web. So you can get the word out yet have a self-contained place for people who want to participate on a deeper level.
If you have time to check it out, we’d love to know what you think.
@doctorlaura cool, i will check it out.
Hi Lawrence, I agree that there is a vast unlit social graph. I’ve co-founded UKourt.com to bring together ballers through a user-generated court directory, player rating system, and check-in system. We’re also releasing a league management system that will revolutionize the ways leagues operate and give back to their players. Would you be interested in connecting? I’d love to get your input on our vision. We’re currently based out of Palo Alto, CA.
How about Training Mobs? Sounds like it’s exactly the solution to your basketball issue and they’ve been thinking up several ways to make it work. (Semi-disclaimer: I know one of the founders through my favourite café, which happens to be where I’m sitting now.)
@Alexandre Enkerli which cafe?
I wanna play some ball! back in amsterdam we tried a host of things. facebook group for the nodes in the network who were already facebook friends – kinda worked. twitter – did NOT work. but yea, I wanna see a curated map of the courts around town and where all to go what times etc.
hi Lawrence, thanks for pointing out some of the shortcomings with Facebook and other mass online social services as it relates to “unlit” networks. While there are some that would live and die on Facebook, Facebook and other mass sites (including Ning) are too “coarse” to serve a niche need.
I run a site called http://www.privy.net/, which is an invite-only transpacific network and online/mobile travel platform that helps Asia jet-setters coordinate roving itineraries and share insider city tips in a filtered and trusted environment..
@stefan Thanks for bringing up Ning. I’m sure they would read this post and throw up their arms. They bet on this trend very, very early and my sense is that it hasn’t worked out to the scale that they had hoped. Not sure if it’s execution, web first vs mobile first, timing, or if the premise is just flat out wrong.
Good post. In addition to these different levels of graphs (maybe one way to think of them are different axis or dimensions), is to think about how the existing -or new- layers can themselves be deeper. e.g. think of the friend-friend connection itself as not an adjective, but a noun, which itself has properties (adjectives) and things it can do or be subject to (verbs).
I wrote about this a while back here:
http://www.kimpallister.com/2009/09/its-complicated-or-beating-facebook.html
@Livefyre User Kim, that’s a good read. I like the idea of treating the relationship as the entity, as opposed to just an attribute of a person.