What Search Tells Us About How To Run a Business
One of the most ironic experiences of working at a web search company is just how awful our internal search is. People just assume that if you can build a great web search engine that delivers highly relevant results, you’d of course have a great search engine for your company’s intranet that delivers highly relevant results. It turns out that there is almost no correlation between the two, and the primary reason for this should be making companies rethink how they go about their business.
If you talk to the experts, this is not a case of the cobbler’s children having the worst shoes. The primary reason that a search engine that provides highly relevant results might not do so when targetting an intranet is that intranets do not have the extensive cross linking that you find on the web. Without this, Google’s famous PageRank becomes mostly useless.
Basically, employees tend not to spend a lot of time generating web pages, and the pages that are generated tend to have limited cross-linking between groups/projects/etc. Consequently, a typical corporate intranet has very little meta data to help a search engine out.
Here is the crazy part: what if you took those same intranet pages and put them up publicly on the web. I’d bet you’d find some tiny fragment of the web would start linking to various pages and voila! suddenly search engines can start doing a great job of showing highly relevant pages.
Of course, companies don’t publish their intranets because there is all kinds of valuable proprietary content there that they don’t want to share with the rest of the world. That makes a lot of sense until you’ve spent some time working in a reasonably sized company. Then you discover something else: most of a company’s intranet is not made up of valuable proprietary content. Sure, the information is useful, but it doesn’t necessarily provide a competitive advantage. However, most companies (and particularly those in my line of work) work on the assumption that by default all internally generated information should be kept proprietary, because IP is so valuable and a secret, once shared, can never be a secret again.
That mentality makes a lot of sense, particularly once you involve lawyers and phrases like “fiduciary responsibility”. However, I think it is rooted in an old school mentality that fails to recognize that there are huge benefits to making said information available to the public. Just given my intranet example… how much more valuable would it be for a company’s intranet content to actually be searchable, with truly relevant results coming out on top? How much value would there be in seeing what the public finds most interesting about this data and what meta data they assign to it?
Open source software is probably the best example of this new reality. The vast majority of software out there doesn’t provide competitive advantages to companies, but merely helps them to get things done. A lot of it is written in house and therefore by default kept proprietary. Occassionally though, you see companies (out of wisdom or ignorance) make their code available to the rest of the world. The outcomes tend to be as follows: a) nobody else cares about this problem, so the code just lies there, b) someone else has built a better solution, and so nobody uses the code and it just lies there, or c) people start using the code, and magically the code becomes ported to multiple platforms, new features get added, bugs get fixed, etc. None of those outcomes leave you worse off than when you started, but the upside of option c) is huge.
A friend of mine put up a site a while ago that tried to fully explain a problem in the web server business. At some point he remarked to me that the best way to become an expert on a subject is to publish a web page on the subject along with some contact info, and then wait for the feedback to roll in. Pretty soon you’ll have all the relevant data on the subject sent to your inbox. Voila! Instant expert. This is basically what Wikipedia is today.
I am starting to suspect that just like innovations before it (the GUI, the personal computer, e-mail, etc.), the corporate world has embraced the Internet in the last decade, but needs another decade to really understand how it changes how business works, and this may be the critical realization: information becomes more valuable if you make it freely available without restriction. That flies in the face of decades of “silo” mentalities that you find in corporations, where sharing information even within the company is seen as exposing you to risk rather than providing you with a benefit. The notion of sharing data on an even wider basis is heresy.