How Much Should You Be Spending on data integration?

https://www.instapaper.com/read/1726011466

This allows Goggle (person-agent identify GoogleBot) to visit each site and directory, while concurrently banning Inquire Jeeves through the web site entirely