Caving into publisher pressure, Google News has launched a news-specific crawler that lets online publishers automatically keep stories, photos or videos out of its index.
The announcement comes after News Limited chairman Rupert Murdoch blasted Google and other news aggregators for “stealing” stories and allowing others to reproduce those stories online without consent. Murdoch also threatened to block Google from indexing his newspapers.
Google senior business product manager Josh Cohen said in a blog post that there are very few publishers who choose to exclude their material from Google, but they respect those who wish to do so.
“Most people put their content on the web because they want it to be found…but we respect publishers’ wishes. If publishers don’t want their websites to appear in web search results or in Google News, we want to give them easy ways to remove it,” he wrote.
The new system will let publishers limit the number of online pages people can view after being routed to their websites by Google’s search engine.
“Previously, each click from a user would be treated as free,” Cohen wrote. “Now, we’ve updated the program so that publishers can limit users to no more than five pages per day without registering or subscribing.”