Tika in Action

Chapter 15. The classic search engine example

 

This chapter covers

• Bixo Labs and the Public Terabyte Dataset Project
• Bixo’s crawler that uses Tika
• Charset detection and language identification with Tika

 

What better way to close out the book then the way we started it—with a classic search engine example?

You’re in for a treat. We interviewed Ken Krugler and his team from Bixo labs about their recent Public Terabyte Dataset Project, http://mng.bz/gYOt, and how Tika was a core component of a large-scale series of tests that helped shed some light on variations between languages, charsets, and other content available on the internet.

This chapter will show you even more of Tika in action, especially how you can leverage Tika inside of a ...