RDig
RDig is a standalone full text indexer and crawler, similar to HtDig. It can crawl a website via HTTP or index a set of documents on a local file system.
It’s implemented in Ruby and uses Ferret for the indexing. RDig is available as a gem, so you can use
gem install rdig
to install it.