Show/Hide Toolbars

WordSmith Tools Manual

Navigation: Utility Programs > Corpus Checker

Aim

Scroll Prev Top Next More

 

ws-48-corupt

 

The purpose is to check whether one or more of your text files in your corpus doesn't belong. This could be because

it has got corrupted so what used to be good text is now just random characters or has got cut much shorter because of disk problems

it isn't even in the same language as the rest of the corpus

it is a copy of another text file or has very similar text

it contains too much boilerplate text

it is relevant to a particular area of enquiry

 

The tool works in any language. It checks corruption by using a known sample of good text (in whatever language) and comparing that good text with all your corpus.

 

See also : How to check corruption