What the duplicate word remover does
The tool finds every word in your text and keeps only the first time each word appears. All later copies are removed, along with the space after them, so the text stays tidy.
Example
The text the cat and the dog and the bird becomes the cat and dog bird.
Options
- Ignore case: treats The and the as the same word. This is switched on by default.
- Whole text or each line separately: remove repeats across the entire text, or start again on every line, so each line keeps its own first copies.
- Keep my text or make a list: keep the layout of the text with repeats taken out, or get a list of the unique words, one per line, optionally sorted A to Z.
What counts as a word
A word is a run of letters and numbers. Apostrophes and hyphens inside a word are part of it, so don't and well-known are single words. Punctuation marks are not words, and they stay where they are. The tool works with accented letters and with scripts that use spaces between words, such as Arabic, Cyrillic and Greek.
Things to know
- Removing repeated words changes the meaning of normal sentences, so use the tool for lists of words, keywords and tags rather than for prose.
- For a comma-separated list of tags, make sure each tag is a single word, or put every tag on its own line and use remove duplicate lines instead.
- Chinese and Japanese are written without spaces between words, so they are not split into separate words.
Related tools
To count how often each word appears, use the word frequency counter. Remove whole repeated lines with remove duplicate lines, see which values repeat with unique lines, or order a list with sort lines. The guide to removing duplicates covers other tools and code.