Remove Duplicate Lines
Remove duplicate lines from a list, with options to trim, ignore case, and sort.
Paste a list and strip out repeated lines. Handy for cleaning email lists, keyword sets, log lines, or any data with one item per line. Options let you ignore case, trim surrounding spaces, and sort the result.
How the options work
Trim whitespace treats "cat" and " cat " as the same line. Ignore case treats "Cat" and "cat" as duplicates. Drop blank lines removes empty rows entirely. Sort A→Z orders the surviving lines alphabetically. The status line reports how many duplicates were removed.
Cleaning real-world lists
Line-based data gets messy fast: an exported email list with the same address in three cases, a keyword file with trailing spaces, or merged logs with repeated rows. The options here handle each cause. Trim whitespace catches entries that differ only by stray spaces; ignore case collapses "Info@site.com" and "info@site.com" into one; drop blank lines clears the gaps left after removal; and sort gives you a tidy, ordered result. The status line tells you exactly how many rows were removed so you can sanity-check the outcome.
Where duplicate lines come from
Duplicates rarely appear on purpose. They creep in when you paste several exports on top of one another, when a spreadsheet column is copied with trailing blanks, or when two teammates merge their own copies of the same list. Email lists gathered from multiple signup forms are a classic case — the same address arrives three or four times with different capitalisation. Server logs repeat identical request lines during retry storms. Because the human eye skims long lists, these repeats survive right up until they cause a double-charged customer or a skewed count.
The fix is mechanical, which is exactly why a tool does it better than manual review. Sorting first can help you spot clusters, but sorting also destroys any meaningful order your list already had — chronological logs, ranked results, a curated playlist. This tool removes repeats in place, keeping the first occurrence and the original sequence intact, so you clean the data without reshuffling it. If you also need alphabetical order, sort afterwards in your editor; keeping the two steps separate leaves you in control of both.
Case, whitespace, and what counts as a duplicate
"Duplicate" is less obvious than it sounds. Are Apple and apple the same line? Does a trailing space make two otherwise identical lines different? For an email list you usually want case-insensitive matching so the same address isn't kept twice, while for code or configuration you often want exact matching so nothing is silently merged. Invisible trailing whitespace is the classic reason a list "won't dedupe" — the lines look identical but aren't. Trimming each line before comparing removes that trap and gives you the clean, unique set you expected.
Frequently asked questions
Does it keep the original order?
Yes, unless you enable sorting. Without sort, the first occurrence of each line is kept in its original position.
Can it ignore differences in capitalisation?
Yes. Turn on 'Ignore case' to treat lines that differ only by letter case as duplicates.
Does removing duplicates keep the first or last occurrence?
The first. Each line is kept the first time it appears, in its original position, unless you enable sorting.
Can it deduplicate case-insensitively but keep original capitalisation?
Yes. With 'ignore case' on, lines are matched without regard to case, but the first version is kept exactly as you typed it.
Does it treat different capitalisation as duplicates?
By default lines must match exactly, so 'Apple' and 'apple' are kept as two lines. Lowercase your list first if you want case-insensitive de-duplication.
Last updated: 2026-01-15