Arjan's World: Quick Tip: Finding The Amount Of Duplicate Lines In A Text File
You are now being redirected to the new housing of Arjan's World. Click here in case nothing happens

Thursday, September 13, 2007

Quick Tip: Finding The Amount Of Duplicate Lines In A Text File

Here's one I didn't know until yesterday (if you didn't too, this will come in handy some day): in need for a way to see how many duplicate lines there are in a big text file, I felt a small utility coming up. Of course this always comes with a loss in time in which you could do something different...
till a colleague told me how UltraEdit (and who knows how many more text editors) is capable of doing exactly this:
  • Open a file, check the line count

  • Go to '/File/Sort/Advanced Sort Options'. Check 'remove duplicates'

  • Press 'sort'
wait... (depending on the size of the file this can take a while, obviously)
  • Check the new line count


That's all. No programming involved. It's always cool not having to write something yourself if it's already there.

One small thing though: UltraEdit put its' temp files directly in the C:\ root. Bad practice, and even dangerous if your C partition is small, and files are big...

0 Comments:

Post a Comment

<< Home