One liner to find and remove duplicate files in Linux

I recently found a one-liner to report all duplicate files under the current directory and its subdirectories here. The command is as follows –

find -not -empty -type f -printf "%s\n" | sort -rn | uniq -d | xargs -I{} -n1 find -type f -size {}c -print0 | xargs -0 md5sum | sort | uniq -w32 --all-repeated=separate

It first compares size and then compares md5 hash in order to find duplicate files. Since this one just reports and doesnt delete the files, I’ve made slight modifications to find and DELETE duplicate files as well. Don’t worry, it’ll ask your permission before running the delete command over all files. Here it goes –

find -not -empty -type f -printf "%s\n" | sort -rn | uniq -d |  xargs -I{} -n1 find -type f -size {}c -print0 | xargs -0 md5sum | sort | uniq -w32 --all-repeated=separate | cut -f3-100 -d ' ' | tr '\n.' '\t.' | sed 's/\t\t/\n/g' | cut -f2-100 | tr '\t' '\n' | perl -i -pe 's/([ (){}-])/\\$1/g' | perl -i -pe 's/'\''/\\'\''/g' | xargs -pr rm -v

The modifications are very boring, but that’s all I could do.  Have a better solution? Let me know. If you want to delete files without asking permission, remove the -p after last xargs in the above command.

Have Fun :).

PS: The command is primarily to be used for deleting duplicate media files (mp3, videos, images etc.). Please dont run it on any sensitive system directory.