Just another tech blog.

A blog abt GNU/Linux, programming, hacking, and my life.

Archive for the ‘Linux’ Category

Some tips and tricks and things I found about Linux operating system.

One liner to find and remove duplicate files in Linux

Posted by ajay on October 16, 2009

I recently found a one-liner to report all duplicate files under the current directory and its subdirectories here. The command is as follows –

find -not -empty -type f -printf "%s\n" | sort -rn | uniq -d | xargs -I{} -n1 find -type f -size {}c -print0 | xargs -0 md5sum | sort | uniq -w32 --all-repeated=separate

It first compares size and then compares md5 hash in order to find duplicate files. Since this one just reports and doesnt delete the files, I’ve made slight modifications to find and DELETE duplicate files as well. Don’t worry, it’ll ask your permission before running the delete command over all files. Here it goes –

find -not -empty -type f -printf "%s\n" | sort -rn | uniq -d |  xargs -I{} -n1 find -type f -size {}c -print0 | xargs -0 md5sum | sort | uniq -w32 --all-repeated=separate | cut -f3-100 -d ' ' | tr '\n.' '\t.' | sed 's/\t\t/\n/g' | cut -f2-100 | tr '\t' '\n' | perl -i -pe 's/([ (){}-])/\\$1/g' | perl -i -pe 's/'\''/\\'\''/g' | xargs -pr rm -v

The modifications are very boring, but that’s all I could do.¬† Have a better solution? Let me know. If you want to delete files without asking permission, remove the -p after last xargs in the above command.

Have Fun :).

PS: The command is primarily to be used for deleting duplicate media files (mp3, videos, images etc.). Please dont run it on any sensitive system directory.

Advertisements

Posted in Linux | 37 Comments »

Totem – the complete media player

Posted by ajay on June 1, 2006

Many of the Linux distro’s ship in totem as a movie player with them. Its there in Applications=>Sound and Media=>Movie Player. But due to some copyright problems they dont provide support for most of the file formats, not even mp3. They do provide support for flac and ogg vorbis formats but no body listens to them as they are not common. So it happens that this dumb player seems to play nothing on the system. I recently figured out that we can configure it to play any file format except rm and rmvb (I dont know the plugins to play rm and rmvb but they also can be played) . You just have to provide totem back-end in terms of codecs for different formats. You have two options to do the same – First and the easy one is Xine-lib, and the second one is gstreamer. Xine-lib seems to have some problem so that the player crashes some times and besides gstreamer has other benifits so I go for the second option.

You need the rpms regarding gstreamer. Here are the links for Fedora Core 5 on i386 architecture¬† (the one which I have) –
gstreamer08-0.8.12-4.fc5.i386.rpm
gstreamer08-ffmpeg-0.8.7-1.fc5.i386.rpm
gstreamer08-monkeysaudio-0.8.2-1.fc5.i386.rpm
gstreamer08-plugins-extra-audio-0.8.8-3.fc5.i386.rpm
gstreamer08-plugins-extra-dvd-0.8.8-3.fc5.i386.rpm
gstreamer08-plugins-extra-video-0.8.8-3.fc5.i386.rpm
gstreamer-plugins-bad-0.10.1-2.fc5.i386.rpm
gstreamer-plugins-ugly-0.10.3-1.fc5.i386.rpm


or if you are using any other linux or any other architecture you can go to freshrpms or might try googling or another option (will work only if you have yum configured properly)

[user@localhost.localadmin]yum -y install gstreamer* [enter] ..

And it will do the rest for you.

If you downloaded the rpms with previous method, you will have to install them using

[user@localhost.localadmin]rpm -ivh gstreamer* [enter]

Thats it. Now your totem player is ready to play anything.

Posted in Linux | 5 Comments »

Login in ssh without a password

Posted by ajay on June 1, 2006

Many of you guys have different accounts on different systems on linux .. and you might log on from one system to other system regularly . like me. But each time you login it asks for a password and some of you might not want this as you have to type the password each and every time and this is unnecessary effort . So here is a method you can use for login in ssh shells without a password .

Suppose you want to login from system A to system B on user X. Then you might try these commands .. But you need your password of user X at system B once .. if you have it then type the following commands on system A ..

[user@A]ssh-keygen -t rsa [enter]

RSA here is the type of encryption used .. you can have RSA as well as DSA on most standard unix systems.

it will give the following output

Generating public/private rsa key pair.

Enter file in which to save the key (/home/ajay/.ssh/id_rsa): [enter]

Enter passphrase (empty for no passphrase):[enter]

Enter same passphrase again:[enter]

Your identification has been saved in id_rsa.

our public key has been saved in id_rsa.pub.

The key fingerprint is:

ef:b8:fd:d8:de:63:d2:f0:ea:3b:ba:1b:ad:e9:b6:ce user@localhost.localdomain

[user@A]

dont worry about it now run the command

[user@A]scp ~/.ssh/id_rsa.pub X@B:.ssh/authorized_keys2 [enter]

X@B's password: ******* [enter]

give it the password ..

you are done .. now each time you login frm system A to system B using user X it wont ask you for password .. But be careful as someone else might login to ur private account on B once it gets access to A ..Nyways yeah it saves a lot of work specially for those who keep long and strong passwords for their accounts ..

Posted in Linux | Leave a Comment »

Problems in file transfer through scp

Posted by ajay on June 1, 2006

Well many of u guys use scp for transferring ur files over network.The problem with this is some times when you try to scp over some systems nothing transfers and some crap thing is outputed .The reason of this is that scp is done in 2 parts .Suppose you want to transfer a file named X from machine B to a machine A and you are sitting on machine B. In first part the programmes  log on to the remote machine ( A in this case )  and in second part the files are transfered from machine B to machine A. But when you log on to any system ( A in this case )  the .bashrc or .bash_profile placed in the home directory of that user gets executed and if there are any printing statements in the like echo or nfrm or somethings like that then the second step does not gets exetuted and this command will only dump whatever echo statements are written in the .bashrc of the remote user on machine A. So if you are having the same types of problem with using scp you might try removing all the print statements from the .bashrc of the remote user on machine A. This might work in some cases.

Posted in Linux | Leave a Comment »