I started using a laptop used by a previous member of them which has 64-bit Ubuntu 12.04 and it seemed fine. Since I'm OK with the existing operating system version and stuff, I decided not to format the hard drive instead keep working on the existing system. However, after a while, I wanted to run some Java program and then realized that the Java versions in this laptop has some issue. Now I can't exactly remember the problem, however I remember the stupid solution I did to remove those existing Java versions. I went into /usr/lib/jvm directory and deleted everything inside it using the rm command. Then I tried to install Java from the beginning but started to receive weired errors.
At the end, I realized that deleting the content inside that directory have messed up everything to a level where its hard to fix everything. Since I was busy, I decided to put it aside and use a VMWare guest instance of Ubuntu on top of my host Ubuntu system to run the required Java programs. Finally today I received an opportunity to search the web with a fresh mind to find a solution. I found a working solution which I followed blindly to get my system fixed however I think its worth writing down the steps I followed. The person who had given that answer is Eric Carvalho in AskUbuntu forum. This is the link to that forum question where he had answered.
Following are the steps to remove the messed up Java related packages from my system as I got to know from the above forum. Honestly, I don't understand the functionality of most of those commands but they worked somehow.
sudo apt-get update
apt-cache search java | awk '{print($1)}' | grep -E -e '^(ia32-)?(sun|oracle)-java' -e '^openjdk-' -e '^icedtea' -e '^(default|gcj)-j(re|dk)' -e '^gcj-(.*)-j(re|dk)' -e 'java-common' | xargs sudo apt-get -y remove
sudo apt-get -y autoremove
dpkg -l | grep ^rc | awk '{print($2)}' | xargs sudo apt-get -y purge
sudo bash -c 'ls -d /home/*/.java' | xargs sudo rm -rf
sudo rm -rf /usr/lib/jvm/*
for g in ControlPanel java java_vm javaws jcontrol jexec keytool mozilla-javaplugin.so orbd pack200 policytool rmid rmiregistry servertool tnameserv unpack200 appletviewer apt extcheck HtmlConverter idlj jar jarsigner javac javadoc javah javap jconsole jdb jhat jinfo jmap jps jrunscript jsadebugd jstack jstat jstatd native2ascii rmic schemagen serialver wsgen wsimport xjc xulrunner-1.9-javaplugin.so; do sudo update-alternatives --remove-all $g; done
sudo updatedb
sudo locate -b '\pack200'
apt-cache search java | awk '{print($1)}' | grep -E -e '^(ia32-)?(sun|oracle)-java' -e '^openjdk-' -e '^icedtea' -e '^(default|gcj)-j(re|dk)' -e '^gcj-(.*)-j(re|dk)' -e 'java-common' | xargs sudo apt-get -y remove
sudo apt-get -y autoremove
dpkg -l | grep ^rc | awk '{print($2)}' | xargs sudo apt-get -y purge
sudo bash -c 'ls -d /home/*/.java' | xargs sudo rm -rf
sudo rm -rf /usr/lib/jvm/*
for g in ControlPanel java java_vm javaws jcontrol jexec keytool mozilla-javaplugin.so orbd pack200 policytool rmid rmiregistry servertool tnameserv unpack200 appletviewer apt extcheck HtmlConverter idlj jar jarsigner javac javadoc javah javap jconsole jdb jhat jinfo jmap jps jrunscript jsadebugd jstack jstat jstatd native2ascii rmic schemagen serialver wsgen wsimport xjc xulrunner-1.9-javaplugin.so; do sudo update-alternatives --remove-all $g; done
sudo updatedb
sudo locate -b '\pack200'
Finally, I installed Java in my system using the following command and everything worked fine ever after.
sudo apt-get install openjdk-7-jdk
I realized how big the mess I had made by simply deleting the content of my /usr/lib/jvm/ directory which I will never do again. :-)