Linux server response times

Before few days I had problem with response times on linux server. I use CentOS. There is apache HTTPD covering about 5 tomcat instances. Inside tomcat instances are 7 different application. Main problem was incredibly slow responses from all pages. When I logged in I seen this:

top - 22:37:01 up 104 days,  7:59,  1 user,  load average: 3.59, 5.09, 3.33
Tasks: 103 total,   1 running, 102 sleeping,   0 stopped,   0 zombie
Cpu(s): 15.7%us,  0.1%sy,  0.0%ni, 84.1%id,  0.1%wa,  0.0%hi,  0.0%si,  0.0%st
Mem:   1018236k total,   988976k used,    29260k free,     1012k buffers
Swap:  2064344k total,  1031440k used,  1032904k free,    20592k cached

meaning of wa is waiting for I/O to complete. I also tried to order processes in top command by used memory "shift + f" and press n and than Enter. After that I run following:

ps axf show among other following problem:

 5627 ?        Ss     0:03 /usr/sbin/httpd
...
10289 ?        D      0:00  \_ /usr/sbin/httpd

It says that HTTPD is waiting for finishing disk IO operation. After checking memory consumption 988976k + 1031440k was clear that all tomcats eat about 2 Gb of memory and there is only 1 Gb of physical memory.

Than it was simple I just reconfigured all tomcats, so now I have just two. One is for critical application and second one contains all resting applications. And after restart response times are again good.