Eigendecomposition is better known as diagonalization. While the description above is correct, it does not explain why this is useful, giving rise to the stereotypical answers above.
Instead of looking at matrices, look at the equivalent group of linear transformations that operate on a given vector space V. Given some linear transformation L, we want to know whether L is simply a rescaling of some basis for the space. If that is so, there is some basis for the vector space that consists totally of eigenvectors.
But if that is true, then one can carry out L by simply changing the basis of the space, scaling the eigenvectors, then changing the basis back to the original basis. Thus L = QDQ^(-1) for the change of basis transformation Q and the linear scaling transformation D which in matrix form is obviously just a diagonal matrix with the eigenvalues.
In other words, L being linear means L is completely specified by its action on the basis vectors since L(a(e1) + b(e2)) is just aL(e1) + bL(e2). However, we would like it if L(ei) was just r*ei for some number r for each basis vector (ei), instead of the usual lengthy linear combination of basis vectors. It is conceptually easier to deal with and we can formulate hypotheses and theories easily in such a basis. If this is true, we say that L is diagonalizable. Working with respect to this basis will probably be more enlightening than working with respect to the standard basis, since this is the "natural" basis for L. As such, diagonalization finds plenty of applications in the study of physics.
As a concrete example, consider the linear transformation T that rotates R^2 by pi/2 radians. There is obviously no basis for R^2 that gets rotated into itself by this transformation.
However, consider a rotation by pi radians. This actually maps the standard basis into itself by negating both axes, so in diagonalization, the change of basis transformation is the identity, and the diagonal matrix is the negative identity.
It is normal in special and general relativity to work with curvilinear and skewed coordinate systems. In these cases, it is very important to find the natural coordinate system for the transformation.
Note that the field over which the space is defined is also important. Rotations are obviously not necessarily going to be diagonalizable over R^n, but they are diagonalizable over the field of complex numbers.