The significance of e cannot be understood without knowing some calculus. One way to define it is with the formula mentioned above, as the limit of the sequence (1 + 1/n)^n, i.e., take out your calculator, and plug in n=10, 100, 1000, 10000, 100000, etc. into the above expression -- you will see that the numbers you get out get closer and closer to 2.718281828... It is true that this is the expression involved in computing the interest added to a bank account if it is "compounded continuously."
Another way to find e is to look at the series:
2 + 1/2 + 1/(2*3) + 1/(2*3*4) + 1/(2*3*4*5) + 1/(2*3*4*5*6) + ..., where what I mean is, add up the first 5 terms of this and see what you get. Then add up the first 10 terms and see what you get. You will find that the more terms you add on, the closer the value gets to 2.718281828..., i.e., to e.
It was named in honor of Leonard Euler (pronounced "Oiler"), a Swiss mathematician who lived in the 1700's and was probably the most prolific mathematician of all time. He published thousands of papers, many of which were seminal results, and he spent the latter years of his life blind, yet still was able to discover tons of new mathematics in his head, and then dictate the results to his servant.
The reason it is important is related to one of the main concepts in calculus, the derivative. Given a function f, i.e., f(x)=x^2, or whatever, you can associate a new function, called its derivative, denoted by f'. Now one may consider all of the so-called exponentical functions f(x)=a^x, where a is some fixed consant. It turns out that the function with a=e=2.718281828... is the nicest of these functions in the setting of calculus, because the associated derivative is itself, i.e., the derivative of e^x is just e^x. One property of this is that whenver you find some quantity in the world that increases, and where the rate of increase of that quantity is proportional to its size, then it will probably be described by a function very close to e^x. For instance, the population of the planet is increasing, and the rate of increase goes up as the number of people on the people go up, because having more people around means that there are more people adding new babies to the population. That is why the population of the Earth is said to exhibit exponential growth.
It turns out the most useful base for a logarithm is log base e, for similar reasons to why e was the most useful base for an exponential function.
Hope this helps.