How about a 1 X 1 matrix with an element that is not -1, 1 or 0.
It will be invertible, and satisfy your requirements.
Additionally, if you want something that doesn't seem too 'cheap', use the observation that any invertible matrix has a non-zero determinant.
So, use a 2 X 2 matrix with the elements a, b, c, and d.
Let ab - cd = 0
Choose values for a, b, and c that are not -1, 1, or 0.
Solve for d, and voila, you get a matrix that satisfies your requirements.