The answer is (E). (side note: this is one of the requirements for a function to be a subspace). Let me show you.
A) F(x) = x^2, so
F(A + B) = (A + B)^2 = A^2 + 2AB + B^2
F(A) + F(B) = A^2 + B^2
And as you can see, they're not equal, i.e.
F(A + B) =/= F(A) + F(B)
B) F(x) = x + 1
F(A + B) = A + B + 1
F(A) + F(B) = (A + 1) + (B + 1) = A + B + 2
So they're not equal, i.e.
F(A + B) =/= F(A) + F(B)
C) F(x) = sqrt(x)
F(A + B) = sqrt(A + B)
F(A) + F(B) = sqrt(A) + sqrt(B)
In no way do we have methods of combining the sum of radicals, so they are not equal.
D) F(x) = 2/x
F(A + B) = 2/(A + B)
F(A) + F(B) = (2/A) + (2/B) = (2B + 2A)/(AB)
Not equal.
E) F(x) = -3x
F(A + B) = -3(A + B)
F(A) + F(B) = -3A + (-3B) = (-3)(A + B)
And as you can see, they are clearly equal.