Should the fence colour match the house exactly?

Not necessarily. A complementary shade usually works better than an exact match. Matching too closely can make the property look flat, while a complementary colour creates contrast and visual interest.