The two storey solid wooden building housing the museum today was built in the year 1897 as a fort by the second White Rajah, Charles Brooke to impose peace and order in the region where warfare among the local tribes prevailed.
Ironically located at a depression of the hills and surrounded by higher terrain, Hilltop Garden is a small residential community that was built in the mid 1980s just off Riam Road.