Farrer MP Sussan Ley won't say if she supports Federation Council's contentious plan to remove Aboriginal and Torres Strait Islander flags, but has said "we should unite under one flag".