Is there any possibility to avoid the accessibility of the whole phonebook by this app???

I don't want Whatsapp to get all of my numbers and it's just a little boring to always carry two cellphones with me...