In the My Connections or Everyone tab, you can remove the connections between you and the users who are your contacts via one of two following ways.
The first way
Clickunder the contact name; or hover your cursor over the contact name and click to remove your established connection.
The second way
Access the profile page of the contact who is your connection, then click the link next to the contact name at the top left corner of the profile page.