If my ex boyfriend bought me a phone, and is now asking for it back, is he allowed to take it?

Your ex can ask for anything he wants, but nothing you've written in your question would give him the right to take the phone back, whether he gave it to you as a gift or in settlement for having damaged your old phone.  Of course, he may have his own story.  For example, he may claim that he only gave you the phone to use while the two of you were together, or that you failed to do something that you were supposed to do in return for the phone, for example pay for it in installments.  If he does claim something at odds with what you've written, there is no guarantee that a judge or jury will not believe his story over yours.