Might I ask why? Without any context the answers to your questions are as follows: Yes you can. As often as the tree can take.
Why do you defoliate a tree?
Why does this tree need a defoliation?
Other questions like these will help make the choice whether to defoliate or not:
Is this tree strong enough to handle a defoliation?
Does defoliation accomplish what I’m trying to accomplish?
For the TL:DR version: Don’t defoliate it. You need to thicken the trunk and that won’t happen if you take away all the solar panels of the tree. #MyOpinion
I agree, it will only slow down the development and with a young tree like this has no purpose. The size of the leaves do not matter when the tree is being developed like this one.
Just had a look to see what the species is as I had not heard of it. It is not an Olive of course, that is the problem with common names. It is a tropical species, but I still think that there is no reason to defoliate, let it grow, then you will be able to start styling the tree.