I would agree with everyone else as to the fact that they are not HIS deer.
But, I was in that same dilemma for three hunting seasons at my place. While I was hunting Splitter, and only Splitter at my place I asked, not demanded, my stepson and hunting buddy to let him walk if they saw him. But, in reality if one of them had killed him, so be it. I played it the way I did for a few reasons. I owned the property, did 90% of all work there, my wife did most of the remaining 10%. I had built and placed all the stands, I paid all the expenses for electricity and did all the upkeep. I ran all the trail cameras, without them no one would have known the deer even existed. All my stepson and my buddy did was show up to hunt for the most part. The only person that helped out much was my SIL that doesn't hunt at all.
Some would say I was selfish, which I don't agree with at all. Any other buck they saw was fair game for the taking. And there were plenty of other decent deer to choose from. If the three of us had been hunting a property that we all just had permission to hunt I'd have never thought the way I did. No one owns a free range deer.