Yeah that's the issue. If his role is returner and gadget guy and let's be real... You can't trust him in a bigger role, it means you have to keep a backup returner and gadget guy on your roster. Because he doesn't go on IR, he takes up roster space when he gets hurt. And he doesn't always finish games.
Not to mention the bad effect of a "locker room cancer" on the team