When I try to check out a book that I have already checked out, koha will ask me if I want to renew the book instead. But this does not work if I have already taken the maximum allowed number of books. In this scenario, I get the "Too many checked out." notification, but I think it is logical to allow renewal as it does not increase the number of books that the patron has taken.
TLDR; If you have reached the maximum number of checkouts, you cannot renew books.
I cannot reproduce this issue in master.