Some managers will try and play down sexual harassment that occurs at social functions. The line between work and a “fun event” can be a fine one. If a function is held at the employer's expense, and is clearly a works party, it is likely that this constitutes sexual harassment at work. In any case, behaviour of this sort is unacceptable, particularly if you make it clear that you do not like it.
Another excuse often given is: "everyone else was doing it". This is no defence, and the fact that other people also behaved unacceptably does not mean that it is okay in each case; some people may not object, but that does not make it unreasonable for others to do so.
Threatening you in this way after behaving in a manner that you found objectionable would be a breach of the sex discrimination provisions of the Equality Act 2010. If you are concerned that your boss may act on the threat and/or you now feel uncomfortable working with him because of his behaviour, speak to your union rep, if you have one. If you are not a union member, you can get advice from the Equalities and Human Rights Commission, ACAS or Citizens Advice.