Tyson Fury’s father had threatened to not talk to him if he fought Deontay Wilder