The Silent Duke by Jess Michaels

Born mute, Ewan Hoffstead, Duke of Donburrow was abused by his wicked father and wretched younger brothers until an uncle swept in to save the day. Still, he carries the scars of his past with him, including a terrible belief that perhaps a “person like him” truly doesn’t deserve the title his uncle fought so hard for him to earn.
Recently widowed Charlotte may be the proper Countess of Portsmith to Society, but when she is around Ewan, she still feels like the enraptured little girl who met him decades before when her brother was best friends with him and his cousin.
A planned trip to spend Christmas with their families at Ewan’s estate reunites the old friends. But when a storm keeps the others from arriving on time, Charlotte decides to stop living in fear and pursue the man she has desired and loved her entire life. Ewan cannot resist her seduction, but he fears giving in to his heart. Especially when the family that disposed of him returns and threatens not only his life, but that of the woman he loves.