Once heir to a small fortune, Torinn Von Pryce was viciously murdered in his own bed by those closest to him. His spirit now roams the streets of his hometown, particularly outside of his long-since-abandoned estate as the Spectral Duelist - a hostile spirit hell-bent on challenging any who come near.
Fighting the Spectral Duelist should feel unwinnable and desperate. Von Pryce was an excellent fighter in life, and even more so when he no longer has to concern himself with minor injuries.
The bread and butter of this fight involve Teleporting Strike and Riposte. It should feel like Von Pryce is responding to every attack, like he's all over the place, and like they need to RUN.
Utilize Challenge as often as possible to enhance this as well. Not only will it make the Challenged target have a fantastic time, but it will also allow for several Ripostes during the first bout. After that, maybe players outside of the Challenge will wise up, or maybe they'll keep swinging.
Bear in mind as well - each Riposte fueled attack will allow the Duelist to use Teleporting Strike again. It all synergizes with itself.
The Duelist side of this character is very two dimensional. He's struggling, in pain, and he's like a wounded animal. Everyone he sees has backed him into a corner, and now he must strike.
Should the party be able to successfully outrun the Duelist, or say something to break through to Von Pryce, well, that's another story entirely.
Von Pryce is angry and bitter, sure, but he also recognizes that there are specific people who caused his death. Von Pryce is very posh, and can be a little dismissive, but is overall a warm personally.
As mentioned above, Von Pryce was a wealthy individual based in a medium sized town. He had a large estate, but was slain in his sleep. The Duelist came into play nearly right after - his tormented spirit lashing out at those nearby. The townsfolk know to just avoid the area - even if someone has been able to defeat the Spectral Duelist, it simply reappears a few hours later. They aren't a fan of the Duelist, and some view him as a menace, but most simply view the whole situation as tragic. "I just wish someone could help put his spirit to rest" is the general consensus.
Perhaps Von Pryce's remains are still inside of the estate, and your players must go and deal with them to put him to rest. Perhaps a ritual needs to be performed to do so. What will it be in your game?
The Spectral Duelist is a heavy combat encounter - most parties won't recognize that there's more to do here, and that's intended. With his combat, we want a quick, and desperate sounding track. Way of the Witcher from Gwent hits that bill pretty darn well.
On the flip side - if they do manage to break through the Spectre, and speak to Von Pryce, a creepy and unsettling track should work. Regardless of Von Pryce's general demeanor, they are still speaking to a spectre who will try to kill them the moment he manifests again. The alternative Gaunter O'Dimm Theme from Witcher 3 works well here.