I have Roon running as a headless server connected directly to my DAC. I do not use HQ Player however. I have heard that doing on the fly conversions does take some serious power. My server has the power I just don't have the desire for HQ Player. I also use a Surface Pro tablet as my controller.
One thought, you mentioned that your issue is with DSD256 files. I play DSD256 files perfectly thru my system, therefore I am wondering what drivers you are using. Many times DAC manufactures include a couple different drivers. DSD256 does not work with DoP drivers, which is the standard install for most DACs. DSD256 requires "Native" drivers, sometimes referred to as ASIO drivers. Please make sure this is what you are using.
By the way, I also have a Tiki. Originally bought as a gaming PC, but I re-purpose it as a music server, taking out video card and spinning drives, etc. Falcon Northwest probably make the best custom PCs on the planet. My wife and I have bought 8 over the years. Be aware, they are very expensive computers, but you get the best in both machine and service! Probably overkill as a dedicated music server

. Also, realize that the high end graphic cards are exclusively for video and gaming and have no baring on music, or desktop apps for that matter. I had a Nvidia GTX980 in my Tiki (the best available at the time, love Nvidia). When making it a music server I took the graphics card out. What is built in to the Intel CPU is fine for the purpose. As a headless server I do not even have a monitor, keyboard, mouse, etc. hooked to it. I use Remote Desktop to perform any maintenance or file copying tasks.