The Apple TV can be brilliant, bit can need some help. The WiFi connection between the two devices (Not your internet speed) may need a little help.
Fortunately, you can usually solve the problem by creating a new Wifi network that uses 5GHZ.
You need to do this instead of sharing the standard 2.2GHZ that is also used by Bluetooth and ANT+. This then leaves plenty of bandwidth for your trainer and other sensors and usually stops trainer dropout issues too.
Another thing that can help a lot is to download the rides first. If you do that, the wifi is only streaming large amounts of data in one direction at once, not two.