Solving the Challenges You Will Face in Your Pursuit of Real-Time