Load times should be high on your roadmap and you should revisit this regularly. There is a lot of research around this. A classic post from Kissmetrics reveals that an 1 second delay can result in a 7% reduction in conversions.

A couple of things you should think about:

• Make performance a metric. Google Analytics provides great tools and documentation on tracking performance. Use tools like Pingdom to track your performance from around the world. Make sure to review these metrics after new features are added and fixes deployed.
• Fixing performance issues is really tricky. Always start measuring. Start with the largest chunks and work your way down. Be on your watch when a developer says “I’m pretty sure this is what makes the page slow”.
• There is a difference between actual load time and perceived load time. This is a great post from Treehouse on perceived performance.