User Interface Painfully Slow

Every since the latest update my game has been running smoother but the user interface (menus, gauges, messages, apps) has been running painfully slow. It often will take a full 20-30 seconds for the menu to pop up after hitting the escape key in-game.

 

I've always experienced this, modded or not. At first it's fine, but over time it gradually gets more and more laggy and just gets to a point where I need to restart the game to fix it.

While the menu generally opens fine, albeit a little slow, the actual use of the menu is horribly slow. More cars makes the issue worse and I think it's because the UI takes up around 10-20% CPU usage.

The problem gets worse over the course of 15 minutes to ~1 hour depending on the amount of cars currently loaded and the issue existed in past versions too.