1. Thread posting restricted: Only the BeamNG team members can post new threads in this sub-forum in order to prevent spam. Posting is available on unlocked threads.

  2. Game not starting lately?
    Fixed in 0.19.4.2
    More information available here

    Dismiss Notice
  3. Intel iGPU (6xx series) crashes
    Fixed drivers available!
    Instructions here

    Dismiss Notice

Vehicle loading changes

Discussion in 'Microblogs' started by tdev, May 8, 2020.

  1. tdev

    tdev
    Expand Collapse
    Developer
    BeamNG Team

    Joined:
    Aug 3, 2012
    Messages:
    2,884
    Hi everyone :)

    We are undertaking quite some bigger changes within the game engine right now to be able to move forward on a lot of interesting topics. I wanted to share this with you a bit and I hope it's not too boring to read ;)

    Lemme start trying to explain a bit how the game engine is working right now :)
    Warning: will get technical now ^^

    Maybe grab a coffee and lets get started:

    coffee.jpg




    So our GameEngine (GE) [we haven't found a better name so far ;)] is running one Lua (specifically we use LuaJIT) Virtual machine inside that is dealing with things like:
    • Settings
    • Gameplay
    • generally, features on top of the game engine core (i.e. camera paths)
    • Editors: Flowgraph, World editor
    • Scene Management
    • Scene (level) loading
    Our GameEngine is written in C++ and is originally forked from Torque3D some 7 years(?) ago. Since then we revamped a lot of things and there are not many things left that we did not touch (performance work, removing horrible network code that made life horrible, render improvements, Lua, so much more ...). Today, it is consisting of around 384 thousand lines of code within roughly 1000 c++ source files. In visual studio 2019 it currently looks like this:
    devenv_PmsviL7fcy.png
    As you can see, it is doing the heavy lifting when it comes to things where we do need the speed of C/C++. It is busy doing:
    • Rendering
    • Storing the scenetree and what's going on on-screen
    • Classes you can drop into your scene, like timeOfDay, Skies, water, TerrainBlocks, etc
    • Filesystem, Networking (Mods, online API, etc)
    • Physics integration
    • User interfaces (currently CEF, ImGui and the old Torque3D GUI)
    • A lot more i forgot to add to this list here ;)
    So, i want to explain the Lua changes that are upcoming:

    ge_arch.png
    Our GE has one Lua running (folder is lua/ge) and the physics core one per vehicle (folder is lua/vehicle).

    So the vehicle spawning in the latest released version is working by having the GE side telling the physics to spawn a new car. The physics will then create new threads and a new Lua VM where the Jbeam loading code is executed in to load the files, assemble the vehicle and 'bake' the final car :)

    This has several drawbacks that we saw growing bigger and bigger over the years. For example, the communication with the GE lua is very slow and is based on sending strings to each other. So getting data from the vehicle to be used in the GE is a slow ping-pong process of sending messages around. Loading 3D meshes to be used for the physics was a complicated issue as the GE needs to expose all the interfaces to do that. Due to many reasons like this, we will be moving the loading code for vehicles from the physics core into the GE itself. This way, we can optimize, parallelize and separate the tasks for each component better. The physics Lua VM will only care about the runtime things and the GE will completely send the physics a pre-baked vehicle it is supposed to load and work with. It can do the 3D meshes on its own - which is a huge relief as it reduces the overall complexity a lot.

    First step some weeks ago was to clean up the jbeam (our JSON flavor) loading. It improved from being one file with 2000 lines of code into a nice structure where it's clear on what's going on:
    Code_PjSKMVUNLd.png
    After that was done, we slowly moved code to run from the vehicle side over to the GE side. That was a very slow and buggy process and we had to fix many problems (i.e. converting wheels math from float3 to our vec3 math library or the serialization breaking with infinite number values).

    So, i hope i briefly rushed over the basics of this, lets talk a bit about performance. We use the tool Optick for profiling our game. It looks like this:

    optick.png
    In the top half are the render frames displayed. Every bar is one frame of the game. 60 FPS means 60 frames per second. The vertical height of that bar means how long the frame took to complete. Ideally you want to distribute the workload of loading a level or a car over multiple frames equally rather than having spikes, which result in stutters or the appearance that the game is hanging. In above screenshot you can see gridmap is loading for 2 second in one frame, which is far from optimal.
    The area below is the detail view of what happened inside the frames: from top to bottom are the things that are calling each other to do stuff. For example 'spawn' is calling 'loadVehicle' etc.

    Lets have a closer look on how a cached (.cdae loaded) vehicle looks like in loading:
    Optick_D6xyRte6U4.png
    On the left side it starts with gathering all the jbeam files off the filesystem and parsing them, which takes around 223 ms in this example). Then it proceeds to go throught he jbeam sections, processing the 3D meshes, etc. It then sends the data over to the vehicle side where it continues with "spawnVehicleOhysics". It then spawns the physics object and finishes the 3D meshes in "spawnObject: post".

    Writing this took a lot longer than i expected, i'll need to continue another day. Also would be cool to see how many people are interested in this, so i know if i should continue in that detail :)

    Hope you enjoyed the little sneak peek :)

    >> Continued with post #2 here. <<

    Questions/Comments/etc? :)
     
    #1 tdev, May 8, 2020
    Last edited: May 12, 2020
    • Like Like x 90
    • Agree Agree x 5
  2. JBatic

    JBatic
    Expand Collapse

    Joined:
    Jan 22, 2015
    Messages:
    1,098
    Very interesting. Faster loading and cleaner code is always good.
     
    • Agree Agree x 18
  3. LJ74

    LJ74
    Expand Collapse

    Joined:
    Oct 27, 2018
    Messages:
    563
    Don't know much about the workings of stuff like this (Programming etc)
    So i'm just asking if this will decrease the CTRL+R loading time because that will be quite helpful when modding :p
     
    • Agree Agree x 8
    • Like Like x 1
  4. Confused_Deer43

    Confused_Deer43
    Expand Collapse

    Joined:
    Jun 8, 2013
    Messages:
    432
    Interesting! So this will lead to faster load times when changing parts and smoother load screens?
     
    • Like Like x 1
  5. EruptionTyphlosion

    EruptionTyphlosion
    Expand Collapse

    Joined:
    Sep 24, 2016
    Messages:
    3,619
    I'm assuming yes as if I remember correctly CTRL+R loads the vehicle similar to the spawning process.
     
    • Like Like x 1
  6. TubroTerra

    TubroTerra
    Expand Collapse

    Joined:
    Jul 24, 2019
    Messages:
    308
    Great faster load time!
     
    • Like Like x 1
  7. TheSedd53

    TheSedd53
    Expand Collapse

    Joined:
    Oct 24, 2014
    Messages:
    6
    You really should continue this post in this level of detail. I don't fully understand everything but it is still very interesting nonetheless :)
     
    • Agree Agree x 6
  8. mtslittow

    mtslittow
    Expand Collapse

    Joined:
    Sep 28, 2012
    Messages:
    125
    I'm interested to see a followup on the impact of these optimizations.
     
  9. ARY5588

    ARY5588
    Expand Collapse

    Joined:
    Feb 2, 2014
    Messages:
    91
    So this will help with low cores(=<4) cpu's performance when using multiple vehicles?
     
    • Like Like x 1
  10. daveywaveyHD

    daveywaveyHD
    Expand Collapse

    Joined:
    Aug 16, 2012
    Messages:
    325
    Would love to see more indepth details like what you've posted, super interesting to see what happens behind the scenes :)
     
    • Agree Agree x 4
  11. Anonymous275

    Anonymous275
    Expand Collapse

    Joined:
    Feb 27, 2016
    Messages:
    11
    i really like the detailed explanations looking forward to these massive optimizations.
     
    • Agree Agree x 5
  12. Sithhy™

    Sithhy™
    Expand Collapse

    Joined:
    Apr 5, 2017
    Messages:
    2,442
    I do enjoy reading technical changes in the game (one of the most favourite things about the updates/hotfixes are changelogs), so seeing the main dev show us the indepths of the GE is definitely a feast for me :p

    But, now a question... Since the vehicle loading is being optimized, will that mean the wait time for the vehicle to load will be reduced substantially? Could that also possibly mean that we will be able to have different cars spawn dynamically in traffic, unlike the pre-set configs like right now?
     
    • Agree Agree x 5
    • Like Like x 1
  13. NOCARGO

    NOCARGO
    Expand Collapse

    Joined:
    Apr 1, 2019
    Messages:
    728
    It would be very nice indeed to see more detailed follow ups on this subject. :)
    Many (including me) might not always understand the smallest details, which is normal, but the main narrative is giving us a lot of valuable information on the functionality and the future of the game ( it's not really a game but fine, at least the missions and the traffic are :p ).
    Thanks for sharing, T-dev !
     
    • Agree Agree x 4
  14. Shtoorminator

    Shtoorminator
    Expand Collapse

    Joined:
    Jun 10, 2017
    Messages:
    42
    It is always interesting to see what's going on "inside". It's like taking apart some complicated mechanism to figure out how exactly it works. Apart from the fact that now there's you who actually explains the details. Deffinetely would like to read more.
    You haven't provided us the info about how do you plan on improving loading yet, but I have a question in advance: will this new way depend on certain hardware/software? What I mean by this is that some time ago BeamNG dropped the support of DX10, so as far as I understand some users who had very old hardware couldn't really play new versions of a game anymore. It didn't really affected me that time, but still I have a very old pc, so I'm kinda afraid that at some point I won't be able to enjoy new updates.
     
  15. Aboroath

    Aboroath
    Expand Collapse

    Joined:
    Aug 25, 2013
    Messages:
    3,806
    Love this stuff. Please continue and be as technical as you want! Thanks for posting this:cool:.
     
    • Agree Agree x 5
  16. SergentFido

    SergentFido
    Expand Collapse

    Joined:
    May 30, 2015
    Messages:
    1,511
    That's amazing Thomas :)
     
    • Agree Agree x 2
  17. crazikyle

    crazikyle
    Expand Collapse

    Joined:
    Sep 3, 2013
    Messages:
    1,018
    I like this writeup, and I can't wait to see the changes in game. Load times between levels and switching vehicles is currently one of my biggest pet peeves, so this is nice to see. Could this be used in the future to allow more dynamic traffic (as in it swaps out vehicle models occasionally, so you're not always seeing the same 5 cars), or do you think the performance gains won't be enough to allow that type of behavior?
     
  18. Nissan Skyline

    Nissan Skyline
    Expand Collapse

    Joined:
    Aug 11, 2013
    Messages:
    258
    This is super interesting! If I am understanding correctly, does this allow for both better scaling of single vehicles that are very complex, along with having many vehicles open at once? Currently, the engine does bog down even if you have less vehicles than the number of cores you have.

    Edit: Also the picture showing the frame times for 60 frames is so cool, it's my new wallpaper. I'm super interested in how games render, and being able to see a representation of what happens in between each frame and the specific time that took to happen is just incredible. Its essentially performance monitoring the frametimes that would typically be done with a program like rivatuner, but being able to see the engine display what is happening is really cool.
     
  19. ARES IV

    ARES IV
    Expand Collapse

    Joined:
    May 6, 2019
    Messages:
    247
    Very interesting.
     
  20. Brother_Dave

    Brother_Dave
    Expand Collapse

    Joined:
    Aug 16, 2012
    Messages:
    1,048
    Note so self:
    When one of the devs posts a very, very intresting in-depth post, start reading it AFTER you get out of the bathroom or your legs will go numb and youll be stuck there for a while.

    This is a typical example of what makes you devs different, you post things like this. I dont know any other game of this magnitude that has devs thatd do that.
     
    • Agree Agree x 2
  1. This site uses cookies to help personalise content, tailor your experience and to keep you logged in if you register.
    By continuing to use this site, you are consenting to our use of cookies.
    Dismiss Notice