1. Thread posting restricted: Only the BeamNG team members can post new threads in this sub-forum in order to prevent spam. Posting is available on unlocked threads.

Vehicle loading changes

Discussion in 'Microblogs' started by tdev, May 8, 2020.

  1. ByteGuy

    ByteGuy
    Expand Collapse

    Joined:
    May 2, 2016
    Messages:
    418
    Fantastic! I love seeing how the game engine works and I love seeing the improvements being made. Cleaner code is awesome! Like someone else said im wondering if this will allow dynamic traffic - i.e. instead of having a select number of a set type of vehicle will we see different models being dynamically loaded?
     
    • Agree Agree x 2
  2. Dothero

    Dothero
    Expand Collapse

    Joined:
    Oct 30, 2018
    Messages:
    101
    Very interesting ! Show one more time that adding content is not necessarily the most difficult part. Optimisation takes longer, don't give a lot of changes in the next update (players will be more hype with a new car that if we say we change the way vehicles spawns) but it is essential to keep the game going forward. Great job devs !
     
  3. italy0047

    italy0047
    Expand Collapse

    Joined:
    Feb 8, 2019
    Messages:
    428
    Aaahh oh amazing:)
     
    • Agree Agree x 1
  4. Dummiesman

    Dummiesman
    Expand Collapse

    Joined:
    Sep 17, 2013
    Messages:
    4,682
    Does this have any negative implications for custom vehicle lua? Could it possibly break some things?
     
  5. 77x5ghost

    77x5ghost
    Expand Collapse

    Joined:
    Jun 24, 2017
    Messages:
    89
    my i3 6300 approves of this
     
    • Like Like x 1
    • Agree Agree x 1
  6. jwacker1

    jwacker1
    Expand Collapse

    Joined:
    Aug 10, 2013
    Messages:
    3
    I'm amazed by all the improvements this game keeps getting, even after 7 years since it's forked from Torque3D. Wonder if there's any new commits to Torque3D that would even benefit your fork with how far it's branched. Sounds like some fun coffee filled debugging days ahead! Good luck, and thanks for the updates
     
    • Like Like x 1
  7. Bubbleawsome

    Bubbleawsome
    Expand Collapse

    Joined:
    Aug 5, 2012
    Messages:
    1,886
    Very cool to see! Thanks for the in depth post
     
  8. Robaatoo

    Robaatoo
    Expand Collapse

    Joined:
    Aug 26, 2013
    Messages:
    113
    Very nice work!!
     
    • Like Like x 1
  9. tdev

    tdev
    Expand Collapse
    Developer
    BeamNG Team

    Joined:
    Aug 3, 2012
    Messages:
    3,030
    such_a_great_coffee.jpg
    Good morning for round two :)

    Thanks everyone for your feedback, i'll continue with what is happening in the last days then :)

    Generally speaking, this is optimizing the vehicle loading as a whole and some other parts of the game engine. I want to show you all the details and effort we are undertaking to optimize and improve the GE. Lets see what happened:
    First, some days ago, we optimized the serialize function that send that data to the vehicle Lua. A change like that looks like this: (Top: before optimization, bottom: after. Optimized from ~100 ms to 59 ms)
    opt1.png
    (note that the timing can vary a bit depending on the operating system load, etc at that time)
    Generally, you usually look at the graph, and see what elements could be optimized, work on that little block for a while and make it hopefully faster.
    Ok, then our amazing @LuisAntonRebollo found that we initialize the Materials for the car far too often. Top image: before optimization, bottom after.
    opt2.png
    It was happening at jbeam materials init and twice at the end. We only need two material inits: one for all flexbodies and one for all props.
    This change improved the overall frame time from 1762 ms to 1385.

    Lets see what followed after that. This was a lot of work (the whole weekend):
    opt3.png
    a) we optimized our Filesystem layer (we use a modified physFS btw) to have less calls to the operating system. I.e. fileExist was super slow because it actually tried to open the file - it now uses stat to figure that out a lot faster. We found that findFiles was really slow due to the way the data was handled, and we replaced the syscall FindFileFirstW with the faster version FindFileFirstExW. Temporary memory allocations usually kill the performance really badly. So if you want to write fast code, make sure you don't allocate in the critical path and allocate everything needed upfront - and big enough.
    b) our wizard @estama rewrote the complete serialize and deserialize functions. We initially tried various ways including going to the C++ side of things, but it turns out that the JIT of LuaJIT is a much better fit for this problem. Using it, we are faster in Lua than in C/C++. The serialize (packing of the data into a string) now takes 35 ms and the deserialize (unpacking from string into a lua table) around 48 ms.
    c) Our code master ninja @LuisAntonRebollo found out that the filesystem is really in the way of compiling the material shader permutations. He optimized it to preload the files and have less FS interaction in the critical path.

    These changes reduced the overall frame time from 1386 to 1264 ms :)

    Next up for today:
    d) optimizing the jbeam reading code. We are currently reading and parsing the jbeam of 2711 parts (see attached parts_loaded.json) and we end up using 90 in my testing pickup configuration. (see parts_used.txt)
    That is highly inefficient and calls for optimization. However, it's not as easy to optimize as one might think, as we need to keep the memory usage down to not overload the Lua garbage collector. Rule of thumb is, the more objects/memory lua has to manage, the slower the garbage collection gets to check if something is still in use.
    e) We want to see if we can remove the fmod .bank loading from blocking the main thread.

    Hope you enjoyed the read :)

    Feedback/Comments/Questions ? :)
     

    Attached Files:

    #29 tdev, May 12, 2020
    Last edited: May 12, 2020
    • Like Like x 40
    • Agree Agree x 1
  10. tdev

    tdev
    Expand Collapse
    Developer
    BeamNG Team

    Joined:
    Aug 3, 2012
    Messages:
    3,030
    Hopefully not, however a lot of stuff is changing right now. If we forgot to add backward compatibility for something, let us know when something breaks and we'll try our best :)
     
    • Like Like x 12
  11. kwai38

    kwai38
    Expand Collapse

    Joined:
    Oct 28, 2017
    Messages:
    643
    1. I have no idea what you're saying.
    2. It sounds really good. This is why I love the dev team so much
     
    • Agree Agree x 11
    • Like Like x 1
  12. Wedel 25

    Wedel 25
    Expand Collapse

    Joined:
    Apr 29, 2017
    Messages:
    9
    You guys are the best.
    Not only are you trying your best to make everyone's experience better but you are doing it *WHILE* telling us all about your work. Making screenshots and writing a whole, detailed article; this is what I wish all game devs did to give people a little more clarity and a sneak peek behind the scenes of their favourite game. Even though most of us, me included, cannot understand everything you are trying to explain, it is still interesting how the game works besides what happens on the outside. Massive props to you for informing us about the progress. I am excited for the next one. :)
     
    • Agree Agree x 4
    • Like Like x 1
  13. YellowRusty

    YellowRusty
    Expand Collapse

    Joined:
    Nov 9, 2016
    Messages:
    1,200
    I enjoyed the read, even though I didn't understand most of it on a technical/coding level.

    On another level, it's really good to see all of this work - otherwise it just looks like a single line in a changelog that, for all I know, took two minutes and moving around ten lines of code. Seeing this really increases my appreciation of what the technical side of the dev team does!

    On the side of questions, other than the reason why (All those 'interesting topics' mentioned in the OP, which I understand is probably still under wraps), the main thing I want to know is what impact (if any) this new system for spawing has after the vehicle is spawned - at what point does the physics side take over?
     
    • Agree Agree x 1
  14. tdev

    tdev
    Expand Collapse
    Developer
    BeamNG Team

    Joined:
    Aug 3, 2012
    Messages:
    3,030
    • Like Like x 30
  15. Drivver

    Drivver
    Expand Collapse

    Joined:
    May 1, 2015
    Messages:
    233
    Daaamn, that's so amazing to look at, and realize how much happens each frame, then multiply it to "FPS" to imagine how much work is done each second - and that's graphics only, what about physics that's calculated with rate of 2000Hz during that time. o_O

    I think I'd have a blast toying with dev build, (usually much lower fps) looks like it includes very interesting stuff. - and I'd love to see that implemented in public build:

    (from 1:04 to 1:24 ^)

    Anyway it's great that you share those datails, always nice to learn something new. Also makes feel each update to non-visible "under the hood works" more valuable when you're familiarized with it and how much effort have to be put around it, than just a string of text on devblog. Thank you for sharing :)
     
    • Agree Agree x 1
  16. CaptainZoll

    CaptainZoll
    Expand Collapse

    Joined:
    Nov 10, 2016
    Messages:
    2,979
    I think they've said that the showing of internal parts is a trick with editing software, they take a video of the normal vehicle and a video with the cab/chassis hidden and with one of the debug visualisations turned on, and just sort of mush the two together.
     
    • Agree Agree x 10
    • Like Like x 2
  17. Masterjoc

    Masterjoc
    Expand Collapse

    Joined:
    Aug 4, 2013
    Messages:
    297
    Thank you tdev for the insight.
    As a regular player i feel more connected to a game, when getting insights and detailed explanations.
    You end up understanding problems more than usually.

    Wreckfest did actually the same what you are doing now. They have speedup their loading times by miles and everyone freaked out.
    You can expect the same from BeamNG community when this package lands.

    It is just great to see you are attacking important points step by step.
    Thanks for Diamond we know there are coming 3 updates on how the car handles.
    Thank to your insight we can add now less loading time, which sounds boring but feels so good when playing the game.
     
    • Agree Agree x 6
  18. Scott Guo

    Scott Guo
    Expand Collapse

    Joined:
    Apr 20, 2017
    Messages:
    25
    Thank you very much @tdev, this post has been really informative and insightful. It's really interesting to see how you and your team approach the optimization of this specific task.
    --- Post updated ---
    BTW, I am wondering why not use streaming for IO (not quite sure whether Lua supports it).
     
  19. shortlildyl

    shortlildyl
    Expand Collapse

    Joined:
    Jun 10, 2019
    Messages:
    15
    when i try to load up a map it says somthing delivery carsh anyone know what to do?
     
  20. ast5515

    ast5515
    Expand Collapse

    Joined:
    Aug 4, 2013
    Messages:
    220
    Try safe mode. If it works, it's caused by a mod. Otherwise look at the known issues and fixes thread.
     
    • Agree Agree x 1
  1. This site uses cookies to help personalise content, tailor your experience and to keep you logged in if you register.
    By continuing to use this site, you are consenting to our use of cookies.
    Dismiss Notice