WIP Beta released Arguably the BEST way to find Lua functions! - DumpNG

Discussion in 'Programming' started by Hank_Montgomery, Sep 22, 2025.

  1. Hank_Montgomery

    Hank_Montgomery
    Expand Collapse

    Joined:
    Jun 9, 2022
    Messages:
    70
    I’m happy to introduce DumpNG!
    This PowerShell script collects all text-based unarchived files (Lua, JSON, JS, etc.) from your BeamNG installation and combines them into a single text file, with headers showing where each snippet came from. That’s over 10,000 files merged into one with over 2.8 million lines of code!

    Who is this script for?
    • Modders who want to explore game functions but don’t know where to start.
    • Experienced content creators who find WinRAR or Agent Ransack too slow for keyword searching.
    • Anyone who prefers using VS Code to jump instantly between matches.
    How to use the script:
    1. Open Notepad (or any text editor) and paste the code at the bottom of this post in.
    2. Update the $OutputFile path, then save the file with a .ps1 extension.
    3. Right-click the saved file and choose Run with PowerShell.
    4. As of the 0.37 game update, wait until the output file grows to around 125 MB, then close the PowerShell window. This may take about 20 minutes. You can track progress by checking the file size in its Properties.
    5. Open the output file in a text editor (VS Code recommended). Enable syntax highlighting for large files (see FAQ).
    6. Use Ctrl+F to search for keywords.
    FAQ.
    Q:
    Text in the output file is white when I open the file with Visual Studio Code. How to fix it?
    A: Syntax highlighting is disabled by default for large size files. Here's how to enable it:

    1. Go to %APPDATA%\Code\User\settings.json
    2. Paste these overrides before the last curved bracket at the file bottom:
    Code:
    "editor.largeFileOptimizations": false,
    "editor.experimental.asyncTokenization": true
    
    3. Save the file, then reload VS Code (Ctrl+Shift+P → Developer: Reload Window)
    Q: My PowerShell window is blank when I run your script.
    A: It's normal.

    Q: What file extensions does your PowerShell script extract code from?
    A: lua, json, js, css, ini, bat, sh, atlas, svg, txt, md, yml, gitignore, markdown, luadoc, postfx, hlsl, h, vue...

    Q: Why don't you just upload the script file to your post?
    A: BeamNG forum doesn't allow to upload .ps1 files.

    Q: Your code has part where it extracts files from archives, but I don't find anything related in the output file.
    A: The part related to extracting of files from .zip, creating "Temp" folder to process the contents is broken. If you know how to fix it, we could discuss it in this thread as well as other improvements.

    Code:
    # DumpNG-v1
    ###############################################################################
    #  Path to the top-level folder containing your files & subfolders:
    $SourceFolder = "C:\Program Files (x86)\Steam\steamapps\common\BeamNG.drive"
    
    #  Single text file to store all extracted text:
    $OutputFile   = "C:\Users\USERNAME\Downloads\BeamNG_Files.lua"
    ###############################################################################
    
    # 1) Create or overwrite the output file:
    New-Item -ItemType File -Path $OutputFile -Force | Out-Null
    
    # --- Function: Quick heuristic to check if a file is “likely” text ---
    function Is-ProbablyTextFile {
        param(
            [string] $Path,
            [int] $sampleSize = 4096
        )
    
        # 1a) Skip these extensions (known binary / non‐text).
        $binaryExtensions = ".pdf",".png",".jpg",".jpeg",".gif",".exe",".dll",".so",".bin",
                            ".doc",".docx",".xlsx",".xls",".ppt",".pptx",".mp3",".wav",
                            ".ogg",".flac",".zip",".rar",".7z",".iso"
    
        $ext = [System.IO.Path]::GetExtension($Path).ToLower()
        if ($binaryExtensions -contains $ext) {
            return $false
        }
    
        try {
            $fileInfo = Get-Item $Path -ErrorAction Stop
            if ($fileInfo.Length -eq 0) {
                # Empty file => treat as text
                return $true
            }
    
            # Read the first 4 KB and check if more than 5% is outside ASCII range.
            $fs = [System.IO.File]::OpenRead($Path)
            [byte[]] $buffer = New-Object byte[]($sampleSize)
            $readCount = $fs.Read($buffer, 0, $sampleSize)
            $fs.Close()
    
            $badBytes = 0
            for ($i = 0; $i -lt $readCount; $i++) {
                $b = $buffer[$i]
                # If the byte is <9 (excluding tab=9) or >127 => “suspicious”
                if ((($b -lt 9) -and ($b -ne 9)) -or ($b -gt 127)) {
                    $badBytes++
                }
            }
            $ratio = $badBytes / $readCount
            return ($ratio -le 0.05)  # treat as text if <=5% suspicious
        }
        catch {
            # If we can't read it, skip it
            return $false
        }
    }
    
    # --- Function: Append the file's lines into the big output file ---
    function Append-FileContent {
        param(
            [string] $FilePath,
            [string] $OutFile
        )
        try {
            # Write a header line to identify which file the text came from:
            Add-Content -Path $OutFile -Value ("=== File: $FilePath ===")
    
            # Read the file line-by-line as an array of strings:
            $lines = Get-Content -Path $FilePath -ErrorAction Stop
    
            # Write each line to the output:
            # (You can also do: [System.IO.File]::AppendAllLines($OutFile, $lines) if .NET version supports it.)
            $lines | Add-Content -Path $OutFile
    
            # Blank line after each file:
            Add-Content -Path $OutFile -Value ""
        }
        catch {
            Add-Content -Path $OutFile -Value "Error reading file $($FilePath): $($_)`r`n"
        }
    }
    
    # -----------------------------------------------------------------------------
    # 2) PROCESS ALL NON-ZIP FILES (RECURSIVELY), USING “Is-ProbablyTextFile”
    # -----------------------------------------------------------------------------
    Get-ChildItem -Path $SourceFolder -Recurse -File |
        Where-Object { $_.Extension -ne ".zip" } |
        ForEach-Object {
            if (Is-ProbablyTextFile $_.FullName) {
                Append-FileContent -FilePath $_.FullName -OutFile $OutputFile
            }
        }
    
    # -----------------------------------------------------------------------------
    # 3) FIND ZIP FILES, EXTRACT EACH TO A TEMP FOLDER, PROCESS THEIR CONTENTS
    # -----------------------------------------------------------------------------
    $ZipFiles = Get-ChildItem -Path $SourceFolder -Recurse -File -Include "*.zip"
    
    foreach ($zipFile in $ZipFiles) {
        $zipFilePath = $zipFile.FullName
        $tempFolder = Join-Path ([IO.Path]::GetTempPath()) ([GUID]::NewGuid().ToString())
    
        New-Item -ItemType Directory -Path $tempFolder | Out-Null
    
        try {
            # Extract the ZIP into a temp folder:
            [System.IO.Compression.ZipFile]::ExtractToDirectory($zipFilePath, $tempFolder)
    
            # Gather all extracted files (excluding nested zips if you like) & apply text check:
            Get-ChildItem -Path $tempFolder -Recurse -File |
                Where-Object { $_.Extension -ne ".zip" } |
                ForEach-Object {
                    if (Is-ProbablyTextFile $_.FullName) {
                        Append-FileContent -FilePath $_.FullName -OutFile $OutputFile
                    }
                }
        }
        catch {
            Add-Content -Path $OutputFile -Value "Error extracting or reading zip: $($zipFilePath). $($_)"
        }
        finally {
            # Remove the temp extraction folder
            Remove-Item -Path $tempFolder -Recurse -Force
        }
    }
    
    Write-Host "Done! See $OutputFile for the combined text."
    
     
    #1 Hank_Montgomery, Sep 22, 2025
    Last edited: Sep 22, 2025
  2. Brokenbraincells

    Brokenbraincells
    Expand Collapse

    Joined:
    Dec 27, 2023
    Messages:
    76
    How long should i wait to see the file appear so i can check the files size? Been running for at least 5 mins now and no file. I changed the output to the correct path because i also had to change the source path.
     
  3. Hank_Montgomery

    Hank_Montgomery
    Expand Collapse

    Joined:
    Jun 9, 2022
    Messages:
    70
    The output file should appear in your Explorer once you click on "Run with PowerShell".
     
  4. Riccarr

    Riccarr
    Expand Collapse

    Joined:
    Jul 28, 2024
    Messages:
    51
    Can you define what "BEST way to find Lua functions" means? I mean from what perspective ... are you suggesting a list that you just scan through looking for interesting function names, or to be able to keyword searches?

    I find the best approach (and I see you said "Arguably " :)) is to open the /lua folder from the beamng installation, using VSCode. This will include ALL the lua files at your search disposal. The global search in VSCode (CTRL-SHIF-F) to search for keywords works fantastic across all the lua files. But ... your using this from a perspective of "I'm looking for a keyword match". You then find functions as well as all related references this way. Especially if you know a function name and want to see how its used across all the lua files.

    If your browsing the lua files (with above opened folder) and want to learn more on the usage of a function you find, then double click to highlight name and CTRL-SHIFT-F to search all references of it across all lua files; you see how its used in the code as well as learn more about its parameters based on context in the code. I would arguably say this is the best approach. :) YMMV
     
  5. AlexKidd71

    AlexKidd71
    Expand Collapse

    Joined:
    Mar 16, 2022
    Messages:
    613
    I’m not very experienced in developing lua. What helps me a lot is using AI coding support with GitHub copilot in VS Code. I just added the BeamNG lua directory with a symbolic link to my project so that the ai agent can search it when I prompt a question. That helps a lot because AI is also not very experienced in BeamNG LUA. :)
     
    • Like Like x 1
  1. This site uses cookies to help personalise content, tailor your experience and to keep you logged in if you register.
    By continuing to use this site, you are consenting to our use of cookies.
    Dismiss Notice