power crystals' XML patcher and data exporter (now under old management)

power crystals
Corporal
Posts: 42
Joined: Tue 8 Apr 2014 00:18
Contact:

power crystals' XML patcher and data exporter (now under old management)

Postby power crystals » Mon 20 Jun 2016 01:12

Hello my name is power crystals and I am addicted to doing things with games that do not include actually playing them. Since for the time being I'm actually paying attention here after a long hiatus due to a variety of personal things I asked the moderators to split this thread so I could own the OP.

Probably my most "famous" creation is the random deck generator at http://www.wargamerandomdeck.com/ which I fully suggest all people use for all games at all times because nothing ever goes wrong when you use a random deck.

But you're likely here for one of my other two creations, located at http://www.wargamerandomdeck.com/tools/
Table Exporter (current version 1.0.0
Usage: WGTableExporter.exe <file.dat> [<file.ndfbin|file.dic>] [<tablename>]

Will export the specified table from an ndf file, or the specified dic/ndf file from a .dat file, or everything in the .dat file, depending on the number of arguments. It can extract TGVs (but not individually right now), but it cannot recurse into PPK files, so you'll need to extract those yourself first.


NDF Patcher (current version 2.2.0
Usage: WGPatcher.exe [action] [...]
(most commonly, WGPatcher.exe apply NDFWin.dat patch.xml)

Fully documenting this will take me some time, so bear with me, but for now there's two actions you'll be concerned about:
  • apply or patch (equivalent): Will apply a patch XML file to a .dat file, producing a new .dat file named "[...]_patched.dat" with the changes.
  • index: Will produce a CSV detailing where the newest version of any given file is. This can recurse into PPKs, so you can use this to locate where TGV files are to replace.

Two small example patches are included to show you how it works.


Both of these tools make a reasonable attempt to explain their arguments if you don't provide enough, so if you type for example "WGPatcher.exe apply" it will tell you what that particular action requires. If one's missing documentation let me know.

Both come with a copy of moddingSuite (which may be out of date) which they require to operate. Both are currently built against .NET 4.5, though I may move them forward to newer versions of the framework at a future date if I need to.

Though I am somewhat busy I'll still watch this thread for comments in case anyone has questions or requests for new features. Bollywood Baloney is already responsible for like 30% of the XML patcher's features so don't be afraid to ask!





Spoiler : Original post that was here before the thread was split :
So only what, five months late? and two years from initial release? but I think I have a list-of-lists deal working now that I had the correct combination of time and energy to do this. Assuming anyone still uses this thing, anyway.

http://www.wargamerandomdeck.com/tools/ has a link to a new, 2.0 version. Changes include:

  • Added operator "select". Use this to navigate inside lists to perform operations on nested lists (see below). Context resets to outermost when the next patch element is encountered.
  • Added new special matchcondition "__createdby" which allows you to directly reference objects created by an earlier ndfcreate
  • Added operator "null" (sets value to null). Note that you could actually already do this with type="Unset" but now there's a shortcut
  • The "type" attribute is no longer necessary in many cases even if the existing value is Unset; the patcher will attempt to figure out the type based on other instances of this class if possible. Specifying a type will always override the inferred type. Note that inference is impossible when inside a nested loop context via use of the select operator.
  • Operation names are now case-insensitive
  • Added some extra input error checking, it should complain a lot more about files that are partially malformed
  • Fixed "multiple matches found, first will be used" to ACTUALLY use the first instead of returning null
  • Fixed remove not working on lists of maps (it was actually setting the value which did god knows what)

So the exciting part is this:

Code: Select all

<wargamepatch>
   <!-- This will insert a record into an instance the below table (chosen by being the first one I found with nested list) -->
   <ndfpatch ndf="pc\ndf\patchable\gfx\everything.ndfbin" table="TIASkirmishDescriptor" name="Nested Lists">
      <matchconditions>
         <matchcondition property="__order">first</matchcondition>
      </matchconditions>
      <changes>
         <!-- Add a map. Here, type is required for it to insert a Map and not overwrite the property with a new MapList
         I'm not sure how to fix that without breaking the file format -->
         <change operation="add" property="MissionAndUnitList_OnlyForTest" type="map">
            <map>
               <key type="Int32">16</key>
               <value type="List" />
            </map>
         </change>
         <!-- Navigate inwards to the selected list. Note the specification of the key -->
         <change operation="select" property="MissionAndUnitList_OnlyForTest" key="16" />
         <!-- Add a record to the inner list we created earlier -->
         <change operation="add" type="ObjectReference">
            <reference table="TUnitSpawnInfos">
               <matchconditions>
                  <matchcondition property="__order">first</matchcondition>
               </matchconditions>
            </reference>
         </change>
         <!-- And update it by specifying its key (0-indexed) -->
         <change operation="set" key="0" type="ObjectReference">
            <reference table="TUnitSpawnInfos">
               <matchconditions>
                  <matchcondition property="__order">last</matchcondition>
               </matchconditions>
            </reference>
         </change>
      </changes>
   </ndfpatch>
</wargamepatch>

(this is included with the download)

It SHOULD be able to work on lists of lists of lists (etc...) if you just repeat the select operation to navigate into the next layer down but I admit I didn't actually find one of those to try it with.

I had to redo most of the internals to get this to work, so there's no guarantee I didn't break something as I definitely haven't had the time to write automated tests for this thing. If I did break someone's mod, please let me know. In the meantime I left the link to the old one just in case.

I'll try to hang around for any specific issues from this until work murders my productivity again, and I hope this effort helps someone! Also hopefully nobody minds me resurrecting my own thread but a quick glance at the main forum didn't show anything that looked more relevant.
Last edited by power crystals on Tue 26 Jul 2016 00:16, edited 1 time in total.

User avatar
Darkmil
Brigadier
Posts: 3025
Joined: Mon 29 Oct 2012 15:17
Location: Massy
Contact:

Re: Power Crystals' Modding Tools

Postby Darkmil » Mon 20 Jun 2016 08:13

Thank you very much !
Image

User avatar
The W:AB Noob
Lieutenant General
Posts: 4568
Joined: Fri 12 Jul 2013 22:29
Location: United States, Central Time Zone
Contact:

Re: Power Crystals' Modding Tools

Postby The W:AB Noob » Mon 20 Jun 2016 08:18

Image

Thanks a ton powercrystals!! So were maps editable before this except a value couldn't be removed? And can you give an example of a "__createdby"? (Sorry for all of these questions, I'm pretty excited to try this out.) But thanks for the "null" operator for people like me who can't be creative and think of a workaround.

Also, I never got to post this, but your Table Exporter is a life saver.
W:RD Sandbox Mod 5.4.3 Click -> Image

power crystals
Corporal
Posts: 42
Joined: Tue 8 Apr 2014 00:18
Contact:

Re: Power Crystals' Modding Tools

Postby power crystals » Tue 21 Jun 2016 00:05

I'll be honest, I don't actually know what trying to do a remove actually did before I fixed it. Probably not what you wanted though!

I had actually already updated my silly example patch to use __createdby (it used to use __order last), so take a look at that. It creates an entry in TUniteAuSolDescriptor using a patch named "New Unit", and then selects it via __createdby "new unit" (thus also demonstrating it is case-insensitive).

User avatar
The W:AB Noob
Lieutenant General
Posts: 4568
Joined: Fri 12 Jul 2013 22:29
Location: United States, Central Time Zone
Contact:

Re: Power Crystals' Modding Tools

Postby The W:AB Noob » Tue 21 Jun 2016 05:11

I have some more questions to ask you if you don't mind:
Using the XML patching tool, can we edit maps?
Okay, I was looking at the nested lists xml for the createdby example. So this makes a new instance? And is it possible to make new units now or was it just a random example?
Lastly, could it be possible to use the Data Exporter to import .xml files back in?

Thanks again powercrystals!
W:RD Sandbox Mod 5.4.3 Click -> Image

power crystals
Corporal
Posts: 42
Joined: Tue 8 Apr 2014 00:18
Contact:

Re: Power Crystals' Modding Tools

Postby power crystals » Wed 22 Jun 2016 00:00

The W:AB Noob wrote:Using the XML patching tool, can we edit maps?


Uh. Maybe? I don't know how to do that in the first place. This will let you do anything to ndfbin files (well, almost anything) but if maps are their own type you're out of luck. For now at least. All this thing does is automate stuff that the moddingsuite tool can already do, so anything it can I can theoretically make the patcher handle, but I'd still have to actually do that.

The W:AB Noob wrote:Okay, I was looking at the nested lists xml for the createdby example. So this makes a new instance? And is it possible to make new units now or was it just a random example?


Random example. I again don't know what's actually necessary for a new unit (true story: I've never actually made my own mod. I originally built this thing basically as a curiosity and was very surprised and confused that it became something that was actually used. I still don't know much of anything about how Wargame's internals work :P)

The W:AB Noob wrote:Lastly, could it be possible to use the Data Exporter to import .xml files back in?


I'm honestly not sure what you're asking me.

User avatar
The W:AB Noob
Lieutenant General
Posts: 4568
Joined: Fri 12 Jul 2013 22:29
Location: United States, Central Time Zone
Contact:

Re: Power Crystals' Modding Tools

Postby The W:AB Noob » Wed 22 Jun 2016 14:42

power crystals wrote:
The W:AB Noob wrote:Using the XML patching tool, can we edit maps?


Uh. Maybe? I don't know how to do that in the first place. This will let you do anything to ndfbin files (well, almost anything) but if maps are their own type you're out of luck. For now at least. All this thing does is automate stuff that the moddingsuite tool can already do, so anything it can I can theoretically make the patcher handle, but I'd still have to actually do that.

Maps are their own type but it is easy to do in the ModdingSuite. It's pretty much a variable attached with another value, typically a TableString or an Int32. Look at the TUniteAuSolDescriptor modules or TDeckRulesModifier for examples. There are also MapLists which is pretty much a list of maps.

power crystals wrote:
The W:AB Noob wrote:Okay, I was looking at the nested lists xml for the createdby example. So this makes a new instance? And is it possible to make new units now or was it just a random example?


Random example. I again don't know what's actually necessary for a new unit (true story: I've never actually made my own mod. I originally built this thing basically as a curiosity and was very surprised and confused that it became something that was actually used. I still don't know much of anything about how Wargame's internals work :P)

So __createdby makes a new instance?

power crystals wrote:
The W:AB Noob wrote:Lastly, could it be possible to use the Data Exporter to import .xml files back in?


I'm honestly not sure what you're asking me.

For some reason, I thought that xml's were extracted, not read. My bad.
W:RD Sandbox Mod 5.4.3 Click -> Image

power crystals
Corporal
Posts: 42
Joined: Tue 8 Apr 2014 00:18
Contact:

Re: Power Crystals' Modding Tools

Postby power crystals » Thu 23 Jun 2016 00:17

The W:AB Noob wrote:Maps are their own type but it is easy to do in the ModdingSuite. It's pretty much a variable attached with another value, typically a TableString or an Int32. Look at the TUniteAuSolDescriptor modules or TDeckRulesModifier for examples. There are also MapLists which is pretty much a list of maps.


Whoops. I thought you meant maps as in the the terrain :) Yes, you can absolutely do this. There's a map replacement example in the second example patch. You actually could always do this, but the absolute lack of documentation probably didn't explain it very well.

Unless it's a map of a map. MapLists are fine (they're just collections of maps in moddingsuite's internals), but there's currently no syntax for a map of a map directly. I don't think such a thing actually exists, though.

The W:AB Noob wrote:So __createdby makes a new instance?


It references an instance created by a previous <ndfcreate> element.

The W:AB Noob wrote:For some reason, I thought that xml's were extracted, not read. My bad.


It does have a "dump" mode, which kinda does what you were thinking of. Though it won't dump nested lists yet until someone asks for it and then I get to it at this rate somewhere around 2018.

User avatar
Bollywood Baloney
Chief Warrant Officer
Posts: 516
Joined: Thu 7 May 2015 21:32
Contact:

Re: Power Crystals' Modding Tools

Postby Bollywood Baloney » Fri 24 Jun 2016 21:16

I'm having a problem using the "select" command twice in the same XML file. Here are two versions of a ndfpatch that changes the T-80U's armory filters:

This one works

Code: Select all

<wargamepatch>
   <ndfpatch ndf="pc\ndf\patchable\gfx\everything.ndfbin" table="TTypeUnitModuleDescriptor" name="A">
      <matchconditions>
         <matchcondition property="NameInMenuToken">5F90780000000000</matchcondition>
         <matchcondition property="GenerateName">True</matchcondition>
      </matchconditions>
      <changes>
         <change operation="select" property="Filters" key="2"/>
         <change operation="clear" property="Filters" type="list"/>
         <change operation="append" property="Filters" type="LocalisationHash">41E22D4DD9380000</change>
         <change operation="append" property="Filters" type="LocalisationHash">5C76718B57360E00</change>
      </changes>
   </ndfpatch>
</wargamepatch>


This one does not

Code: Select all

<wargamepatch>
   <ndfpatch ndf="pc\ndf\patchable\gfx\everything.ndfbin" table="TTypeUnitModuleDescriptor" name="A">
      <matchconditions>
         <matchcondition property="NameInMenuToken">5F90780000000000</matchcondition>
         <matchcondition property="GenerateName">True</matchcondition>
      </matchconditions>
      <changes>
         <change operation="select" property="Filters" key="2"/>
         <change operation="clear" property="Filters" type="list"/>
         <change operation="append" property="Filters" type="LocalisationHash">41E22D4DD9380000</change>
         <change operation="append" property="Filters" type="LocalisationHash">5C76718B57360E00</change>
         
         <change operation="select" property="Filters" key="3"/>
         <change operation="clear" property="Filters" type="list"/>
         <change operation="append" property="Filters" type="LocalisationHash">41E22D4DD9380000</change>
         <change operation="append" property="Filters" type="LocalisationHash">5C76718B57360E00</change>
      </changes>
   </ndfpatch>
</wargamepatch>


And neither does this one

Code: Select all

<wargamepatch>
   <ndfpatch ndf="pc\ndf\patchable\gfx\everything.ndfbin" table="TTypeUnitModuleDescriptor" name="A">
      <matchconditions>
         <matchcondition property="NameInMenuToken">5F90780000000000</matchcondition>
         <matchcondition property="GenerateName">True</matchcondition>
      </matchconditions>
      <changes>
         <change operation="select" property="Filters" key="2"/>
         <change operation="clear" property="Filters" type="list"/>
         <change operation="append" property="Filters" type="LocalisationHash">41E22D4DD9380000</change>
         <change operation="append" property="Filters" type="LocalisationHash">5C76718B57360E00</change>
      </changes>
   </ndfpatch>
   <ndfpatch ndf="pc\ndf\patchable\gfx\everything.ndfbin" table="TTypeUnitModuleDescriptor" name="B">
      <matchconditions>
         <matchcondition property="NameInMenuToken">5F90780000000000</matchcondition>
         <matchcondition property="GenerateName">True</matchcondition>
      </matchconditions>
      <changes>
         <change operation="select" property="Filters" key="3"/>
         <change operation="clear" property="Filters" type="list"/>
         <change operation="append" property="Filters" type="LocalisationHash">41E22D4DD9380000</change>
         <change operation="append" property="Filters" type="LocalisationHash">5C76718B57360E00</change>
      </changes>
   </ndfpatch>
</wargamepatch>


I always receive a "ERROR: Unable to determine type while in nested context in patch (A or B, depending on the version) and specified type is invalid."

The armory filters are structured as follows:
[Filters property]
-List 1
--Filter 1
--Filter 2
--Filter n...
-List 2
--Filter 1
--Filter 2
--Filter n...
-List 3
--Filter 1
--Filter 2
--Filter n...
-List 4
--Filter 1
--Filter 2
--Filter n...

The filter values themselves are localization hashes. I'm able to write a patch that edits List 2 by itself or List 3 by itself, but not one that edits both list 2 and 3, no matter what I do.

The table is called TTypeUnitModuleDescriptor

Edit: I get those errors even if I strip out everything but the select operation lines

power crystals
Corporal
Posts: 42
Joined: Tue 8 Apr 2014 00:18
Contact:

Re: Power Crystals' Modding Tools

Postby power crystals » Sat 25 Jun 2016 23:36

Good catch - I forgot to tell it that a select doesn't need to know a type to do anything. If you stuck a type="list" (or anything, really) on it it would have worked, but that's dumb. http://www.wargamerandomdeck.com/tools/ is updated with a 2.0.1.

I also added an unselect operation since it wouldn't support what you're trying to do, but your idea was much nicer anyway. A second select is relative to the first, so use an unselect to go back up one level.

Finally, you don't need property names inside of a select'ed context. They don't hurt, either, but it's extra noise. Here's your "A" patch with the above changes:

Code: Select all

<wargamepatch>
   <ndfpatch ndf="pc\ndf\patchable\gfx\everything.ndfbin" table="TTypeUnitModuleDescriptor" name="A">
      <matchconditions>
         <matchcondition property="NameInMenuToken">5F90780000000000</matchcondition>
         <matchcondition property="GenerateName">True</matchcondition>
      </matchconditions>
      <changes>
         <change operation="select" property="Filters" key="2"/>
         <change operation="clear" type="list"/>
         <change operation="append" type="LocalisationHash">41E22D4DD9380000</change>
         <change operation="append" type="LocalisationHash">5C76718B57360E00</change>

         <change operation="unselect" />
         
         <change operation="select" property="Filters" key="3"/>
         <change operation="clear" type="list"/>
         <change operation="append" type="LocalisationHash">41E22D4DD9380000</change>
         <change operation="append" type="LocalisationHash">5C76718B57360E00</change>
      </changes>
   </ndfpatch>
</wargamepatch>


It now runs with no errors and I verified the output is what it should be. I didn't try "B" but it should be the same thing.

edit: wait, no, B's because I never actually cleared the select context between patches. Whoops. That should also work now.

Return to “Modding”

Who is online

Users browsing this forum: No registered users and 7 guests