Skip to content
This repository has been archived by the owner on Dec 22, 2024. It is now read-only.

[Help]: .unit format discovery thread #1

Open
Xaymar opened this issue Feb 25, 2024 · 31 comments
Open

[Help]: .unit format discovery thread #1

Xaymar opened this issue Feb 25, 2024 · 31 comments
Labels
type:help Further information is requested

Comments

@Xaymar
Copy link
Owner

Xaymar commented Feb 25, 2024

Everything about the .unit file structure.

Known Information
  • X is Up, forward is either Z or Y
  • Vertex Data:
    • Types:
      • 0: Position
      • 1: Color or Normal
      • 4: Texture Coordinates
      • 6: Unknown
      • 7: Unknown
    • Formats:
      • 2: float32[3]
      • 26: uint8[4] or sint[4]
      • 29: float16[2]
      • 24: Unknown
        • Only seen together with type 6 or 7. Usually no space left at this point.
Related links:
@Xaymar
Copy link
Owner Author

Xaymar commented Feb 25, 2024

This mesh should have animation data: 0x990B45D5D75FFF3A, 28. File contains errors #4, #6 and #3. The values in 0x7, 0x1f suggest it is indeed (type_2b_wide)[4], but the order is not exactly described. It looks like it'd be half, though it is unclear what the 4th value is for. Most of the time, it is 0, but rarely it is non-zero in the "exact" amount that's missing from [nX, nY, nZ].

@Xaymar

This comment was marked as resolved.

@Xaymar

This comment was marked as outdated.

@Xaymar

This comment was marked as outdated.

@Xaymar

This comment was marked as outdated.

@Xaymar

This comment was marked as outdated.

@Xaymar

This comment was marked as outdated.

@Xaymar

This comment was marked as outdated.

@Xaymar

This comment was marked as resolved.

@Xaymar
Copy link
Owner Author

Xaymar commented Feb 26, 2024

We've had the mesh structure all wrong, and didn't notice it until now.

  1. The "offsets" entry is not relative to count, but relative to the list.
  2. Each mesh starts with two uint32s.
  3. There can be multiple materials.
    • 0x64 is the number of materials, 0x68 is the offset into the structure -4.
  4. There can be multiple vertex groups.
    • 0x74 is the number of groups, 0x78 is the offset into the structure.

This is big. It means that several meshes were incomplete by default, since we assumed there was only one set of vertex/index information. I'm not sure why the vertex_offset and vertex_count repeat as well, but they haven't changed yet.

@Xaymar

This comment was marked as outdated.

@Xaymar

This comment was marked as outdated.

@Xaymar
Copy link
Owner Author

Xaymar commented Feb 26, 2024

Hashes appear to be MurmurHash(2) 64bit version. Not quite sure how to implement it, but I'll focus on proper exports for now.

@Xaymar Xaymar changed the title .MESHINFO/.MESH format discovery thread .unit format discovery thread Feb 27, 2024
@Xaymar

This comment was marked as resolved.

@Xaymar
Copy link
Owner Author

Xaymar commented Feb 29, 2024

The matrices and vectors in __unk1 seem to be related to something. There's some kind of relationship tree happening there, but I don't quite understand how it works yet.

@Xaymar
Copy link
Owner Author

Xaymar commented Feb 29, 2024

Similarly, __unk6 seems to belong to the same type of relationship.

@Xaymar
Copy link
Owner Author

Xaymar commented Feb 29, 2024

The end of __unk1 overlaps with __unk5 if __unk5 is present.

@Xaymar
Copy link
Owner Author

Xaymar commented Feb 29, 2024

No apparent relation between bones file and __unk1. Number of bones doesn't ever match up with the number stored in there.

@Xaymar
Copy link
Owner Author

Xaymar commented Feb 29, 2024

I'm probably approaching this from the wrong angle. I'm assuming that the game engine uses a model format, but this is clearly multiple models in one. It could very well have merged all the meshes into one cluster. It doesn't quite explain where animation and bone data goes though.

As far as I can tell, each of these .unit files is usually accompanied by .bones, .state_machine, .physics and .dfde6a8797b4c0ea. For units without a .bones file, the ones that are present are .physics and .dfde6a8797b4c0ea. For units with a .bones files, the .dfde6a8797b4c0ea is replaced with .state_machine often.

@Xaymar
Copy link
Owner Author

Xaymar commented Feb 29, 2024

We also see .336bdb8766bd591d accompanying the same file - possibly another part of the .unit file.

@Xaymar
Copy link
Owner Author

Xaymar commented Feb 29, 2024

Also of note is that several of the files are very clearly Havok Physics, and they all seem to be related to the bones in the model.

@Xaymar
Copy link
Owner Author

Xaymar commented Mar 1, 2024

May have found some sort of LoD or similar?

__unk0.__unk_data[].__unk_data[].data[] is always lower than the maximum amount of meshes. Seems to cluster together around meshes.

@Xaymar
Copy link
Owner Author

Xaymar commented Mar 1, 2024

__unk5 appears to also limit itself to the number of meshes in the unit file.

@Xaymar Xaymar changed the title .unit format discovery thread [Help]: .unit format discovery thread Mar 1, 2024
@Xaymar Xaymar added the type:help Further information is requested label Mar 1, 2024
@Xaymar
Copy link
Owner Author

Xaymar commented Mar 1, 2024

Unfortunately my theory about type 7 and 6 not fitting in the vertex stride was destroyed by another file having a vertex stride where they fit again.

@Xaymar
Copy link
Owner Author

Xaymar commented Mar 1, 2024

Format 24 might be "Bone Indices" and Format 25 is "Bone Weights". These both appear to be u8vec4_t

@Xaymar
Copy link
Owner Author

Xaymar commented Mar 1, 2024

Format 24, 25 and 26 are still unknown. 25 is definitely not bone indices, as either there are 65535 possible bones, or most models refer to bone 255 and assign a weight of 0 to it - assuming 24 is bone weights.

@Xaymar
Copy link
Owner Author

Xaymar commented Mar 1, 2024

Formats:
  • 1: float[2]
  • 2: float[3]
  • 4: ???
    • Can't be longer than 4 bytes.
  • 24: ???
    • Can't be longer than 4 bytes.
  • 25: ???
    • Can't be longer than 4 bytes.
  • 26: ???
    • Can't be longer than 4 bytes.
  • 29: half[2]
Types:
  • 0: Position
    • Known formats: 2
  • 1: ?
    • Known formats: 26
  • 4: Texture Coordinates
    • Known formats: 1, 29
  • 5: ?
    • Known formats: 4
  • 6: ?
    • Known formats: 24
    • Related to .bones file
  • 7: ?
    • Known formats: 25
    • Related to .bones file

@Xaymar
Copy link
Owner Author

Xaymar commented Mar 1, 2024

content/objectives/obj_common/gneeric_intel/generic_intel.unit
  • No .bones file, and entry for .bones is empty.
    • Coincidentally, no mention of type 6 and 7 suggesting a relation between the two.
  • Type 1 is still present.
  • Type 5 is also present.

@Xaymar
Copy link
Owner Author

Xaymar commented Mar 1, 2024

content/fac_helldivers/vehicles/lav/lav.unit
  • Proof that gpu_resources is treated separately: All data is off by 4 bytes with 8 byte offset, but it's off by -4 in other files if we use 12 byte offset.

@Xaymar
Copy link
Owner Author

Xaymar commented Mar 1, 2024

content/fac_helldives/vehicles/combat_walker/combat_walker.unit
  • __unk1.__unkf32mat4x4_1 may be Translation, Rotation and Scale in one Matrix?
  • __unk1.__unk_u16vec2_0 may be u16vec2_t determining some sort of relationship between nodes.
    1. Unknown, seems to always be 0 or 1?
    2. The parent node to attach to.
  • __unk1.__unkcommonhash seems to be the hash table for nodes.
    • This is suggested by the 7th mesh having a hash entry that is at location 102 in the above table.

Xaymar added a commit that referenced this issue Mar 1, 2024
Tough nut to crack, but I've finally made progress. Turns out it wasn't two 4x4 matrices, but instead one structure containing the data to make the matrix, and an actual list of precalculated matrices. This could have been used to offset the abysmal performance of the AMD Bulldozer CPUs/APUs that were in use around the time the engine was alive and maintained.
@Xaymar
Copy link
Owner Author

Xaymar commented Mar 1, 2024

We now have a reasonably deep understanding to reproduce a mesh tree in glTF format.

Xaymar added a commit that referenced this issue Mar 2, 2024
All of this will aid in the correct conversion of the mesh in the end.
Xaymar added a commit that referenced this issue Apr 12, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
type:help Further information is requested
Projects
None yet
Development

No branches or pull requests

1 participant