Converting GH3 tracks to FOF - New format used in GH3 ?

[YMS]

can it be added into yms's converter?

By now I'd prefer to see Pudding's converter being developed further, rather than mine. It's simpler, and since it's written in Python it could be integrated into FOF easily.
It would be possible to update mine, but why maintain two programs doing the same?

If I can help with Pudding's, I will.


PS: In the long run, we should care to get any information out of the song's PAK, and any other information that there is, at least as long as it is related to gameplay. There seem to be some data-chunks about each song in pak/pak/qb.pak.ps2, in pak/pak/qb.pab.ps2, and especially in pak/pak/dbg.pak.ps2. We should take a look at those sometime, too.

[debr5836]

So the reason that the songs work with the constant tempo is that the integers in the qb file are simply millisecond offsets from the start of the song. Thus, they will time really well with the music of a 125bpm song with 480 ticks per quarter note (125bpm * 480 ticks/beat = 60000 ticks per minute = 1000 ticks per second, so 1 tick == 1 millisecond), but they won't be aligned to the beat marks on the fretboard.

In order to get these to align, we would have to both alter the tempo per measure (like in GH2), and then figure out where in each measure the notes are. I'm working on that right now for my charts (today is a slow day at work)-- I'll post a link to my PERL code once I'm done (probably sometime this weekend).

[debr5836]

PS: In the long run, we should care to get any information out of the song's PAK, and any other information that there is, at least as long as it is related to gameplay. There seem to be some data-chunks about each song in pak/pak/qb.pak.ps2, in pak/pak/qb.pab.ps2, and especially in pak/pak/dbg.pak.ps2. We should take a look at those sometime, too.

So far, I think we have:

a) All the notes for all the tracks
b) All the SP phrases
c) All the battle mode phrases (at least something to do with battle mode)
d) The tempos
e) The time signatures

Things I am missing (perhaps in the qb file, perhaps somewhere else in the .pak file) -- others may have figured this out already, but I'm pretty slow here:

a) Section Titles
b) HO/PO overrides (perhaps the 6th bit of the note id is for this??)
c) Animation and lighting events
d) When in battle mode do you switch between player 1 and player 2 and unison play?

[FTGF]

Well its been annouced the DLC will have 3 Foo fighters songs, 3 velvet revolver songs, and the boss battles for quickplay. Hopefully not in a new format.

[tma]

Well its been annouced the DLC will have 3 Foo fighters songs, 3 velvet revolver songs, and the boss battles for quickplay. Hopefully not in a new format.

It won't be a new format in the sense that the midi files etc will be the same, however they will be contained in an Xbox 360 container (easy enough to open) however the files themselves may be encrypted, which means little chance of success unless we can find the private key (which will either be an Xbox 360 thing (buckley's chance) or something specific to GH3 (better chance, but still difficult).

The GH2 DLC was not encrypted, but as it's a new developer all bets are off. We'll know more when it's available.

[tibiazak]

question: when you guys converted the GH3 songs, were you using the PS2 version or the 360 version, or did that matter? because i'm itching to play Tina by Flyleaf (unreleased content from one of my favorite bands :p ) but it's only available as a download on xbox live after buying the GH3 soundtrack. (sorry if somebody mentioned this before, too lazy to look through the thread tonight)

[tma]

question: when you guys converted the GH3 songs, were you using the PS2 version or the 360 version, or did that matter? because i'm itching to play Tina by Flyleaf (unreleased content from one of my favorite bands :p ) but it's only available as a download on xbox live after buying the GH3 soundtrack. (sorry if somebody mentioned this before, too lazy to look through the thread tonight)

PS2 version. I don't have access to the Xbox 360 version yet, and the Demo version files are encrypted and can't be used.

Once I've got access to the 360 version (late next week) and the DLC (hopefully around the same time) I'll be able to say more on whether it's possible to extract them.

[GameZelda2]

Unbelievable work ??? . When I looked to the files, I was thinking that they were very obfuscated.

About MSV files
Hopefully I can help on anything. I have a MSV2VAG conversor in C (I've designed it to convert the tracks to WAV automatically; I should code it later)

My plans are actually to write a MSV encoder and test if it works on the game.

About HED/WAD files
I also have a question... I have also a HED/WAD extractor. I also can write a rebuiler, but I have a big doubt. The format is: DWORD offset, DWORD size, and a string containing the filename (variable length, NULL-terminated), that's evident.
But after the end of the string and between the DWORD containing size, there are some zeros, the problem is the quantity of them. Sometimes there's only one zero, but sometimes 4, etc... it's relatively easy unpack without knowing this number of 0s (My unpacker finds the next slash, then goes 8 bytes before it). But I need to know how those 0 work in order to create a new file.
So the big question is... of what depends the quantity of 0es? Can they be removed/added without damaging the file? etc...

About image (.img.ps2) files

I haven't too much info about them, but I've determined the width and heigth offsets and where the palette starts. They seem to be indexed color like bmp_ps2 files. The palette also has the same particularity than them (Deordered colors & indexes).

I've written a tool that converts SOME images. I can convert almost every file manually with the hex editor, but I can't determine automatically where the image data part starts, so not too much about them.

About X360 charts (Demo)

The charts are on the PIRS file, data/compressed/songs/anything.pak.xen

The files are completely (from offset 0 to the end) compressed with Zlib. After decompressing them, you can get "comprensible"PAK files. I haven't looked too much, but the header doesn't contain file names. After that, there's the chart (logically). The data about the notes seem to be in the same format than the PS2. And again, I haven't checked too much them.

About anything

If I can help on anything, tell me about it :;):

I know PHP and the basic things of C, and I have mediocre Hex-Editing skills.

Sorry for that long post

[kajsoldier]

"playable content"? I downloaded the gh3 songs off the other thread and extracted them to the right place, I read all the posts on both threads and still I can't get it to work. It's probably too complicated to normal Pc users such as myself :p Ill just wait for the "Play GH3 songs on FoF for dummies" thread. Anyway thx for all the work and Flyleaf ftw

[tma]

About HED/WAD files
I also have a question... I have also a HED/WAD extractor. I also can write a rebuiler, but I have a big doubt. The format is: DWORD offset, DWORD size, and a string containing the filename (variable length, NULL-terminated), that's evident.
But after the end of the string and between the DWORD containing size, there are some zeros, the problem is the quantity of them. Sometimes there's only one zero, but sometimes 4, etc... it's relatively easy unpack without knowing this number of 0s (My unpacker finds the next slash, then goes 8 bytes before it). But I need to know how those 0 work in order to create a new file.
So the big question is... of what depends the quantity of 0es? Can they be removed/added without damaging the file? etc...

The quantity of zeros appears to be padded out to the next DWORD. Once you put the filename in, add the terminating zero, then you need to add 0-3 more to make it fall on a DWORD (4 byte) boundary.

About X360 charts (Demo)

The charts are on the PIRS file, data/compressed/songs/anything.pak.xen

The files are completely (from offset 0 to the end) compressed with Zlib. After decompressing them, you can get "comprensible"PAK files. I haven't looked too much, but the header doesn't contain file names. After that, there's the chart (logically). The data about the notes seem to be in the same format than the PS2. And again, I haven't checked too much them.

Zlib?! You're kidding. I thought I tried that already, I'll have to take another look. Thanks!

[tma]

Quick note: pudding's pak extraction has a major bug. The file offset is incorrectly calculated, meaning the end of the first file in the pak is cut off, and every file after the first will be incorrect.

The offset is actually from the position of the header entry, not from the start of the file, so the second file in the pak starts at offset + 0xC0 (each header is 0xC0 bytes in length), the third is at offset + 0x180 (2 * 0xc0), etc.

I'm going to write my own pak extraction tool in perl over the next hour or so, I'll link it here when it's done.

[GameZelda2]

Mmm, I was going to do the same thing ( a PAK extractor) . Anyway, I want to talk about another thing (Not related to FoF, but...)

Yesterday I spent 2 hours writing a HED Extractor AND Rebuilder. After doing it, logically I've tested it it works, unpacking and packing the original files, and the result unpacked fine (even if the size of the WAD wasn't the same).

(Link -> http://www.savefile.com/files/1170845). Source in C included. Usage "tool.exe DIRECTORY file.hed file.wad" for both.
But in the game, it doesn't work. The game seems to start, but nothing is shower/any sound/anything.

So I've decided to try to modify my tool to generate IDENTICAL files to the original ones, so a action of unpacking/packing should produce a identical file to the original.

I've solved one of the things (that was a stupid thing, if the size was multiple of 2048, it added a extra sector + his 2048 zeroes, this doesn't happen in the original file.

But now there seems to be (ANOTHER) thing that makes it different: The first PAK reference has a bad offset (coincidence???).

Sector previous file: 87310
Filesize: 23312

With maths:

Ceil(FILESIZE / 2048) = 12
87310+12=87322

... But in the game, it's at 87328. On the WAD, those +6 sectors are pure zeroes.

This happens with a lot of files more. Anyone can check this and see if it has any logical reasoning?

[tma]

Mmm, I was going to do the same thing ( a PAK extractor) . Anyway, I want to talk about another thing (Not related to FoF, but...)

Yesterday I spent 2 hours writing a HED Extractor AND Rebuilder. After doing it, logically I've tested it it works, unpacking and packing the original files, and the result unpacked fine (even if the size of the WAD wasn't the same).

(Link -> http://www.savefile.com/files/1170845). Source in C included. Usage "tool.exe DIRECTORY file.hed file.wad" for both.
But in the game, it doesn't work. The game seems to start, but nothing is shower/any sound/anything.

So I've decided to try to modify my tool to generate IDENTICAL files to the original ones, so a action of unpacking/packing should produce a identical file to the original.

I've solved one of the things (that was a stupid thing, if the size was multiple of 2048, it added a extra sector + his 2048 zeroes, this doesn't happen in the original file.

But now there seems to be (ANOTHER) thing that makes it different: The first PAK reference has a bad offset (coincidence???).

Sector previous file: 87310
Filesize: 23312

With maths:

Ceil(FILESIZE / 2048) = 12
87310+12=87322

... But in the game, it's at 87328. On the WAD, those +6 sectors are pure zeroes.

This happens with a lot of files more. Anyone can check this and see if it has any logical reasoning?

I've nearly finished the pak extraction perl script, however I am finding a very similar thing within the PAK/PAB files as you are finding in the HED/WAD files, in that there is often extra nulls between files. I don't see any particularly logical reason for it at this stage, and I'm just skipping them.

While I've got you, do you have something that will unpack the zlibed X360 files? I started to delve into it with Perl but it didn't seem all that straightforward and I wanted to focus on the PAK files first.

[GameZelda2]

Mmm, I have a PHP script, here it is if it helps you (you will need to install Apache+PHP (or you can use WAMP).

Code: Select all


<?php
// Zlib decompress, for GH3...!
if (empty($_GET["file"]))
{
   if ($handle = opendir('compress/'))
   {
      echo '<b>Compress FOLDER:</b><br /><br />';
      while (false !== ($file = readdir($handle)))
      {
         if ($file != "." && $file != ".." && $file != "uncompress" && $file != "bforced")
         {
            echo '<a href="zlibsimple.php?file=' . $file . '">' . $file . '</a><br />';
         }
        }
    }
    closedir($handle);
}
else
{
   $data = file_get_contents("compress/" . $_GET["file"]);
   $decompress = gzinflate($data);
   file_put_contents("compress/uncompress_" . $_GET["file"], $decompress);
   echo 'File created on compress/uncompress_' . $_GET["file"] . '!!!<br /><a href="zlib.php">Return</a>';
}
?>

The script should be called "zlibsimple.php" and the compressed files should be put to a folder called "compress".

If you want, I can try to port it to C.

[tma]

Mmm, I have a PHP script, here it is if it helps you (you will need to install Apache+PHP (or you can use WAMP).

Code: Select all


<?php
// Zlib decompress, for GH3...!
if (empty($_GET["file"]))
{
   if ($handle = opendir('compress/'))
   {
      echo '<b>Compress FOLDER:</b><br /><br />';
      while (false !== ($file = readdir($handle)))
      {
         if ($file != "." && $file != ".." && $file != "uncompress" && $file != "bforced")
         {
            echo '<a href="zlibsimple.php?file=' . $file . '">' . $file . '</a><br />';
         }
        }
    }
    closedir($handle);
}
else
{
   $data = file_get_contents("compress/" . $_GET["file"]);
   $decompress = gzinflate($data);
   file_put_contents("compress/uncompress_" . $_GET["file"], $decompress);
   echo 'File created on compress/uncompress_' . $_GET["file"] . '!!!<br /><a href="zlib.php">Return</a>';
}
?>

The script should be called "zlibsimple.php" and the compressed files should be put to a folder called "compress".

If you want, I can try to port it to C.

Thanks for that, I'll have a look at it once I've finished the DEPAK script (very close now).