Collision data hacking

CaptainBee · Aug 20, 2015 4:18 AM

does this work with epsxe 180? i downloaded version 160 to try and i still cant get this to work. epsxe 160 just crashes and epsxe 180 just flashes blue and green. im stuck at the ramtop.bat part, it says "ram not found"

CrystalFissure · Aug 20, 2015 5:09 AM

Try the new ePSXe. But it should work with the older versions, not sure why it doesn't.

aleksusklim · Aug 20, 2015 9:25 PM

I played with ground types in Sunny Villa’s skatepark. Types are stored in fourth part.
I found out thah some of numbers are really responsible for orange/blue ramp, so filling all the values with the same number can give me that everything physically turns into blue ramp (then every my jump from a corner would pull me straight up) or orange ramp (so I hen jump really far); but I couldn’t jump by X key where I want – only from corners.

Other values mean nothing special (no ramps at all), some values cause emulator crash.
I looked at disassembly – main collision function just read one of the values (likely one that belongs to current triangle where collision occurs) but simply stores it somewhere in the memory. It used by other functions…

Then I found a function that uses this value as an offset to pointer, where stored other numbers…

But one question is bothered me all the time: if every triangle seems to have a type, then why there’s less element in types array that triangles in a list?

In the beginning of collision data, there are number of triangles in 3-part, number of bytes in 4-part and number of bytes in 5-part (almost everywhere in game binary code 4-part and 5-part are somehow used together). For Sunny Villa Skatepark it’s: 6030 triangles, 5474 types and 5340 units in 5-part.

Why there are more triangles then types!?

Then I got a crazy idea: what if in our big list of triangles, there are "essential" ones located before; and then – "simple" triangles with no specific type? So, I loaded full model in (still beta…) SWV and also compared it with model only with 5474-triangles:

http://klimaleksus.narod.ru/Files/H/grid_04.jpg
http://klimaleksus.narod.ru/Files/H/grid_05.jpg

It DOES make sense! There are invisible walls that restrict skatepark arena (to disallow player from falling off the level). And every other triangle has a type, but these restrictions are not! Really, they don’t need a type.

Also, they are more often have that strange type-bit (in triangle, that is a pair to water-bit, yellow on my screens).

So, my theory is that first N triangles are typed triangles, where N is the size of 4-part.

New questions: how types are working? Seem that if it’s a small number – it is an offset to something; if big – maybe bit-mask. But I’m not sure. F.e. 01 was blue ramp, 02 orange ramp, 03 = crash, 04 = working. Looks like bit-mask (floor cant be blue+orange=1+2=3), but values like FF or 3F are working too. Also, type "3F" is hard-coded in game (and then number is replaced with FF, or shifted left by 2 bits if was not equal to 3F – this is in one of functions that reads this stored earlier value from memory, and not directly from triangle).

CrystalFissure wrote:

By the way, I made a video of the Spyro Freeze Cheat!

Thanks a lot!
But I wonder why you didn’t show what happens if a level would be loaded with active cheat (when all gems are red and Sparx is black).

Also, mirror my download link (re-upload my file to any different place) if you can – some people sometimes tell that they for some reason couldn’t download anything from my site.

CaptainBee wrote:

im stuck at the ramtop.bat part, it says "ram not found"

Are you sure that you did everything correct? What kind of RAMTOP did you use: EXE/PSX/BIN/ISO ?

Sheep · Aug 22, 2015 11:09 AM

aleksusklim wrote:

So, my theory is that first N triangles are typed triangles, where N is the size of 4-part.

Yep, this has seemed quite clear to me as well for a while smile All special triangles(like ramps, lava etc.), and also all the completely normal triangles that are listed before or between special triangles, are given a value. Are you able to tell how the 'normal terrain' type value is used in the game code?

aleksusklim wrote:

New questions: how types are working? Seem that if it’s a small number – it is an offset to something; if big – maybe bit-mask. But I’m not sure. F.e. 01 was blue ramp, 02 orange ramp, 03 = crash, 04 = working. Looks like bit-mask (floor cant be blue+orange=1+2=3), but values like FF or 3F are working too. Also, type "3F" is hard-coded in game (and then number is replaced with FF, or shifted left by 2 bits if was not equal to 3F – this is in one of functions that reads this stored earlier value from memory, and not directly from triangle).

All I know is that a value does not necessarily mean the same thing in different levels. And for portals, there are two different ranges of values that both work as portal values, leading to the same levels. So could be they either just lead through the portal from two different sides, or maybe some of the bits are "ignored".

Also, if you're able to edit the game's files and then run it, does anything appear different if you change the value of the water-bit's twin? Since nothing seemed to change when I edited it at run time, I suspect they are used only when the level loads.

CrystalFissure · Aug 23, 2015 6:13 AM

aleksusklim wrote:

But I wonder why you didn’t show what happens if a level would be loaded with active cheat (when all gems are red and Sparx is black).

I didn't realise you could do that! So do you pause everything while in a loading screen, then load the level? I'll re-read the Readme to see if you mention it, and then I'll make a video. Keep up the great work!
Also, someone pointed out that this freeze cheat has been done before, years ago. I actually commented on it:

But I'm pretty sure yours is better, as you can manually choose to unfreeze particles, and also slow down movements. Not sure whether this freeze hack can do the same thing with the superflame!

aleksusklim · Aug 25, 2015 5:38 AM

Good news! Now I know how collision-world animation is implied.

Every level can have some block of data, that represents animated parts of collision ground, for example bridges or portals (that appears only after collecting enough eggs). Every block consists of two (or more) frames that represent "states". For example, a portal can be disabled underground, or be enabled at its proper place.

Earlier I found out that all animated parts in collision model – is a total mess! Only relative place of bunch of triangles is hardly represents the original object. Then I dumped the collision model from emulator memory and loaded it to viewer. All triangles were on their normal positions! That means that model in memory is changed at level loading time.

So I searched a function that changes them. I found two things: one is working after an action and changes one block (a portal right after talking to person with enough eggs); second changes everything at level-loading time. Bingo!

There was a function that goes through fourth (4,6,8,10) sub-subfile and "jumps" (sections with sizeof() as first integer) with changing something. Also seems that this thing cares about visible world model too, and after it – collision. Actually, collision is the last thing that is modified at this function. So I had to decompile and lean about hundred of assembler lines of code.

Wow, I forgot to explain some of MIPS when I explained it last time. There is a multiplication command "mult a0,v0", which multiplicities two registers and stores the result to special "LO" register. There’s also "HI" register where will be upper part of 64-bit multiplication result (it like a pair of two 32-bit registers that works like a 64-bit register, and the product is stored there). Also division works with them, giving integer result and reminder.

But MIPS specification said that result will not be ready immediately, but need to take an undefined (platform-specific) amount of additional cycles (lines) to prepare. Looks like the game gives multiplication a three line to process operation. But those were small numbers and was used only LO-part, so maybe for big multipliers somewhere else would be more lines between actual multiplication and taking the product. (Again I think that in Epsxe "mult" will be done in 1 instruction just as "lw" that originally needs 2 instructions, but I never tested any my code with integer multiplication or division. And how to take sqrt() or sin() I don’t know either…)
There are commands to use LO-HI registers: to put and get any value – "mflo at" ("at = LO"); "mtlo t1" ("LO = t1");"mfhi ra" ("ra = HI");"mthi s3" ("HI = s3");

So, OK. That function multiplies length of data in one frame-state to current frame-state index with "mult". Also it uses LO register as a temporary storage for a value (I see this first time). But I don’t want to show disassembly this time, I just can say that for GH version this function is at 0x80021a94, but collision-related stuff located from 0x80021ce8 till 0x80021ef4. There are two main part of logic: the short and tricky reading of header, and then long but simple processing the triangles’ data.

The header of data is very uncommon; this function uses not all bytes of it (maybe others are for in-game changing of frame-states or even for interpolating vertexes). Main part is 12-triangles in more strange format that ours in collision list. Every triangle has to be parsed to X1/X2/X3/ Y1/Y2/Y3/Z1/Z2/Z3/T components and then combined again. Here is the format for Spyro3 (32-bits; not in-data bytes order, which is inverted)

Cccc-cccc = cBbb-bbbb = bbTA-aaaa = aaaa-aaaa

C = 3-vertex (relative), 9-bits
B = 2-vertex (relative), 9-bits
A = 1-vertex (absolute), 13-bits
T = water-type bit.

This format is very close to X and Y, but needs to be fixed for Z coordinate (where 2-3 vertexes are 8-bit). Water-type is shifted one bit to left, and then the most significant bit (15) of our types (that one bit left after water-type) is turned to =1. So, all resulting triangles will have that strange bit set. And neither will be "water" (at least, not portals. Maybe in Spyro2 at Aquaria Towers…), because T-bit = 0 (where I looked).

You know what? In any of z2 or z3 was negative (all 9 bits are used) – here comes the pain!! Everything is messed up with subtraction and addition… Well, I’ll show the code:

(also look at my article about implementing arithmetic right shift: http://stackoverflow.com/questions/32189509/ )

uses Classes,SysUtils;

function srl(a,b:Integer):Integer;overload;
begin
Result := (a shr b) and ((1 shl (32-b))-1);
end;

function sra(a,b:Integer):Integer;overload;
begin
Result := (a shr b) or (( 0-((a shr 31) and 1)) shl (32-b));
end;

function srl(a,b:Cardinal):Cardinal;overload;
begin
Result := (a shr b) and ((1 shl (32-b))-1);
end;

function sra(a,b:Cardinal):Cardinal;overload;
begin
Result := (a shr b) or (( 0-((a shr 31) and 1)) shl (32-b));
end;

function sll(a,b:Cardinal):Cardinal;overload;
begin
Result := a shl b;
end;

function sll(a,b:Integer):Integer;overload;
begin
Result := a shl b;
end;

function bits(b:integer):integer;
begin
Result:=(1 shl b)-1;
end;

var
s1,s2:string;
stream,save:Tfilestream;
cnt,oldpos,start,next,word1,word2,word3,mul1,mul2,off1,off2,x,y,z,z1,z2,z3,x1,x2,x3,y1,y2,y3,h:integer;

procedure jump(cnt:integer);
var i:integer;
off:Cardinal;
begin
for i:=1 to cnt do begin
stream.ReadBuffer(off,4);
stream.Seek(off-4,soFromCurrent);
//if stream.Position>size then Abort;
end;
end;

procedure move(off:Integer);
begin
stream.Seek(off,soFromCurrent);
//if stream.Position>size then Abort;
end;


begin
s1:='4_sub-subfile_of_Spyro';
s2:='collision_third_part.3';
stream:=TFileStream.Create(s1,fmOpenRead or fmShareDenyNone);
save:=TFileStream.Create(s2,fmOpenReadWrite or fmShareDenyNone);
stream.Seek(0,soFromBeginning); // Assume we are at fourth sub-subfile

move(48); // for Spyro3
jump(5);
{
move(44); // for Spyro2 instead
jump(4);

move(136); // for Spyro1 instead
jump(4);
}

move(4);
start:=stream.Position;
stream.ReadBuffer(cnt,4);
while(cnt>0)do begin
Dec(cnt);
stream.ReadBuffer(next,4);
oldpos:=stream.Position;
stream.Seek(start+next,soFromBeginning);
stream.ReadBuffer(word1,4);
stream.ReadBuffer(word2,4);
stream.ReadBuffer(word3,4);
mul1:=srl(word1,16)and bits(8);
off1:=word2 and bits(16);
off2:=srl(word2,16);
stream.Seek(mul1*8+5,soFromCurrent);
stream.ReadBuffer(mul2,1);
// mul2:=0; // uncomment this line to use the first frame-state
stream.Seek(start+next+word3+off1*12*mul2,soFromBeginning);
save.Seek(off2*12,soFromBeginning);
while(off1>0)do begin
Dec(off1);
stream.ReadBuffer(x,4);
stream.ReadBuffer(y,4);
stream.ReadBuffer(z,4);
h:=sll((z and (1 shl 13)),1) or (1 shl 15);
z3:=sra(z,23);
z2:=sra(sll(z,9),23);
z1:=z and bits(13);
if(z2>=0)and(z3>=0)then begin
z2:=sll(z2,16);
z3:=sll(z3,24);
z:=z1 or z2 or z3 or h;
end else begin
x3:=sra(x,23);
x2:=sra(sll(x,9),23);
x1:=x and bits(14);
y3:=sra(y,23);
y2:=sra(sll(y,9),23);
y1:=y and bits(14);
if z3>=z2 then begin
Inc(x1,x2);
x:=sll(-x2,23);
x2:=sll((x3-x2)and bits(9),14);
x3:=x;
x:=x1 or x2 or x3;
y:=sll(-y2,23);
Inc(y1,y2);
y2:=sll((y3-y2)and bits(9),14);
y3:=y;
y:=y1 or y2 or y3;
Inc(z1,z2);
z:=sll(-z2,24);
z2:=sll((z3-z2)and bits(9),16);
z3:=z;
z:=z1 or z2 or z3 or h;
end else begin
Inc(x1,x3);
x:=sll(x2-x3,23);
x2:=sll((-x3)and bits(9),14);
x3:=x;
x:=x1 or x2 or x3;
Inc(y1,y3);
y:=sll(y2-y3,23);
y2:=sll((-y3)and bits(9),14);
y3:=y;
y:=y1 or y2 or y3;
Inc(z1,z3);
z:=sll(z2-z3,24);
z2:=sll((-z3)and bits(9),16);
z3:=z;
z:=z1 or z2 or z3 or h;
end;
end;
save.WriteBuffer(x,4);
save.WriteBuffer(y,4);
save.WriteBuffer(z,4);
end;
stream.Seek(oldpos,soFromBeginning);
end;
stream.Free;
save.Free;
end.

Assume we decoded x1,x2,x3;y1,y2,y3;z1,z2,z3 as signed integers from animated data. Then if both z1>=0 and z2>=0 we can just use it and encode as triangle for a list.

If not, then we check, z2<z3? If so, then: Nc1=c1+c2; Nc2=c3-c2; Nc3=-c2; if not then: Nc1=c1+c3; Nc2=-c3; Nc3=c2-c3. Where "c" is x,y,z – I mean we must do this with every coordinate. So, Nc1, Nc2, Nc3 now are proper values for X,Y or Z, and we should encode them to a triangle.

It’s not so hard, but (especially in game assembler) rather long. Also I don’t know, for what all of it? Maybe in animation there are "normalized" values for proper interpolation between frames, but in final collision model there must be "optimized" vertexes, where z1<=z3<=z2 for quick discarding far triangles from collision calculations.

Now about the header…
First integer = P = number of parts here. Then there are P integers = pointers to every part relative to beginning of data (from P, not from preceding "jump" value). Now for every part:

Third byte = S = show a number of 8-byte blocks that will follow below later. 2-byte values at 4-5 = N and 6-7 = F bytes (half-words of second word) = number of triangles in a frame; and triangle index in list from where this data must be written.

Then, if we’ll seek to "12+S*8+5" from beginning of this part, we somehow will get special byte that represents "number of frames - 1" = M. Usually it = 1 (two states, for portals).

Integer at 8-11 bytes (third word) = L= is a relative (to this part) pointer to actual data. So to get it, we should seek in this part to "L+N*12*K", where K must be 0<=K<=M. I plan to use only K=0 and K=M to get last and initial frame-states.

Offset of writing triangles in list (third part of collision data) is just "F*12".

At run-time pointers to parts became absolute:
http://klimaleksus.narod.ru/Files/H/live.png

Well, looks like I need to make a new version of CollisionCracker!

Here’s some screens with different states:
http://klimaleksus.narod.ru/Files/H/live_0.jpg
http://klimaleksus.narod.ru/Files/H/live_1.jpg
http://klimaleksus.narod.ru/Files/H/live_2.jpg
http://klimaleksus.narod.ru/Files/H/live_3.jpg
http://klimaleksus.narod.ru/Files/H/live_4.jpg
http://klimaleksus.narod.ru/Files/H/live_5.jpg
http://klimaleksus.narod.ru/Files/H/live_6.jpg

I think I got why in a list there’s such mess and not for example initial state instead. For grid! Since triangles are moving and optimization grid is not – every triangle should be included in every cell where it could possibly be. Simplest way is to make a mask – the very long triangle that just overlaps everything on its way – and then calculate grid with masks.

I suppose that grid creating was an automated process, so these masks must be kept to the very end, and there is no reason why developers would delete them: space must be preserved, and every state will be controlled by the logic at level loading. They could just zerofill masks, but for some reason they decided to keep them as unused data, because they were used long lime ago, at game-compilation time.

Sheep · Aug 29, 2015 8:49 AM

Yet again, good job! big_smile

Sorry for being inactive and not contributing in a while... I started school again two weeks ago and am generally too tired to do anything at the moment hmm

#26 Aug 20, 2015 4:18 AM

Re: Collision data hacking

#27 Aug 20, 2015 5:09 AM

Re: Collision data hacking

#28 Aug 20, 2015 9:25 PM

Re: Collision data hacking

#29 Aug 22, 2015 11:09 AM

Re: Collision data hacking

#30 Aug 23, 2015 6:13 AM

Re: Collision data hacking

#31 Aug 25, 2015 5:38 AM

Re: Collision data hacking

#32 Aug 29, 2015 8:49 AM

Re: Collision data hacking

Board footer