Grid Sage Forums

Grid Sage Forums

  • November 22, 2024, 12:57:22 AM
  • Welcome, Guest
Please login or register.

Login with username, password and session length
Advanced search  

News:

LINKS: Website | Steam | Wiki

Author Topic: mini-REXSpeeder for C++ v2  (Read 6082 times)

Mreuwu

  • Unaware
  • *
  • Posts: 9
    • View Profile
mini-REXSpeeder for C++ v2
« on: June 04, 2016, 11:44:04 AM »

Features:
0. doesn't use zlib, so to use .xp files, you have to use 7zip to extract their contents first (i.e. right click, extract here). If there's no compression anywhere, this will result in a slowdown if your .xp files are large.
1. only reads files. no saving. no transparency tests. no flattening. no creating new RexFiles. whereas REXSpeeder can do those things.
2. header-only: include the 3KB file and that's it, no need for configuration changes or libraries
3. no exceptions. instead, dumps errors in the std::cerr stream and then abort()s, so in case you're wondering where a bug is in your program, then wonder some more because this library sure won't tell you any useful information
4. if the .xp file is maliciously crafted, and you try to use the vector<> layers directly, then you are vulnerable to a buffer overflow. using getTile() is safe. I'm too lazy to fix this at the moment.

License: I place my changes in the public domain. This builds upon the existing REXSpeeder library, which is MIT-licensed. But the two libraries are only superficially similar now.
« Last Edit: July 02, 2016, 09:01:24 PM by Mreuwu »
Logged

Kyzrati

  • Administrator
  • True Cogmind
  • *****
  • Posts: 4477
    • View Profile
    • Cogmind
Re: mini-REXSpeeder for C++
« Reply #1 on: June 04, 2016, 08:09:33 PM »

Ouch, can't use zlib? Everybody uses zlib ;)

Funny that you can use 7zip to extract an .xp file; never thought of that :P

Thanks and I've added this to the resources page!
Logged
Josh Ge, Developer - Dev Blog | @GridSageGames | Patreon

Mreuwu

  • Unaware
  • *
  • Posts: 9
    • View Profile
Re: mini-REXSpeeder for C++
« Reply #2 on: June 04, 2016, 08:46:25 PM »

7zip is robust WRT extensions; you can take a .zip file and rename it .txt and 7zip will extract it perfectly anyway. You can extract things like .cab and .iso.
Logged

Pyridine

  • Unaware
  • *
  • Posts: 5
    • View Profile
Re: mini-REXSpeeder for C++ v2
« Reply #3 on: July 02, 2016, 03:13:27 PM »

In my experience, not every library loads files instantly. Sure, for like 20x20 files it doesn't matter how you load it, but I've seen big differences for files as small as 60x60. And for, say, game development, every millisecond counts.

Anyway, I'm impressed by the simplicity of your library. You certainly seem to know your stuff.
Logged

Mreuwu

  • Unaware
  • *
  • Posts: 9
    • View Profile
Re: mini-REXSpeeder for C++ v2
« Reply #4 on: July 02, 2016, 03:56:58 PM »

Since Kyzrati stated he won't go UTF-32, and instead is thinking about sprite sheets, then you should use my 4 x uint8_t instead of the uint32_t, and then merge your read operations. There's no way the extra 3 bytes can be used without breaking compatibility, since sprite-sheet files are assumed to be a fixed 16x16 grid, since there is no place to specify width and height of the sprite sheet. Even if the rationale was future compatibility with arbitrary-sized sprite sheets, it is still a mistake since it would require a breaking change in the .rex specification. The width should not have been 4-bytes. Too late now. Although it still doesn't matter much because the 0s are compressed away.

Your REXSpeeder library also has a buffer overflow if the width and heights are maliciously set for each layer. I guess mine does too since I let the user rummage around the vector directly, which is why my getTile used .at() to avoid these problems. But I forgot about the public vector. My bad.
« Last Edit: July 02, 2016, 03:59:34 PM by Mreuwu »
Logged

Kyzrati

  • Administrator
  • True Cogmind
  • *****
  • Posts: 4477
    • View Profile
    • Cogmind
Re: mini-REXSpeeder for C++ v2
« Reply #5 on: July 02, 2016, 04:16:55 PM »

Even if the rationale was future compatibility with arbitrary-sized sprite sheets,
This is an eventual goal. The current restriction to 16x16 spritesheets is again only a UI limitation, and there is nothing internal to REXPaint that limits it to 256 characters. In Cogmind for example, the same underlying system uses hundreds more values than CP437 to mix multiple spritesheets together, they just can't be edited in RP itself.

So later on (with a more advanced UI), users would be allowed to specify as large a spritesheet as they want--it's actually the most requested feature so far since REXPaint's introduction :P. This wouldn't have any impact on the specification, though, because images don't carry any spritesheet data, merely an integer index independent of whatever font is chosen.
Logged
Josh Ge, Developer - Dev Blog | @GridSageGames | Patreon

Mreuwu

  • Unaware
  • *
  • Posts: 9
    • View Profile
Re: mini-REXSpeeder for C++ v2
« Reply #6 on: July 02, 2016, 04:18:53 PM »

Ok, and how do you go from an integer index to a location in an arbitrary-sized sprite sheet? If the index is 300, it means very different locations for a 14x30 sprite sheet or a 100x200 sprite sheet...
« Last Edit: July 02, 2016, 04:20:41 PM by Mreuwu »
Logged

Pyridine

  • Unaware
  • *
  • Posts: 5
    • View Profile
Re: mini-REXSpeeder for C++ v2
« Reply #7 on: July 02, 2016, 04:34:25 PM »

> Since Kyzrati stated he won't go UTF-32, and instead is thinking about sprite sheets, then you should use my 4 x uint8_t instead of the uint32_t, and then merge your read operations.

Yes, I think I'll try and see if I can do that. That would cut down immensely on the number of reads. As for the possible future capability of XP to index arbitrarily-sized spritesheets, I think it's best to build the library to support what XP can actually do now, as opposed to what it may do later. Particularly if that allows code simplification.

> Your REXSpeeder library also has a buffer overflow if the width and heights are maliciously set for each layer.

I'm not worried about security for REXSpeeder until the need arises. Feel free to convince me otherwise. But if this particular vulnerability can be fixed with just an at() then I might as well fix it. thanks.
Logged

Kyzrati

  • Administrator
  • True Cogmind
  • *****
  • Posts: 4477
    • View Profile
    • Cogmind
Re: mini-REXSpeeder for C++ v2
« Reply #8 on: July 02, 2016, 06:05:28 PM »

Ok, and how do you go from an integer index to a location in an arbitrary-sized sprite sheet? If the index is 300, it means very different locations for a 14x30 sprite sheet or a 100x200 sprite sheet...
Good question. My intent would be to keep it as it is now, simply handled in row-major fashion. Using the current layout as an example: 0-15 on first row, 16-31 on second, and so on.

Also, my guess is the future UI would likely always have a width of 16, but with an arbitrary height accessible via vertical scrolling.

Really though, these are UI/editing considerations without any real impact on what is stored/read from an image, instead only determining how you might want to lay out a spritesheet itself, and interpret the indexes in the end-use program.
Logged
Josh Ge, Developer - Dev Blog | @GridSageGames | Patreon

Mreuwu

  • Unaware
  • *
  • Posts: 9
    • View Profile
Re: mini-REXSpeeder for C++ v2
« Reply #9 on: July 02, 2016, 06:25:23 PM »

Ok, I understand what you mean. The sprite sheet dimensions are a sprite sheet property, not a .xp file property. So it does make sense to have 4-byte indices.
Logged

Kyzrati

  • Administrator
  • True Cogmind
  • *****
  • Posts: 4477
    • View Profile
    • Cogmind
Re: mini-REXSpeeder for C++ v2
« Reply #10 on: July 02, 2016, 09:09:37 PM »

Right, I wanted to keep .xp files to the bare minimum of data so they're really flexible. It's basically like building a roguelike where the font you use is independent of the game data itself. Of course, as one user has requested it would be nice to have a variable amount of space in the header for optional meta data, but I don't really want to mess with the structure, so it continues to remain as it was years ago...
Logged
Josh Ge, Developer - Dev Blog | @GridSageGames | Patreon

gumix

  • Cyborg
  • ***
  • Posts: 134
    • View Profile
Re: mini-REXSpeeder for C++ v2
« Reply #11 on: July 03, 2016, 05:13:58 AM »

 :P i can sacrifice separate layer for metadata though.
Logged

Kyzrati

  • Administrator
  • True Cogmind
  • *****
  • Posts: 4477
    • View Profile
    • Cogmind
Re: mini-REXSpeeder for C++ v2
« Reply #12 on: July 03, 2016, 08:58:33 AM »

That's what I do :). And it has the advantage of providing positional data as well. So now we have 8 extra layers for that!
Logged
Josh Ge, Developer - Dev Blog | @GridSageGames | Patreon