Print

Please login or register.
1 Hour 1 Day 1 Week 1 Month Forever
Login with username, password and session length

[Lucide]

After a day spent trying various font atlas generators unsuccessfully, I got upset and made one that provided the features I needed:

• Custom font support
• Custom charset
• User-defined font fallback stack
• No sprite packing
• Supersampling
• Offline support

Out of perhaps a dozen tools I've tried, some extremely old and some newer, none supported all three, so, if you're interested, you can try it out:
Font Atlas Generator

an extract from the readme:

Generate a texture atlas from a specified font's glyphs, that you can use on REXPaint, lvllvl.com, Playscii, ecc

• write the glyphs you want to use (charset), the default one is REXPaint's default
• enter the font name if you have it already available in your system (e.g. it's installed), otherwise, you can load a font file
• tweak the settings to achieve the desired layout
• save the image by clicking on save image

Note: due to the way text is rendered in the HTML Canvas, the output will not look good by default for very small font sizes. It's a matter of anti-aliasing algorithms, which are not designed to work with such tight pixel counts. You can try using supersampling+uncheck smooth scaling to get better results.
Keep in mind that the smaller you go, the less sense it makes to use a full-fledged font file.

[Lucide]

I take this opportunity to ask the devs one or two things about fonts in REXPaint:
I'm working with some friends on a demo for a text graphic library I've been working on, which will consists in a text-art animation runnable from the terminal. Notice text-art animation, not ASCII, not ANSI, not PETSCII. The library supports full unicode.

I'm in the process of choosing an editor or two to support natively (the library is just a graphic library, it provides an "objects with sprites" environment, like Scratch, I'd like to encourage the usage of a proper editor to draw the sprites).
With the latest update rexpaint came a lot closer to an "unicode editor", with the addition of extended fonts. This way we can use the atlas more like a palette, we pick a charset of unicode characters and we use them.

This tile-based approach is quite common, and probably versatile, (also required if dealing with PETscii art: I've found out from talking with Polyducks that the commodore's glyphs have just been added to unicode, almost no fonts support them yet), but it's incomplete, as the "text" concept is lost.
Playscii deals with this by using some metadata along the atlas to pair up every tile with the corresponding character. It would be nice if rexpaint supported this too, to have a proper tex export feature. (that now is just a gimmick from what i gather, as the characters are hard-coded from the default charset).

lvllvl.com has the same "issue" as now, it's something that becomes apparent as soon as different charsets are allowed.

If the glyphs used do not resemble any unicode character (or no metadata is provided), the export behaviour may be the same as now, backward compatible.
This, or the charset would have to be explicitly defined every time a .xp file is loaded in my library, because i use real text. Would be nice to have a native format.

[Kyzrati]

After a day spent trying various font atlas generators unsuccessfully, I got upset and made one that provided the features I needed:

This is awesome! Thanks for sharing

Later I'll definitely even want to put a link to this over on the Resources page. To be honest I, too, have never found such a generator that I liked (I've tried many over the years), but yours looks just great.

(By the way, sorry took me a bit to reply--I saw your posts and skimmed them, and even though I've been intermittently responding to other things in the meantime, this thread looked like it would require more thought and attention than I've had recently so I needed to wait for the right opportunity )

It would be nice if rexpaint supported this too, to have a proper tex export feature. (that now is just a gimmick from what i gather, as the characters are hard-coded from the default charset).

So what is the specific output here? You mean adding an external way to designate the Unicode associated with a particular glyph, and that is used for a new type of text export option? REXPaint currently does have text-based exporting, although it uses UTF-8 and the default CP437 codepoints.

What is needed here? Just making sure I'm getting this right since I'm not 100% sure I am

[Lucide]

Ooh no worries! It has never cross my mind to demand immediate answers on the web

You mean adding an external way to designate the Unicode associated with a particular glyph, and that is used for a new type of text export option?

Yes. The new txt export feature would work exactly like the current one for the user, but under the hood it would search for some metadata (to define where and how) to match the corresponding character to every tile. A file with a "charset string" would be sufficient, named like the atlas. The output UTF-8 encoded of course.
To enable text export from a text art editor

Some tilesets in the site look customized (i.e. with non-existent unicode characters). Those tilesets would simply come without the corresponding charset file, and REXPaint would behave like it does now if the user asks for a txt export. (assume modified ANSI charset)

Then I would modify my importers to ask for those files as well, instead of requiring the user to insert a charset manually when loading an .xp file. People who use REXPaint to make text art for the terminal (like me) would benefit from this.

[gumix]

You've spent just a day!
So, there are just 2 things i miss in it:
- option to clip glyphs in cells
- manual fine x,y global shifting

Other than that it works really great!
(I've tested it on the phone too!)

[Kyzrati]

Yes. The new txt export feature would work exactly like the current one for the user, but under the hood it would search for some metadata (to define where and how) to match the corresponding character to every tile. A file with a "charset string" would be sufficient, named like the atlas.

Okay, so for example right now REXPaint actually uses the following table to define the CP437 UTF-8 code points for exporting, and basically all we'd need to do here is make that file/data externally accessible and require that each font have an associated file defining how to convert its characters for export, eh? (which happens to be similar to the external-file-with-default-provided method of dealing with the planned customizable glyph mirroring behavior)

Code: [Select]

const wchar_t CP437_UTF8[256] = {
L'\u00A0', // 0 (nothing really, but using hard space)
L'\u263A', // 1
L'\u263B', // 2
L'\u2665', // 3
L'\u2666', // 4
L'\u2663', // 5
L'\u2660', // 6
L'\u2022', // 7
L'\u25DB', // 8
L'\u25CB', // 9
L'\u25D9', // 10
L'\u2642', // 11
L'\u2640', // 12
L'\u266A', // 13
L'\u266B', // 14
L'\u263C', // 15
L'\u25BA', // 16
L'\u25C4', // 17
L'\u2195', // 18
L'\u203C', // 19
L'\u00B6', // 20
L'\u00A7', // 21
L'\u25AC', // 22
L'\u21A8', // 23
L'\u2191', // 24
L'\u2193', // 25
L'\u2192', // 26
L'\u2190', // 27
L'\u221F', // 28
L'\u2194', // 29
L'\u25B2', // 30
L'\u25BC', // 31
L'\u00A0', // 32 (must use hard space instead of actual space, to prevent collapsing)
L'!', // 33
L'\"', // 34
L'#', // 35
L'$', // 36
L'%', // 37
L'&', // 38
L'\'', // 39
L'(', // 40
L')', // 41
L'*', // 42
L'+', // 43
L',', // 44
L'-', // 45
L'.', // 46
L'/', // 47
L'0', // 48
L'1', // 49
L'2', // 50
L'3', // 51
L'4', // 52
L'5', // 53
L'6', // 54
L'7', // 55
L'8', // 56
L'9', // 57
L':', // 58
L';', // 59
L'<', // 60
L'=', // 61
L'>', // 62
L'?', // 63
L'@', // 64
L'A', // 65
L'B', // 66
L'C', // 67
L'D', // 68
L'E', // 69
L'F', // 70
L'G', // 71
L'H', // 72
L'I', // 73
L'J', // 74
L'K', // 75
L'L', // 76
L'M', // 77
L'N', // 78
L'O', // 79
L'P', // 80
L'Q', // 81
L'R', // 82
L'S', // 83
L'T', // 84
L'U', // 85
L'V', // 86
L'W', // 87
L'X', // 88
L'Y', // 89
L'Z', // 90
L'[', // 91
L'\\', // 92
L']', // 93
L'^', // 94
L'_', // 95
L'`', // 96
L'a', // 97
L'b', // 98
L'c', // 99
L'd', // 100
L'e', // 101
L'f', // 102
L'g', // 103
L'h', // 104
L'i', // 105
L'j', // 106
L'k', // 107
L'l', // 108
L'm', // 109
L'n', // 110
L'o', // 111
L'p', // 112
L'q', // 113
L'r', // 114
L's', // 115
L't', // 116
L'u', // 117
L'v', // 118
L'w', // 119
L'x', // 120
L'y', // 121
L'z', // 122
L'{', // 123
L'|', // 124
L'}', // 125
L'~', // 126
L'\u2302', // 127
L'\u00C7', // 128
L'\u00FC', // 129
L'\u00E9', // 130
L'\u00E2', // 131
L'\u00E4', // 132
L'\u00E0', // 133
L'\u00E5', // 134
L'\u00E7', // 135
L'\u00EA', // 136
L'\u00EB', // 137
L'\u00E8', // 138
L'\u00EF', // 139
L'\u00EE', // 140
L'\u00EC', // 141
L'\u00C4', // 142
L'\u00C5', // 143
L'\u00C9', // 144
L'\u00E6', // 145
L'\u00C6', // 146
L'\u00F4', // 147
L'\u00F6', // 148
L'\u00F2', // 149
L'\u00FB', // 150
L'\u00F9', // 151
L'\u00FF', // 152
L'\u00D6', // 153
L'\u00DC', // 154
L'\u00A2', // 155
L'\u00A3', // 156
L'\u00A5', // 157
L'\u20A7', // 158
L'\u0192', // 159
L'\u00E1', // 160
L'\u00ED', // 161
L'\u00F3', // 162
L'\u00FA', // 163
L'\u00F1', // 164
L'\u00D1', // 165
L'\u00AA', // 166
L'\u00BA', // 167
L'\u00BF', // 168
L'\u2310', // 169
L'\u00AC', // 170
L'\u00BD', // 171
L'\u00BC', // 172
L'\u00A1', // 173
L'\u00AB', // 174
L'\u00BB', // 175
L'\u2591', // 176
L'\u2592', // 177
L'\u2593', // 178
L'\u2502', // 179
L'\u2524', // 180
L'\u2561', // 181
L'\u2562', // 182
L'\u2556', // 183
L'\u2555', // 184
L'\u2563', // 185
L'\u2551', // 186
L'\u2557', // 187
L'\u255D', // 188
L'\u255C', // 189
L'\u255B', // 190
L'\u2510', // 191
L'\u2514', // 192
L'\u2534', // 193
L'\u252C', // 194
L'\u251C', // 195
L'\u2500', // 196
L'\u253C', // 197
L'\u255E', // 198
L'\u255F', // 199
L'\u255A', // 200
L'\u2554', // 201
L'\u2569', // 202
L'\u2566', // 203
L'\u2560', // 204
L'\u2550', // 205
L'\u256C', // 206
L'\u2567', // 207
L'\u2568', // 208
L'\u2564', // 209
L'\u2565', // 210
L'\u2559', // 211
L'\u2558', // 212
L'\u2552', // 213
L'\u2553', // 214
L'\u256B', // 215
L'\u256A', // 216
L'\u2518', // 217
L'\u250C', // 218
L'\u2588', // 219
L'\u2584', // 220
L'\u258C', // 221
L'\u2590', // 222
L'\u2580', // 223
L'\u03B1', // 224
L'\u00DF', // 225
L'\u0393', // 226
L'\u03C0', // 227
L'\u03A3', // 228
L'\u03C3', // 229
L'\u00B5', // 230
L'\u03C4', // 231
L'\u03A6', // 232
L'\u0398', // 233
L'\u03A9', // 234
L'\u03B4', // 235
L'\u221E', // 236
L'\u03C6', // 237
L'\u03B5', // 238
L'\u2229', // 239
L'\u2261', // 240
L'\u00B1', // 241
L'\u2265', // 242
L'\u2264', // 243
L'\u2320', // 244
L'\u2321', // 245
L'\u00F7', // 246
L'\u2248', // 247
L'\u00B0', // 248
L'\u2219', // 249
L'\u00B7', // 250
L'\u221A', // 251
L'\u207F', // 252
L'\u00B2', // 253
L'\u25A0', // 254
L'\u25A1', // 255 (changed this to be empty version of 254)
};
Incidentally, while exploring this topic in the code for a bit I'm now pretty sure that the new unlimited character support is not yet compatible with the existing UTF-8 txt exporting and will give an out of bounds and potentially crash if it's attempted with an extended font xD (so I'll clearly have to address that as well...)

[Lucide]

Ok only mind that those are code points, the actual encoding to UTF-8 seems to be done elsewhere
I'd suggest against the use of wchar to store unicode characters, it was really designed for UCS-2 encoded text, when Unicode was smaller and 2B were sufficient.
Now that Unicode counts 143'859 chars, it's used to store UTF-16 text. That means that a character can take up to 2 wchar. If I remember correctly, wchar I/O functions are also considerably more complex (and slower), as they re-encode the text to match the current locale, but this in Linux I don't know about Windows.
In my library, for example, I store them in uint32 variables, already UTF-8 encoded, but that's an edge case because I move them around like pixels, so that approach might be a waste of memory in other contexts. In your situation, I think a fixed-size approach is still the best way to go, something like char[4], or even uint32, but you have to be careful with endianness.

which happens to be similar to the external-file-with-default-provided method of dealing with the planned customizable glyph mirroring behavior

Yeah, I've seen related requests here and there in the forum, if you think about it, you could implement some character-based logic to flip/mirror layers, and the mapping would make sure that the most appropriate tile is chosen. But going beyond that looks non trivial. That would require some syntax to define unique pairs, and everything should still be optional. Json may prove to be more versatile (maybe for the settings too, just sayin') :)

[Lucide]

You've spent just a day!
So, there are just 2 things i miss in it:
- option to clip glyphs in cells
- manual fine x,y global shifting

Other than that it works really great!
(I've tested it on the phone too!)

Ahahah no, it took me about a week, from 0 to up and running. I'm practical of C and Java, I had to learn most of that stuff from scratch, and it's been painful.
I don't like to fiddle with tools until they work, I like to be confident on the choices i make, and that generally requires an understanding of how things work.

That said, I've implemented those features yesterday, but then I ran into a serious font fallback bug I didn't notice till then: if the font chosen doesn't have the requested glyphs, the browser seamlessly chooses an alternative. That's not what the user expects. I'm working to have squares shown instead, I've asked for some help to do that correctly.

[Kyzrati]

Ok only mind that those are code points, the actual encoding to UTF-8 seems to be done elsewhere

Right, I was only showing the part that would be exposed since that's what we're talking about here, to see if this would be sufficient for the purposes at hand--designating a code point. Unless you mean you're looking for REXPaint to expose/do more than that?

I'd suggest against the use of wchar to store unicode characters, it was really designed for UCS-2 encoded text, when Unicode was smaller and 2B were sufficient.
Now that Unicode counts 143'859 chars, it's used to store UTF-16 text. That means that a character can take up to 2 wchar.

Yeah the underlying conversion process I've been using is actually based on uint32_t, just the CP437 reference array used wchar_t for the hell of it. At least I don't think switching the array over to uint32_t would lead to any issues...

If I remember correctly, wchar I/O functions are also considerably more complex (and slower), as they re-encode the text to match the current locale, but this in Linux I don't know about Windows.

Well this part hasn't really matteredd much since REXPaint images use a mere handful of characters

[Lucide]

Unless you mean you're looking for REXPaint to expose/do more than that?

No. was just to make sure
to me, utf-8 should be the only encoding available everywhere, when exporting text

[Lucide]

The site has returned to a working state, clipping has been implemented and unavailable glyphs produce blank cells.
I'm doing some research to optionally show nodef glyphs (they look like an uglier version of ☒), but that requires someone who's able to edit font files.
I'll ask in a typography forum, if they exist. But not sure if anyone would find that useful

Phone support is what it is. The page relies on the browser honoring unicode variation selectors, they tell the text engine to not spray emoji everywhere and use the font's glyphs instead.
Chrome for Android ignores them for example. The result is ..interesting but probably not what you're looking for ( ´ ▽ ` )

Fun Fact™! The title is a link that opens the github page

[Lucide]

Updates!
I was a bit dissatisfied with the output at lower resolutions (16x16 and lower). Given that REXPaint does no downscaling and all monitors I have are 1920x1080, generating an useful 16x16 output was a feature I needed myself.
New feature: supersampling! for crisper images at lower resolutions (higher resolution rendering+downscaling)

My first attempt was CSS-based, as it lets you choose the scaling algorithm. Unfortunately I later realized I had no way to get the processed image out of the browser.
To keep things simple I'm using a built-in Canvas flag which controls smoothing when scaling. Maybe the choice offered by css was a bit overkill anyway. And inconsistently supported.
It looks better on Chrome.

Improvements: smart page auto-resizing policies
I put a lot of blood and tears into this. I found very difficult to translate vague automatic resizing behaviour ideas to actual, implementable, rules. Now they are here, doing their work unnoticed and they're here to stay.

[Lucide]

Updates!
The app can now be used offline too, once opened for the first time.
When you're in offline mode (no network connectivity and http caches have expired) the animated header line becomes dashed.

There currently are some uncaught exceptions sprinkled in the console. I'm trying to resolve this with workbox's authors. They shouldn't affect functionality.
The issue has been fixed

[Kyzrati]

Hey Lucide, as per your earlier request I'm back with a new preview patch (for 1.50) to let you try it out before it goes into the next official release. Download the patch here(edit: patch removed since this is now included in 1.60) and it includes only the new/modified files.

With this version you can assign code points to glyphs on a per-font basis by specifying a file where those values can be found. Notice that the font _config.xt also includes two new columns to make new features possible. (This same patch also includes support for customized glyph mirroring, as well as a few other QoL features.)

The updated manual should have the necessary details:

...

Notice in _config.xt that each art font also includes two additional parameters: Unicode and Mirror. These specify two text files, also expected to be found in the "data/fonts/" directory, which define code points for UTF8 text exporting and glyph mirroring behavior for the paste tool, respectively. The default files provided are compatible with a standard CP437 layout. If neither of those features is required for a given font, using the default files is fine, or even just insert a single hypen ('-') for a filename there and it will be ignored. See the sections below for more information on customizing these features.

...

 Custom Unicode Codepoints
---------------------------
By default, images exported as text use UTF8 code points based on the CP437 layout, but some users might want to either override those values or specify UTF8 code points for glyphs outside the CP437 range when using extended custom fonts. You can do this by creating a new text file in the "data/fonts/" directory that defines Unicode code points and assign it to its respective font in _confit.xt via the Unicode column. See the default CP437 _utf8.txt file for an example of the required layout. Each line should specify a single glyph index followed by its decimal code point. Indices start from 0 and proceed in row-major order, as displayed in the font bitmap, and any index not specified in the file will produce a '?' where an imagine using that glyph is exported. Add optional comments at the end of a line by preceding them with two slashes.

Let me know how it works out.

[Lucide]

It's cool, comment support (for displaying glyphs), shared files, shorthand "-", it's all there.
If I may, I would suggest calling the charset files with their charset name "_cp437.txt" instead of "_utf8.txt", it's clearer to show what they are for. (and what they contain).
The "Unicode" parameter might be better called "Charset" too.

The default fonts are named after their charset, fair enough. If I were to create some custom cp437 font, I'd call it "Bob" and assign the "_cp437.txt"/"-" charset to it.
If I were to create some custom kaomoji font, I'd call it "Adaji" and assign the "_kaomoji.txt" charset to it. An eventual "Alice" font could use "_kaomoji.txt" too.

[Kyzrati]

Cool. And yeah I had decided to be less specific with the naming since it applies across all the fonts by default and makes it a little more apparent what it is at first glance (in both place).

This kinda helps people who just glance at the files themselves who then go "huh, Unicode values? maybe I can do something with this...", sort of as a way to discover the feature in the first place. On the contrary for those who know about these sorts of things and want to use them more widely it would indeed make more sense to call it cp437 instead, though I prefer to do what's better for discovery in this case, since everything can be edited to your liking anyway.

Did you try to use it yet?

[Lucide]

Not sure of what I'm doing wrong, but I can't get it to work. Might be the late hour.
Here's my setup

[Lucide]

Updates!
you can now define an arbitrary stack of fallback fonts
In the real world, text rendering is often a hodgepodge of different fonts. Even on the Ubuntu terminal, the fallback font when DejaVu doesn't deliver is Symbola.
But why stop to one fallback font when you can have many

[Kyzrati]

Not sure of what I'm doing wrong, but I can't get it to work. Might be the late hour.
Here's my setup

Nah, not just you, there's things to fix

Odd, some of what appears to be wrong, if only because the CP437 code points worked just fine... Thanks for giving me this other test data to work with, though, I'll check it out and get back to you once I know more.

One thing I can tell you at first glance is the reason for a bunch of 'A's in the background--that would be because the 'A' character is found at index 32, which is always interpreted by the engine to be a space. (This info is covered in the font customization part of the manual.)

Since REXPaint does not store any additional information about a cell aside from glyph, foreground color, and background color (not even in the engine), there is no other dedicated way to indicate "there is no character assigned here", so that's the job of the space glyph, which the engine hard codes to be at index 32, where it resides for the CP437 layout. Notice that the 'A' at index 32 is actually blank as it appears in the font selection area with your test font.

[Lucide]

Yes I'm aware of that, I thought I had filled the empty cells with an actual "space" from the font in the layer below. If I don't leave any empty cell but I keep a custom space-filled layer at the bottom shouldn't I be fine?

[Kyzrati]

Layers are purely for editing convenience, though, so that's unrelated here. In the end all layers are combined into a single layer for export, top-down, and in your test scenario there, the first/bottom layer is filled with what your code point defines to be 'A' characters (at index 32), hence all the A's in the output where no upper layer overwrites them.

_visualt.txt contains the line "32 65 //A", so when REXPaint mashes the layers together and finds anything matching index 32, it outputs that as an 'A'. For testing/exploring your sample there without the extra noise, I switched it to "32 160 //space".

[Kyzrati]

Okay, new patch! I've uploaded the archive that the earlier post's link leads to. This one should do the trick--looks good to me, anyway. (Note the newer version of the patch also requires that you overwrite your skins file, since a new variable/color was added.)

Also while testing your image I noticed most of the results had a one-off error, and found the source to be that whatever language you used to generate your font's code points likely treated '\' as an escape character and skipped that one entirely, causing everything after it to be incorrect. So you'll need to update that to get the proper results.


I've also made several updates and improvements based on other things I noticed while working on this, features that should've been in v1.50 but I had overlooked

Like now REXPaint actually knows how to treat images that were created using glyphs with IDs beyond any that are available from current fonts, seen here identified with a new marker as a reminder:

[Lucide]

Hooray! I apologize for the file error, must've been really frustrating. I wouldn't have noticed in a year.
I also had misinterpreted the space behaviour, and on top of that I thought I was using my space (my font was actually missing the space char altogether!). Didn't notice REXPaint was showing space instead of A. Which makes perfectly sense of course.

If you tell me this will be roughly the final layout, I'll add the first export option to the generator. with a non-space '32' warning

[Kyzrati]

Yep as long as everything's working in this patch it shouldn't need any more changes and will go into the next 1.60 release. Basically just waiting to for gumix (or someone else who needs it) to test out the new glyph mirroring support to make sure it meets needs as well and it'll be all set to go.

Hooray! I apologize for the file error, must've been really frustrating. I wouldn't have noticed in a year.

No problem, it did throw me for a bit, but fortunately I also met the exact same issue regarding CDDA format exporting to JSON (also added recently), so even had a good reasoning to understand it once I'd tracked it down

[Lucide]

Updates!
image and charset export buttons have been added
Additional export options will be added if requested or needed