I made it my goal to harden SBWG before I start to implement new features. Before I call the next version of this project 1.0 I want to make sure that unexpected
input from the command line or from source files, absurd numbers of absurdly long tags and content items, stupidly weird filenames or random binary data as tag
values as well as purposfully created traps in the various places where input is processed are handled well, meaning that nothing fails unless there is no sensible
way around it, and if something fails, that nothing breaks. Data should be filtered carefully, errors should be handled well and whereever possible data should be
made processible if it was supplied in an unprocessible form to reduce the chances of errors. On top of that I wanted to make sure that the script did its job in a
reasonable amount of time considering the circumstances. I mean, it will never be very fast. Bash is just not the right language for that. But there certainly were
some repetitive tasks that could be improved. Fot the latter I created a simple caching functionality that will probably be extended in the future. I managed to
reduce the (calculated/estimated) generation time of my biggest test web site from almost 300 years to a few days. Actual web sites will of course not take that long
to generate, even on a slow machine. A huge web site will maybe take up to one day to generate completely, even without the new options that keep the script from
re-generating existing unchanged parts of the web site. But before sombody will try to create such a big web site with SBWG I will probably have improved speed
further. And even then it's a worst-case time.
As part of the aforementioned goals I have started working on last new feature before version 1.0. I call it the pathreducer. Since many of the files created by SBWG are
named after the tags they represent, they can become quite long and contain almost any printible character, including multi-byte unicode characters or characters of
character sets I haven't even heard of. I definitely don't want to restrict more than I already have what characters and how many of them tag values can contain.
Especially filesystems used by operating systems from Microsoft are relatively restrictive in maximum allowed directory, path and filename length and allowed
characters. By default the pathreducer is not used. But if enabled via command line option or in a web site's settings file, it will filter directory and filenames
and shorten them to a user-defined maximum length. If the pathreducer decides to change a path elements it also adds a 6-character hash value to make shortened or
otherwise reduced path elements as good as unique.
That works well for now and even can create 8.3 or 6.3 filenames for old DOS filesystems. But the result is not very nice because it isn't aware of what
filesystem it is going to write a file to. To be save it removes more characters than it would have to for ext and NTFS filesystems. In the future I may extend the
pathreducer to detect the filesystem at least of the root of the output directory automatically and decide how exactly path elements should be reduced according to
the actual limitations of the present filesystem. Than it may even be enabled by default, even though it can increase the generation time quite a bit.
There are still some tests that I want to do and I will probably find some more things that I want to fix before version 1.0. But I see light.