FEATURE: Make more tools read stdin for input

[DrMcCoy]

(Split from #58)

Which tools exactly do you need to be able to read from stdin?

Many (Most? All? of the tools that can read textual data should already be able to read from stdin.

Many (Most? All?) of the tools that read binary data currently don't, because they need SeekableReadStreams, and stdin obviously isn't seekable. However, often this is only used for SeekableReadStream::skip() and only in a forward direction.

So I could probably fix that by:

1. Introduce a companion method to skip() that seeks backwards. Though I'm not sure what to call it. rewind()? Or rather rewind() is seek(0, kOriginBegin), while rewind(size_t n) is seek(-n, kOriginCurrent)?
2. Change all uses of skip() with a negative parameter to use that new method
3. Change SeekableReadStream::skip() to take an unsigned value and mandate it to only skip forwards
4. Add a virtual method skip in ReadStream that implements it using read(). SeekableReadStream still overrides it with the seek()-using-implementation

That would give us a ReadStream that can do skip() and we might be able to downgrade some uses of SeekableReadStream to ReadStream instead, which means we can use StdInStream there.

I would need to evaluate that on a case-by-case basis, though.

Thoughts?

[DrMcCoy]

@clone2727 had a better idea: introduce a CachedSeekableReadStream (or whatever it should be called) that basically reads into a dynamic memory area, so that it can provide true seeking.

Of course, that's the more complex solution that would be a bit more work than the quick fix I proposed above. Not sure I can give it adequate focus at the moment.

Btw, what exactly is your use-case? What temporary files are you generating and reading and why? Can you guide me through an full example run of what you're doing, so that I can understand what you need?

[DrMcCoy]

Oh, right, explicitly tagging @lachjames to make sure you actually see this :P

[lachjames]

Thanks for creating this issue and tagging me in it 👍

Probably the main use case would be reading/editing/writing files (GFF obviously being the main target, but the container structures like RIM and MOD would be useful too (although probably beyond the scope of this issue as it would require more than redirecting stdin).

The text-reading ones (I guess the ones which convert from xml to a given format) are probably fine as you describe them (though admittedly I never thought to try using stdin for them as I assumed it wouldn't work). The ability to take a binary stream and pass it straight through stdin in the same fashion would be fantastic too.

A full example of the workflow I'm imagining would be as follows (for a KOTOR dialog editor):

• Read the .dlg file using gff2xml
• Modify the xml using the UI of the tool
• Save a .dlg file using xml2gff (or perhaps modify a RIM/MOD/ERF file to include it, but again that's beyond the scope of this issue - I might open another issue in relation to this).

Currently, the above workflow involves writing a gff file to disk (if reading it from a RIM/MOD/ERF) to read it with gff2xml, then writing the xml file to disk for xml2gff (though as you say, this is already not necessary).

I think any solution which allows a binary stream to be used as input would be fine. To be honest, the implementation details you discuss are a bit beyond my experience with reading/writing streams so I'll just say that I'm sure you'll make a fine judgement on the method which would be preferable :)

[DrMcCoy]

Okay, if you want to read GFFs from stdin, that already makes the easy approach moot, because you need proper seeking in GFFs.

That means the cached approach is the way to go. I can't promise when I'll get to that, though.

[lachjames]

There's no rush at all on this; it's just a suggestion that would make the tool much more efficient when integrating it in our own code via subprocesses. As always, let me know if there's anything I can do to help out :)