As per this article, when statically compiling a debug CRT it will allocate memory with specific values to assist in memory mapping: http://stackoverflow.com/questions/370195/when-and-why-will-an-os-initialise-memory-to-0xcd-0xdd-etc-on-malloc-free-new
The check for calling an RVExtension, which checks for a "buffer overrun", simply checks that output[outputSize-1] == 0x00. This is not the case in debug extensions, as they may be initialized as 0xFE or 0xCD
Build an RVExtension in debug mode, and attempt to call it. It will error with the buffer overflow protection, although it was not in fact an overflow and is simply debug memory.
This behavior should be disabled, and if necessary, a protection byte after the extension should be used instead. The check can be modified to supply the program with less memory than in the buffer, so then the caller can check the guard byte (memory after what was provided to the user) so it would not be modified by the DLL and as such, not modify the memory.
This would look like:
char buf[bufSize+1];
buf[0] = 0; buf[bufSize+1] = 0; rvExtension(buf,bufSize,extFunction); if (buf[bufSize+1]!=0) // Buffer overflow detected
If that's true - how on earth do extensions work now? If in debug mode the check hits an unallocated space, then in release it should have hit uninitialized area => only 1/256 chance of having zero there.
Something does not compute.
You did not read my ticket correctly or do not understand it.
when compiling with the Debug CRT, it *does not zero the memory*, it sets it to different debug values depending on the type/method of allocation (see the link I provided (http://stackoverflow.com/questions/370195/when-and-why-will-an-os-initialise-memory-to-0xcd-0xdd-etc-on-malloc-free-new). Additionally, it will fill buffers for different string and memory copy/allocation functions for debug purposes.
This means you *cannot run extensions compiled against the Debug CRT*, only built against the release CRT. It's not world ending, but its annoying when working on complex extensions. That means, if you do something to the affect of:
ACE_VD_EXPORT void __stdcall RVExtension(char *output, int outputSize, const char *function) {
sprintf_s(output, outputSize, "My Balls");
}
The debug CRT writes:
output = My Balls 0x00 0xFE 0xFE 0xFE 0xFE
Thus triggering this warning. Which has nothing to do with a buffer overflow, its just that the 2nd to last byte they provided me is not null.
This can be worked around in the extension by setting the 2nd-to-last output byte to 0x00 no matter what.
Ideally though, this should be appropriately handled with a correct guard byte after the buffer - not a guard byte inside the buffer sent to the function.
This does not occur in release builds because ArmA3 zero'es the buffer prior to passing it to you; its specifically your extensions debug flag which is re-writing the buffer.
Yes, you're right, I didn't understand your ticket. You have attributed this issue to Debug CRT itself, which is actually wrong. That got me confused.
Debug CRT only sets the memory to the provided values *if the buffer is allocated within it*. When the buffer is passed into extension by ArmA, that is obviously not the case - the buffer has been already generated by Release CRT that is used by ArmA and stays there. The runtime which your extension was linked against has no effect on ArmA's runtime - meaning ArmA will always give you a zero-filled buffer, regardless of whether you start with debug
Judging from your description (unless there are some more details missing), what you're actually looking at is an issue with sprintf_s function. It is not a C standard function, but rather a Microsoft extension; and Microsoft did their best there in not following the "least surprise principle". Debug version of sprintf_s just fills the rest of the buffer with bogus data - that is why your buffer ends up filled up with 0xWHATEVER instead of 0x00 that were there previously.
Why am I saying it is not Debug CRT fault, if it's debug version of function?
Because with a slightly different code you can get it working regardless of runtime:
memset(output,0,outputSize)
std::string balls("My balls");
memcpy(output,balls.c_str(),balls.size()); // Copying without the 0-byte, as the buffer is zero alreadyNeither your code nor my example above use any explicit allocations; hence, Debug CRT's stack overflow protection is irrelevant.
To workaround this issue, you can try to:
[1] see more here: https://social.msdn.microsoft.com/Forums/vstudio/en-US/16ce9c2a-341e-4601-82cd-bb437ab8a6bf/sprintfs-slower-in-vs2010-than-in-vs2008
I agree with your suggestion regarding ArmA's behavior. What I'd like to note is just that your issue is not with Debug CRT per se, but with a specific gotcha in one Microsoft's function.
P.S. output[outputSize-1] is not "2nd-to-last" byte, it is the last.
Arma is already setting the last byte in the outputBuffer to 0 before calling the Extension like:
output[outputSize - 1] = 0;
RVExtension(output, outputSize, function);
if (output[outputSize - 1])
{
error("Buffer overrun detected");
}The buffer is already 0 initialized and when calling a Debug Extension it wont be reinitialized to 0xCD or something like that.
By the way it looks like buffer overrun just wont be detected if last byte is 0.. Nothing keeps the extension from overfilling the buffer when the last byte in the buffer is set to 0