Page MenuHomeFeedback Tracker

str command does not escape nested double-quotes for strings
New, WishlistPublic

Description

private["_text"];
_text = "--->""<---";

str(_text)

produces

"--->"<---"

which is technically a broken string.

and makes it very hard to parse the data externally.

Just this simple change would go a long way for helping improve performance of many extensions that want to read structured data from the game, without doing hacks like calling toArray on strings.

Maybe introduce a new version of the str command, that produces a parser friendly syntax of the data.

Details

Legacy ID
3335717426
Severity
None
Resolution
Open
Reproducibility
Always
Category
Scripting
Steps To Reproduce

======================================================
Example # 1 (Passing data into an extension):
//======================================================

private["_data", "_key", "_value"];
_key = "foo";
_value = "--->""<---";
_data = [_key,_value];

"xyz" callExtension str(_data);

/*

  • The xyz extension would receive a buffer with this:
  • ["foo","--->"<---"]
  • Which is not very usefuly, it's technically broken syntax.
  • */

======================================================
Example # 2 (Compiling code):
//======================================================

compile str(_data); //does not work, will error out

Event Timeline

micovery edited Steps To Reproduce. (Show Details)Oct 25 2014, 8:09 PM
micovery edited Additional Information. (Show Details)
micovery set Category to Scripting.
micovery set Reproducibility to Always.
micovery set Severity to None.
micovery set Resolution to Open.
micovery set Legacy ID to 3335717426.May 7 2016, 7:41 PM

Looks exactly like it should, what were you expecting the result would be?

When the argument to the str command is a variable (or literal) of "STRING" data type ... the str command wraps the result with double quotes.

That's not enough. Simply slapping double quotes around some text does not make it a syntactically valid string.

Any other programming language that has a "stringify" function would escape the nested string delimiters, when stringifying variables of "STRING" data type.

So in this case I would have expected str to produce:

"--->""<---"

Added a couple of examples in the repro steps

EDIT: I see, compile does fail.

It seems all str command does is it places "" around the argument, that's all.
Because str deals with double quotes only you can use single quotes

_value = '--->""<---';

to work around the problem, or use format and form the string you send to extension exactly the way you want.

I doubt this is going to change as this is ancient command and it may break too many scripts. What needed is new command, parseString that would correctly parse all escaped " within.

format has a maximum size limitation, so it won't work correctly for this purpose.

A command for escaping strings would be nice, but not sufficient.

str does more than just serialize strings. It traverses the entire data structure you pass in, and serializes it.

All I am asking is for it to play nice with strings :-)

str has other problems, for example, a null object will be serialized as:

<NULL-Object>

obviously that's not valid SQF, why couldn't the devs just serialize it as objNull ...?

In anycase, <NULL-Object> is easily parsable, you can simply use a regex to match it, and replace it with the correct valid SQF objNull.

I know that breaking backwards compatibility is probably out of the question, even if the old behavior was bad.

At the moment, my work-around for being able to serialize stuff correctly, before passing it into an extension is to recursively traverse the structure ... and convert strings using toArray. However, both the traversal, and the toArray command add a lot of overhead.

Finally, consider this ... a lot of the time the contents of the variables being passed around is beyond the control of the developers (player names is one such example).

Torndeco added a subscriber: Torndeco.EditedMay 20 2016, 4:10 PM

This is still an outstanding issue :(

Extension wise we just need a str command that escapes quotations.
This is an issue when passing user inputted data to an extension that contains quotes.
Parsing the data beforehand in SQF = slow & ulgy.

As for the other serialize issues we can work around them.

Either a new command i.e strX
Or new syntax i.e str (value, bool) bool = to escape quotations.
That way, it will not break any existing scripts.

Grim added a subscriber: Grim.May 20 2016, 5:49 PM

New implementation of "str" command would really help due to "format" command has a size limitation witch is an issue when dealing with extensions.

Issue is sorted in latest arma dev builds :)
Thanks

This has broken basic functionality of the compile command. String arrays "["text","text,"text"]" now compile as [""text"",""text"",""text""] and throw errors because it is not written as "",text,"","",text, etc...

You always had to escape quotes for compile command.

i.e
compile "["text","","text"]" ; would have failed before aswell and now
compile "[""text","""","text"]"; would have worked fine

Nothing currently works in my Altis Life server when reading from the database anymore...
It shouldn't have been changed. If it ain't broke, don't ""fix"" it.

It was broken ;)
That is why my extension had to use a seperator character etc

Anyway its already fixed in lifeservers.
https://github.com/AsYetUntitled/Framework/commit/db66e545ef02faec2a2e9fd152eb5260b3deab8a

Torndeco is correct, the str command was always producing illegal strings under certain circumstances, but because it was an old bug it doesnt make it ok.

That one element might be fixed, but now if you compile "2" it becomes {""2""} instead of {2}... why?

Fixed str functionality only escapes " which are inside "". Nothing else has changed.

str 2 will still make "2" and
str "2" will still make ""2"" as before

Is that the same for how diag_log shows that information?
Because all of my database queries now have ""2"" etc around numbers from selected lists where as before in diag_log they displayed as 2...
For those fields, I don't have any of them using str() they're loaded and used as integers once compiled from the string format in the query.

BIS_fnc_KK added a comment.EditedSep 26 2016, 9:43 AM

I dont know specifics of your database setup but going from 2 to ""2"" is unlikely caused by single use of str. 2 to "2" or "2" to ""2"" maybe, but str doesnt double quotes like that not before not now.

Unless str is inside the compile command, then it isn't used.

The Server pulls either "2" or 2 from database ( can't tell which because diag_log is already a string). It's compiled, and used as an integer... Diag_log used to show "5,100,2", now displays as " ""5"", ""100"", ""2"" "

The only command used against these is the query to retrieve the data, the compile to convert from a "string" to code for a usable integer, and then that is stored and used in a variable.

So... Something else has been changed as well. The outputs are now completely different.

BIS_fnc_KK added a comment.EditedSep 26 2016, 10:08 AM

return of the callextension is string already if it contains "2" then it will be compiled into a string. You should know what the database output for sure, you can do that by running database query on the database server and not rely on diag_log

BIS_fnc_KK added a comment.EditedSep 26 2016, 10:10 AM

Anyway why dont you look at the link provided by Torndeco, the issue there was caused by double use of str command, you might have the same issue.