Twine 2, Sugarcube 2 variable library performance limitations

Question

Hi everyone,

I'm currently making an RPG with a huge amount of randomized custom equipment (objects with a minimum of 5 and a maximum of 20 properties per object), and at least 50 "slots" of storage space to store all those objects for the player to retrieve them whenever they please. AND on top of that I'm implementing procedurally generated dungeons (Array maps of 100 x 100 objects... with each object having its own properties e.g. {Type: "Floor", Terrain: "Grass", Special: "Treasure Chest"}) ... some of which I would like to store in Sugarcube's variable library for retrieval later if at all possible (i.e. i don't want to keep randomizing a dungeon each time a player visits it... only when the player dies will a dungeon reset).

I also remember encountering slowdowns in my old Twine game that I made when i did not disable the state history's states to 1, which I assume came from the variable library being too large due to the variable library copying itself everytime I moved to a new passage.

So it's got me thinking, can I achieve my complex RPG and have a massive library of complicated objects and big map arrays in which each element is an object in itself within the constraints of the Twine 2 / Sugarcube 2 engine?  What are the limitations?  (P.S. I guess what I'm looking for is a rough estimation of an actual number of just how many Variables can be stored within Sugarcube's variable library before I start to encounter unacceptable slowdowns as I move from passage to passage.)

Answer 1

Well, there isn't any specific number of variables, because it depends on both the browser and the hardware it's running on.

The best thing to do is to try to minimize your use of story variables (the "$name" variables) by offloading what you can into the Twine and JavaScript code, and then using the "setup" object to store the more static data where you can with the remaining data (like the layout of your map, which I assume won't change).

For an example of offloading the data into Twine or JavaScript, you could have it so that if an object has a (unique) name and a description, use the name to call a function or widget which will give you the description based on the name you pass to it, instead of storing the description on the object.  That way you won't be storing that data in the history.  Also, you could give items "default" values in various properties, and only store data about properties of the item when it doesn't match the default value, otherwise you pull the default value from your code.

For an example of using the "setup" object, you could randomly create your map and objects at the beginning, and then store the constant features of that data in variables attached to the "setup" object.  If the objects stay the same throughout the game (after they're initially randomly generated), and only their locations change, you could store the descriptions of the objects on the "setup" object, and store the locations of the objects in story variables.

If you do this you'll need to write some functions for Config.saves.onLoad and Config.saves.onSave so that the data would be stored on and retrieved from the metadata of the save object.  So, just to give some basic sample code for how to use the save object, you could put something like this in you JavaScript section:

Config.saves.onSave = function (save) {
	save.setup.VariableName1 = setup.VariableName1;
	save.setup.VariableName2 = setup.VariableName2;
};

Config.saves.onLoad = function (save) {
	setup.VariableName1 = save.setup.VariableName1;
	setup.VariableName2 = save.setup.VariableName2;
};

You would just put your own "setup" variables in place of the above examples.  That would allow you to store and retrieve the data for your game that's on the "setup" object, so that it doesn't fill up your history.

Basically, just figure out how to store the minimum possible in story variables, and that should help minimize any slowdown.

Hope that helps!

(Note: The above has been edited to apply TheMadExile's corrections.)

Comments

Some notes about save object manipulation:

1. You don't need to manually JSON encode the data, that happens automatically. You're just bloating the serializations by doing so.
2. That's really not how the metadata property is intended to be used. You'd likely be better off adding a custom property to the save object (e.g. you could name it custom or extra).

---

Best answer.  Thank you. I will need to rethink how I code my game I guess, but you made some really interesting points that a beginner like me should take note of when it comes to coding.  Thanks a bunch.

Answer 2

There's no theoretical limit to how large the game state can become in SugarCube 2, but there are of course practical limitation. What's "unacceptable" in terms of performance can only really be answered by yourself, though. As a general rule, I have a few guidelines I like to follow:

1. Unless it's some specific kind of game where arbitrarily going back in history without explicitly loading a save file is a game mechanic, this goes into the initialisation of the game.

Config.history.maxStates = 1;

2. Things that aren't variable game state aren't stored in state variables (the ones starting with the dollar sign). Most of the rest lands in the "setup" object and the games make heavy use of the flyweight pattern and procedural generation coupled with non-persistent memoization of the results.

3. Game logic is done in JavaScript.

 

Comments

Yeap.  I've already set my max states to 1 as I said in my question.  Never heard of the flyweight pattern technique.  Though this will probably complicate my game even further, I feel this is very useful information.  Thanks.

Answer 3

There are no numbers because the actual limit where you'll see issues depends on device and browser, among other things. Generally, you want to be as smart as possible about what needs to be in story variables and what doesn't. If information is largely constant, or just doesn't need to be saved, use temporary or JavaScript (setup object, probably) variables.

Other than that, you can limit the number of moments / states the user can move backward to also improve performance. Turning this all the way down to one moment will mean the player can't undo/rewind at all, but will maximize performance.

Other than that, write code as smart as you can and optimize as problems arise.

Procedurally generated dungeons shouldn't really ever be stateful. Maybe the one the player is currently in. Even then, you can choose which data you need to save. For example, if you're pulling data from arrays to generate stuff, you could probably save the array index in a story variable rather than a string or object, significantly cutting down on what needs to be tracked.

Note about using JavaScript for game logic: 90% of the time the performance issues are going to come from a bloated state and history system. If only specific passages are taking a while to load, specifically ones with a lot of macro code, then you can get a little bit better performance out of pure JS. If the slowdown is consistent, or gets worse as the game goes on, it's the state.

In either case, it's generally more operator error (poorly written code) than that your ideas are just too big for Twine, though the latter can potentially happen.

Comments

Duly noted.  Currently i have a 100x100 array of zeroes (I haven't yet punched in the random dungeon algorithm) and I've stored it in a story variable "$map".  This alone has had an effect on passage load times (even with my max state history being set to 1).  Currently I want the player to be able to go back to a lower floor in a dungeon, and I'm planning to have 99 floor dungeons...

Perhaps I need to rethink my code.

---

Well, instead of having a 100x100 array, you could just have a 10,000 character string, and then get or set the values by putting this in your JavaScript section:

// getMapVal: Returns the character in the $map string at position (X, Y), or throws an error if position (X, Y) is outside of the map.
window.getMapVal = function (X, Y) {
	if ((parseInt(X) >= 0) && (parseInt(Y) >= 0)) {
		var pos = parseInt(X) + (100 * parseInt(Y));
		var map = State.variables.map;
		if (pos < map.length) {
			return map.substring(pos, pos+1);
		} else {
			throw new Error('getMapVal location outside of map.  Position ' + pos + ' >= $map.length (' + map.length + ')');
		}
	} else {
		throw new Error('getMapVal invalid X (' + X + ') or Y (' + Y + ') value.  Values should be numbers >= 0.');
	}
};

// setMapVal: Sets the character at position (X, Y) to Val, or throws an error if position (X, Y) is outside of the map or if Val is undefined.
// This function should not be used to create $map, it's just for modifying it.
window.setMapVal = function (X, Y, Val) {
	if ((parseInt(X) >= 0) && (parseInt(Y) >= 0)) {
		var pos = parseInt(X) + (100 * parseInt(Y));
		var map = State.variables.map;
		if (pos < map.length) {
			if ((Val != null) && (Val != undefined) && (Val != "")) {
				Val = String(Val);
				State.variables.map = map.substr(0, pos) + Val + map.substr(pos + Val.length);
				return true;
			} else {
				throw new Error('setMapVal failed.  Val (' + Val + ') parameter cannot be null, undefined, or an empty string.');
			}
		} else {
			throw new Error('setMapVal location outside of map.  Position ' + pos + ' >= $map.length (' + map.length + ')');
		}
	} else {
		throw new Error('setMapVal invalid X (' + X + ') or Y (' + Y + ') value.  Values should be numbers >= 0.');
	}
};

You can invoke those functions in Twine like this:

<<run setMapVal(10, 0, "@")>>
<<set _val = getMapVal(10, 0)>>

You'll still need to create $map by creating a 10,000 character string once, but this way means that the system will only have to copy one variable in the history for the map, instead of 10,000 variables, so I believe that should be a lot faster.

Hope that helps!