Howdy, Stranger!
It looks like you're new here. If you want to get involved, click one of these buttons!
Improving performance in a complex Harlowe game
So, as explained in this thread, my game has very bad performance problems and I have no idea of how to debug and fix it. This is a summary of the way my game works. Sorry it's so long. I'd be incredibly grateful for any help (in fact, this complicated game exists thanks to all the help I've received here over the last year).
What happens when you enter a passage
My game has an engine, which builds each turn starting with a base passage and adding bits from a lot of places. Each passage runs more than 160 lines of code, sometimes a lot more.
Each passage is a location. Choices can add text blocks to any location, to be printed the next time you enter it. This is recorded like this: there is a $narration datamap, whose names are all the locations in the game (base number is 25).
The values of this datamap are nested arrays with the text to be printed. But this would make the history gigantic and probably exhaust localStorage and break savegames before the game ends. So, the arrays include passage names that are displayed with a macro.
So, when you enter a passage, the engine:
* Gets the value for the passage name from the 25 items datamap.
* Checks the array is longer than zero.
* Loops through every value in the array and feeds it to a (display:) macro. (I'm using my patented loop method.)
And then it repeats this between one and three other times. Why? Because:
a) Locations are grouped in regions, and the writer can define text for a region. So, for example, if you have a neighbourhood with several locations, and there's a fire, you can add text about smelling smoke to all the region, instead of each location.
b) NPCs move between rooms, and they can also have text attached to be printed in the same way. Any number of NPCs can be in any room at the same time.
Regions and NPCs are also items in the $narration datamap, so it's almost 40 items now (probably more when I finish writing).
So, in summary, the game loops through the multiple concepts (room, region and NPCs) to print the bits of text extracted from a 40 item datamap.
This is only to print the narration and the description. Then comes the other half: printing the choices.
The mechanism is the same: arrays nested inside a big $choices datamap, loops, repeated for the room and the NPCs (not the regions).
After all this and some more secondary mechanisms (increasing the turn counter, checking for scheduled events, etc.), your turn is on the screen to play.
What happens when you click on a choice
Each choice can do four main things when you click:
* adding one or several values to an array nested in the $choice datamap
* removing one or several values from an array nested in the $choice datamap
* adding one or several values to an array nested in the $narration datamap
* removing one or several values from an array nested in the $narration datamap
So, when you click a choice, those two datamaps are written a number of times (I'd say an average of three times) and then you automatically (goto:) the destination passage.
So, the total delay between clicking and getting the new screen is the sum of whatever is run when you click, and the passage generation.
In my current machine, with the contents I have, this takes an average of 4-5 seconds. In mobile it's totally unplayable.
Options
* Porting to Snowman: not feasible. I'm not a programmer and Javascript is way above me.
* Porting to Sugarcube: possible. But I'd need to find a replacement for hooks (guess it's easy) and datamaps.
* Reducing the size of datamaps in Harlowe: if reading/writing a 60 item datamap is noticeably slower than a 40 item datamap, I could try to streamline it somehow. I think I took a performance hit recently when I doubled the size of the datamaps for a special mechanism that could maybe be achieved with a less elegant but faster way.
* ??? I'm out of other ideas.
If you have read all this, thanks a lot!
What happens when you enter a passage
My game has an engine, which builds each turn starting with a base passage and adding bits from a lot of places. Each passage runs more than 160 lines of code, sometimes a lot more.
Each passage is a location. Choices can add text blocks to any location, to be printed the next time you enter it. This is recorded like this: there is a $narration datamap, whose names are all the locations in the game (base number is 25).
(datamap: "room 1", "", "room 2", "", ... , "room 25", "")
The values of this datamap are nested arrays with the text to be printed. But this would make the history gigantic and probably exhaust localStorage and break savegames before the game ends. So, the arrays include passage names that are displayed with a macro.
"room 1", " 'room1textA' , 'room1textB' "
So, when you enter a passage, the engine:
* Gets the value for the passage name from the 25 items datamap.
* Checks the array is longer than zero.
* Loops through every value in the array and feeds it to a (display:) macro. (I'm using my patented loop method.)
And then it repeats this between one and three other times. Why? Because:
a) Locations are grouped in regions, and the writer can define text for a region. So, for example, if you have a neighbourhood with several locations, and there's a fire, you can add text about smelling smoke to all the region, instead of each location.
b) NPCs move between rooms, and they can also have text attached to be printed in the same way. Any number of NPCs can be in any room at the same time.
Regions and NPCs are also items in the $narration datamap, so it's almost 40 items now (probably more when I finish writing).
So, in summary, the game loops through the multiple concepts (room, region and NPCs) to print the bits of text extracted from a 40 item datamap.
This is only to print the narration and the description. Then comes the other half: printing the choices.
The mechanism is the same: arrays nested inside a big $choices datamap, loops, repeated for the room and the NPCs (not the regions).
After all this and some more secondary mechanisms (increasing the turn counter, checking for scheduled events, etc.), your turn is on the screen to play.
What happens when you click on a choice
Each choice can do four main things when you click:
* adding one or several values to an array nested in the $choice datamap
* removing one or several values from an array nested in the $choice datamap
* adding one or several values to an array nested in the $narration datamap
* removing one or several values from an array nested in the $narration datamap
So, when you click a choice, those two datamaps are written a number of times (I'd say an average of three times) and then you automatically (goto:) the destination passage.
So, the total delay between clicking and getting the new screen is the sum of whatever is run when you click, and the passage generation.
In my current machine, with the contents I have, this takes an average of 4-5 seconds. In mobile it's totally unplayable.
Options
* Porting to Snowman: not feasible. I'm not a programmer and Javascript is way above me.
* Porting to Sugarcube: possible. But I'd need to find a replacement for hooks (guess it's easy) and datamaps.
* Reducing the size of datamaps in Harlowe: if reading/writing a 60 item datamap is noticeably slower than a 40 item datamap, I could try to streamline it somehow. I think I took a performance hit recently when I doubled the size of the datamaps for a special mechanism that could maybe be achieved with a less elegant but faster way.
* ??? I'm out of other ideas.
If you have read all this, thanks a lot!
Comments
First. Let me say that, while it's difficult to be certain, your "engine" seems overly complicated solely based upon your description. That could be a significant source of your processing delay right there. Again, I cannot be certain—maybe it simply sounds complicated.
Beyond that, one of the causes of your project's slowness might be because of the way Harlowe's variable store works or, more specifically, the follow-on effects upon modifications to reference types*. Whenever you modify a reference type in Harlowe, it must create a new copy of the reference type.
As an example, if you have a datamap which you created in a passage and you add an entry to it in a following passage: The addition of the new entry forces the creation of a new datamap which will contain the contents of both of the old ones.
That's not bad in and of itself, all story formats make copies of the contents of their variable stores at some point, however, the issue is how often this happens. I'm unsure whether Harlowe does this every single time you do a modification or only the first time within a moment. If it is the former, then that is, potentially, a whole lot of copying.
Contrast this with other story formats, which by and large use a clone-at-navigation style variable store, as opposed to Harlowe's prototypal one. Using the same example scenario given above, the normal thing to do would be to simply make whatever modifications you needed to the existing map, which would only get cloned/copied once during passage navigation.
* By "reference types" I mean any data type within Harlowe that is backed by a JavaScript reference type—e.g. (datamap:), (dataset:), (array:).
I take note of your suggestions. I'm going to do an effort to test systematically every solution I can find, and your ideas point some interesting directions.