Biggest performance impact(s) in Harlowe?

Question

Hi all, I have an open-ended question. I've been writing a fairly complex (I think) Twine game in Harlowe 1.2.4. It's run like a dream until recently - I've been adding tons of passages, and a *lot* of non-temporary variables. 

Recently, my story has been running a bit slow. When I click a button to go to the next passage (often in the format of (click:?hook)[(goto:"passage")], the ensuing passage will take a bit of time, 1-2 seconds, to load. This only started happening recently. 

I just wondered if some of the more experienced people here could talk about the practices with the biggest negative performance impacts while using Harlowe, and how to avoid them. My biggest specific questions: 

• Is it worth it to use temporary variables wherever possible? Is the performance benefit very significant? 
• Is it better to have larger passages, and fewer of them? 
• What is the most efficient way to go from passage to passage? 
• Any other Harlowe tricks? Should I upgrade to 2.x, would that provide a significant boost in any sense? 

Thanks so much for your thoughts :) 

 

Lily

Answer 1

Note. These answers are by no means definitive, and are all gleaned from my limited understanding of the situation.

1. About story vs temporary variables.

Generally, the issue you encounter with variables is related to the state history. Without going too far into the weeds, basically, using variables unwisely can cause slowdown as the game goes on in longer games. It is unlikely that the amount of information you're tracking could ever lead to noticeable slowdown, the problem lies in the fact that a copy of the entire state (including variables, passage names, etc) is passed to each new moment in the history. This has the effect of multiplying the number of variables being tracked.

So, in short, it's a best practice to use temporary variables when possible. Liberal and unnecessary use of story variables is always going to bog down your story on some level, but it's important to point out that this takes a lot.

2. About passage length vs number.

As far as I can tell, not really. The amount of passages may effect initial load times and some early transitions. I wouldn't worry about this. It could cut down on moments as mentioned above, which would indirectly help with any variable issues, but it's better to probably attack that problem from the variable side.

3. About links.

Generally, the standard double-square-bracket link. Using the (click:) macro forces the page to be reparsed, from my understanding. It is, quite literally, the slowest and most expensive possible way to create a link of all the available options. I wouldn't use it as a general replacement for normal links, which is what you seem to be doing.

4. About Harlowe 2.

Generally, using the most recent major version of anything is best. There are quite likely a number of optimizations and improvements that have and will come to 2.x that you'll never see. I would be surprised if 1.x received anything but major bug fixes and security fixes. Any new and exciting ways to improve performance will probably not make it to version 1. If possible, you should migrate over.

Comments

RE: About Links.

The following example shows four different methods to create a link to a target passage named Next.

Some text with a [[markup-based link->Next]]

Some text with a (link: "macro-based link")[(go-to: "Next")] in it.

Some text with a [named hook]<thehook| in it.
(click: ?thehook)[(go-to: "Next")]

Some text with an unique phrase in it.
(click: "unique phrase")[(go-to: "Next")]

The following describes the effort the Harlowe engine needs to go through to product the clickable link. For the sake of simplicity the following information is not 100% correct technically.

Harlowe generally starts at the top of a passage's content and processes/executes each 'line' of TwineScript as it's read, then adds the HTML elements required for that TwineScript to a temporary HTML structure and once all of the passage content has been processed any delay effects (like click macros) are resolved before the final temporary structure is rendered as part of the existing page.

A. The markup based link example.
The markup is processed as it is encounter within the passage contents and produces HTML elements like the following:

Some text with a
<tw-expression type="macro" name="link-goto">
	<tw-link tabindex="0" passage-name="Next" data-raw="">markup-based link</tw-link>
</tw-expression>
in it.

 

B. The (link:) macro based link example.
The macro is processed as it is encounter within the passage contents and produces HTML elements like the following:

Some text with a
<tw-expression type="macro" name="link"></tw-expression>
<tw-hook>
	<tw-link tabindex="0" data-raw="">macro-based link</tw-link>
</tw-hook>
in it.

 

C. The (click:) macro with named hook link example.
The TwineScript related to this example is processed in a number of steps and basically produces the following HTML for each one:

1. After the text and named hook is processed but before the (click:) macro is processed.

Some text with a
<tw-hook name="thehook">named hook</tw-hook>
in it.
<tw-expression type="macro" name="click" class="false"></tw-expression>
<tw-hook></tw-hook>

2. After the (click:) macro is processed but before the whole of temporary structure is scanned to locate the related name hook so that the related click handler can be attached.

Some text with a
<tw-hook name="thehook">named hook</tw-hook>
in it.
<tw-expression type="macro" name="click" class="false"></tw-expression>
<tw-hook></tw-hook>

3. After the scanning of the whole structure and the attaching of the click handler.

Some text with a
<tw-enchantment class="link enchantment-link" tabindex="0">
	<tw-hook name="thehook">named hook</tw-hook>
</tw-enchantment>
in it.
<tw-expression type="macro" name="click" class="false"></tw-expression>
<tw-hook></tw-hook>

... you will notice that a tw-enchantment element has been inserted into the structure and that this element wraps the tw-hook element from step 2.

D. The (click:) macro with an unique phrase link example.
The TwineScript related to this example is processed in a number of steps and basically produces the following HTML for each one:

1. After the text is processed but before the (click:) macro is processed.

Some text with an unique phrase in it.

2. After the (click:) macro is processed but before the whole of temporary structure is scanned to locate the unique phrase so that the related click handler can be attached.

Some text with an unque phrase in it.
<tw-expression type="macro" name="click" class="false"></tw-expression>
<tw-hook></tw-hook>

3. After the scanning of the whole structure and the attaching of the click handler.

Some text with an 
<tw-enchantment class="link enchantment-link" tabindex="0">unque phrase</tw-enchantment>
in it.
<tw-expression type="macro" name="click" class="false"></tw-expression>
<tw-hook></tw-hook>

... you will notice that a tw-enchantment element has been inserted into the structure and that this element wraps the tw-hook element from step 2.

In the case of a (click:) based macro link the scanning of the whole structure for the relevant item needs to be done for each and every (click:) macro within the Passage content, this means the the whole of the structure can be scanned multiple times.

This is why it is generally better to use either markup-based links or (link:) macro based links whenever possible.

NOTE: There are situations (related to post-render structure changes) which can cause the scanning required by the attaching of click handlers to be done again, which in turn can interfere with the end-users ability to interact with the page.

---

Would it help Lily to put

Config.history.maxStates = 1;

in her Story JavaScript?

---

Would it help Lily to put

Config.history.maxStates = 1;

in her Story JavaScript?

Unfortunately, no. That's a SugarCube-only feature. It will potentially correct similar problems in SugarCube if you don't need the history system. 

---

As I understand** it SugarCube's Config.history.maxStates setting only controls how many moments are stored in the History system (and thus a Save), it does not stop the cloning of the variable state during passage transition.

** It has been quite a while since I visited this subject or read the related source code, so my memory could be faulty or the code may of changed.

---

Thanks for your fantastic responses! I'm going to switch to those direct links wherever possible.

I also believe some of my performance issues are also with the (display:) macro. What I often do is have passages that contain "synonyms" for other words, and (display:) them in other passages to provide more variety in commonly-used text.

Do you think it's a potential problem in a Project with a growing number of passages (400+), if I use (display:) too much? Is there any way to partition passages so that I can only search through a group?

---

I also believe some of my performance issues are also with the (display:) macro. 

I would do one thing at a time if I were you. There's no real need to go farther than necessary. If changing the click macros fixes the issue, it's entirely likely that that's all you'll need to do. If that doesn't work, I think cleaning up variables is likely a better use of your time. Passage data is just stored as text, and getting it and putting it on the page is not super taxing. It's parsing it, if anything, that's taxing. Using a display, therefore, is more about the code you're displaying; you're only saving yourself one macro call per instance of the macro, and that's unlikely to be the issue, as far as I understand. And searching through 400 things for something is not all that hard on a computer. It's not like it's rendering every passage on the way or anything. 

Is there any way to partition passages so that I can only search through a group?

Not other than tagging, as far as I'm aware.