An example of the contents of your Passage and how you are implementing your input related functionality would help us know exactly what your Passage is doing.
The visual effect portion of Harlowe's passage transition process includes the following two steps:
1. Wrapping the new passage's HTML elements within a tw-transition-container element which is then added to the DOM of the current page. (Start of passage transition visual effect.)
2. Removing the tw-transition-container element from the DOM of the current page and then re-adding it's child elements directly to the DOM. (End of passage transition visual effect.)
The removal and then re-adding of your input related elements may be causing the side effect you are seeing.