[BUG] Externally-modified file overwritten

[hbarel]

Actual behaviour

Starting one of the recent updates, when a note is modified externally, often but not always, instead of having it reloaded by the running application, the older version is re-saved, overwriting the externally modified note. Essentially, the modification is not detected, and the next auto-save stores the old version.

Steps to reproduce

Steps to reproduce the behavior:

1. Have the application open on a note file.
2. Edit that note file using another application. (In my case it was rclone syncing with a NextCloud instance, where the note was edited.)
3. Notice that in some cases the modification is not detected, and instead of the new contents being loaded, the old contents are being saved after up to as many seconds as auto-save prescribes.

Relevant log output in the Log panel

...
[18:30:26] [debug] storeCurrentLayout
[18:33:50] [debug] storing note file: "aaa.md"
[18:33:50] [debug] stored note: Note: <id>429 <name>"aaa" <fileName>"aaa.md" <noteSubFolderId>4 <relativePath>"xxx" <hasDirtyData>false
[18:33:50] [debug] storeUpdatedNotesToDisk - 'count': 1
[18:33:50] [status] 💾 Stored 1 note(s) to disk
[18:33:50] [debug] storeUpdatedNotesToDisk - 'wasStripped'
...

The note was not edited in QOwnNotes, and the stored version overwrote a new version that was stored by rclone.

[hbarel]

UPDATE

I identified that this results from the way several sync agents modify files. I am not sure why it started happening with a recent version of QOwnNotes.

A workaround is to configure the sync agent to modify the file directly rather than through temporary files (e.g., the --inplace option in rclone).

I'm not sure if QOwnNotes authors would like to also address this within QOwnNotes, but in any case I think it's up to QOwnNotes to somehow not overwrite an externally-modified file which was not modified in QOwnNotes, even if the external modification was not detected.

[pbek]

Sounds like a setting you are using...
Can you please post the output from the Debug settings in the settings dialog?

[hbarel]

It's pretty lengthy, anything I should focus on?

I'd claim that whatever the settings are that may make an external change go undetected, the program should anyway refuse to re-save a file that has not been modified in it, for the case that it might have been edited elsewhere in the meantime. The user may have even selected to not be notified of external edits at all, and would still not like the content to be overwritten by QOwnNotes when it was not edited in QOwnNotes... (If the user edited the note in two editors -- his problem. :-) )

[pbek]

It's really hard to help you when you don't fill out the issue template. ️🤷🏻

[hbarel]

Sorry for the delay. The settings required some anonymization.
I get: when trying to upload. Any recommended delivery method? (Pasting all here would be ugly...)
I am using the in-note-text-tagging script; I will try disabling it, although not sure how this can help.
Problem is that external changes are sometimes not detected, and the file is overwritten by the old version upon the next auto-save.

[pbek]

(Pasting all here would be ugly...)

That's why it is pasted in a detail block in the issue template. But you can paste it as it is, I made it Markdown.

[pbek]

Problem is that external changes are sometimes not detected, and the file is overwritten by the old version upon the next auto-save.

When under Linux there are too little inotify handler available to listen to all changes this could happen.
Or when a sync tool or editor doesn't write into the file, but rather creates a temporary file and moves the new file over the old or deletes the old file and creates a new one...

[hbarel]

Problem is that external changes are sometimes not detected, and the file is overwritten by the old version upon the next auto-save.

When under Linux there are too little inotify handler available to listen to all changes this could happen. Or when a sync tool or editor doesn't write into the file, but rather creates a temporary file and moves the new file over the old or deletes the old file and creates a new one...

True. I added --inplace to the rclone config so it edits the file in-place. This reduced the frequency of missed external changes, but it did not prevent it completely, which supports your explanation.

I would argue, however, that even if an external edit is missed, QOwnNotes shall never overwrite a file with what is has in memory if that file in memory has not been modified. Saving an unchanged file gives nothing and may overwrite an undetected change...

[pbek]

There is something like

[hbarel]

I already tried this, but admittedly I tried it only before the --inplace addition to rclone. Will try again; nothing to lose.

Consider, however, that the note is not "unsaved". It is not modified in QOwnNotes at all.

[pbek]

The filesystem watcher will catch the filechange or at least the changes in note folder (and then reload it). And the checksum check is the last resort. But I don't know anything about your system. 🤷🏻

[hbarel]

Understood, thanks. Detection cannot be expected to be perfect.

But regardless of whether a change is detected or not, or of any other setting, how could it happen that QownNotes writes to disk a file that hasn't been edited in it?

[pbek]

how could it happen that QownNotes writes to disk a file that hasn't been edited in it?

There is a "dirty" flag in the in-memory database that will be set when the note is changed when something edits the notes in the app. Notes are only written when that flag is set. But since you never shared any context asked for in the issue template I have no clue what you are doing. 🤷🏻