Compilation is slow after updating to 4.4.0

[patricklafrance]

Hey!

After updating to retypeapp version 4.4.0, compilation is now a lot slower when starting retype and when updating the docs.

retype start

  INPUT: C:\XYZ\workleap\wl-squide\docs
 OUTPUT: [in-memory] 
Retype finished in 14.5 seconds!
  127 pages built
    0 errors
    0 warnings

Content change detected...

Retype finished in 11.5 seconds!
  1 page updated
  1 page built
  0 errors
  0 warnings

View at http://localhost:5001/wl-squide/
Press Ctrl+C to shut down

Content change detected...

Retype finished in 13.1 seconds!
  1 page built
  0 errors
  0 warnings

View at http://localhost:5001/wl-squide/
Press Ctrl+C to shut down

Content change detected...

Retype finished in 12.6 seconds!
  1 page updated
  1 page built
  0 errors
  0 warnings

View at http://localhost:5001/wl-squide/
Press Ctrl+C to shut down

I am on Windows 11.

Thanks,

Patrick

[geoffreymcgill]

Retype finished in 14.5 seconds!

Do you know approximately what the retype start or rebuild times were in the previous release?

I think the lastUpdated rendering is causing this performance issue. We have done some work in v4.5.0 to improve the performance.

Can you re-run your tests with the following added to your retype.yml file?

lastUpdated:
  date:
    enabled: false

That will disable the Last update label rendering.

[patricklafrance]

It always been too fast for even noticing that it recompiled 😅

A lot better with last updated disabled:

retype start

  INPUT: C:\XYZ\workleap\wl-squide\docs
 OUTPUT: [in-memory] 
Retype finished in 4.6 seconds!
  128 pages built
    0 errors
    0 warnings

Content change detected...

Retype finished in 1.7 seconds!
  1 page updated
  3 pages built
  0 errors
  0 warnings

[geoffreymcgill]

I was leaving enabled: false set locally and then letting it run with enabled: true on GitHub as the project build time isn't as noticeable during the gh build process.

I'm seeing some of your live site rebuilds are in the 6.2 seconds range. Looks like with the prior release they were in the 5.3 second range.

Retype finished in 6.2 seconds!

The Last updated functionality is a performance hog and requires Retype to query the .git repo to fetch the last modified date values.

We definitely improved the efficiency in v4.5.0, so it will be interesting to see the build times with the new release, with also re-enabling the lastUpdate config.

[patricklafrance]

Startup time with 4.5.2 is still quite slow:

retype start

  INPUT: C:\Dev\workleap\wl-web-configs\docs
 OUTPUT: [in-memory] 
Retype finished in 7.6 seconds!
  58 pages built
   0 errors
   0 warnings

Used to be instant locally.

For now, I will disable the lastUpdate feature!

Thank you,

Patrick

[patricklafrance]

As a side note... when downgrading to 4.1.0 for the icons rendering issue, Retype start was completing in 1.1 second. When going back to 4.5.3, even with:

lastUpdated:
  date:
    enabled: false
  by:
    enabled: false

It's about 4 seconds.

[geoffreymcgill]

As a side note... when downgrading to 4.1.0 for the icons rendering issue, Retype start was completing in 1.1 second. When going back to 4.5.3, even with:

Perfect. Thank you for those numbers. We still have some work to do on the performance. There has been some big features added in the last few releases and they are obviously costing more of a performance hit than our original models and testing predicted. We need to do better.

[patricklafrance]

On our biggest docs site, even with last updated disabled, we are at 4.3 seconds for an update:

Retype finished in 4.3 seconds!
  1 page updated
  1 page built
  0 errors
  0 warnings

View at http://localhost:5001/wl-squide/
Press Ctrl+C to shut down

[geoffreymcgill]

There are measurable performance improvements coming with v4.6.0. Here's a brief summary:

lastUpdated not enabled

We are setting lastUpdated to be NOT enabled by default. So by default, the feature will be disabled. From our testing, we found that basically 1/2 of the build time was coming from this single feature. We tried a few ways to work-around the limitation of querying the local git data, but its just too much of a performance bottleneck for now.

We have some ideas more ideas to explore, but for now, that feature is not being enabled by default.

The lastUpdated functionality is still available, but you'll need to explicitly enable using the following retype.yml settings:

lastUpdate:
  date:
    enabled: true

The lastUpdated is a super useful feature to have, but for all our projects, we will just enable it when using retype build and not have it enabled if authoring locally when running retype start. For retype.com, it will be enabled on the live site being hosted on GitHub Pages with GitHub Actions doing the build, but not-enabled when running the project on our local machine with retype start.

Faster builds

Improvements have been made to the build performance.

I compared building your wl-squide project with v4.5 vs v4.6 and had the following results:

Important

These are numbers running on my machine. Your results may differ, but the ratios should be similar.

Scenario	v4.5	v4.6	Diff
Full build with retype start	4.8 sec	4.0 sec	1.2x faster
Change detected	1.8 sec	0.6 sec	3.0x faster

Adding the new sanitized .md file for every page does come with some cost. Essentially every page in the project has two hard files created; one .html and one .md. Those file write operations are relatively expensive and add up for bigger projects. We're working on ideas to improve the performance of the sanitized .md file generation.

Full console results below:

Retype 4.5

Full build with retype start

Retype finished in 4.8 seconds!
  129 pages built
    0 errors

Change detected

Retype finished in 1.8 seconds!
  1 page updated
  1 page built
  0 errors

Retype 4.6

Full build with retype start

Retype finished in 4.0 seconds!
  129 pages built
    0 errors

Change detected

Retype finished in 0.6 seconds!
  1 page updated
  1 page built
  0 errors

Other performance tips

If you are not using any of the templating functionality, disable it using the following retype.yml configuration:

templating:
  enabled: false

[patricklafrance]

Hey @geoffreymcgill,

I bumped from 4.5.2 to 4.5.3 and can now get performant results that are similar to yours for an update with lastUpdated and templating disabled. For the a start, it's in the 6 seconds range for me.

Here's my setup info:

  System:
    OS: Windows 11 10.0.26200
    CPU: (24) x64 AMD Ryzen AI 9 HX PRO 370 w/ Radeon 890M       
    Memory: 17.12 GB / 55.65 GB

Thanks,

Patrick